Research Article
BibTex RIS Cite

Microsoft SQL Server’da Büyük Veri Yönetiminin Optimizasyonu: Normalizasyon ve İleri Analitik Teknikler ile Performansının Artırılması

Year 2025, Volume: 9 Issue: 1, 23 - 36, 30.06.2025

Süleyman Burak Altinişik , Turgay Tugay Bilgin

https://doi.org/10.46460/ijiea.1563777

Abstract

Bu çalışma, kablo üretim sektöründe Microsoft SQL Server (MSSQL) kullanarak Büyük Veri yönetimi zorluklarını ve çözümlerini incelemekte, performans optimizasyonu, normalizasyon ve ileri düzey analitik tekniklere odaklanmaktadır. Büyük Veri'nin 4V'sini ele alan örnek olay incelemesinde, 45 TAG'dan bir dakikalık aralıklarla veri toplanmakta ve günlük yaklaşık 56 milyon kayıt oluşturulmaktadır. Veri toplamak için OPC teknolojisini, stratejik normalizasyon süreçlerini ve ileri düzey MSSQL optimizasyon tekniklerini kullanmaktayız. Normalizasyon, veri tekrarını önemli ölçüde azaltmış, veri setini günde 56 milyondan 283 satıra düşürmüş ve karmaşık analitik sorgular için sorgu yürütme sürelerini 40 dakikadan 0.1 saniyenin altına indirmiştir. Ayrıca, maliyet ve performans dengesini sağlamak için veritabanından bağımsız yazılım geliştirme yaklaşımı önermekteyiz. Bu araştırma, endüstriyel ortamlarda büyük ölçekli veri işleme zorluklarıyla karşılaşan organizasyonlar için performans optimizasyonu, ölçeklenebilirlik ve maliyet etkin çözümler konusunda pratik bilgiler sunmakta, teknik performans ile ekonomik hususlar arasında denge kuran etkili bir Büyük Veri yönetimi için bir yol haritası sunmaktadır.

Keywords

Büyük Veri Yönetimi Veri Normalizasyonu Kablo Üretimi Performans Optimizasyonu

References

  • Malik, P. K., Sharma, R., Singh, R., Gehlot, A., Satapathy, S. C., Alnumay, W. S., Pelusi, D., Ghosh, U., & Nayak, J. (2021). Industrial internet of things and its applications in industry 4.0: State of the art. Computer Communications, 166, 125–139.
  • Ghasemaghaei, M. (2021). Understanding the impact of big data on firm performance: The necessity of conceptually differentiating among big data characteristics. International Journal of Information Management, 57, 102055
  • Fan, C., Yan, D., Xiao, F., Li, A., An, J., & Kang, X. (2021). Advanced data analytics for enhancing building performances: From data-driven to big data-driven approaches. Building Simulation, 14(1), 3–24.
  • Naeem, M., Jamal, T., Diaz-Martinez, J., Butt, S. A., Montesano, N., Tariq, M. I., De-la Hoz-Franco, E., & De-La-Hoz-Valdiris, E. (2022). Trends and future perspective challenges in big data. In Advances in Intelligent Data Analysis and Applications (pp. 309–325). Springer.
  • Ranjan, J., & Foropon, C. (2021). Big data analytics in building the competitive intelligence of organizations. International Journal of Information Management, 56, 102231.
  • Larrea, M. L., & Urribarri, D. K. (2021). Visualization technique for comparison of time-based large data sets. In Conference on Cloud Computing, Big Data & Emerging Topics (pp. 179–187). Springer.
  • Dinneen, J. D., & Brauner, C. (2017). Information-not-thing: Further problems with and alternatives to the belief that information is physical.
  • Vaitis, M., Feidas, H., Symeonidis, P., Kopsachilis, V., Dalaperas, D., Koukourouvli, N., Simos, D., & Taskaris, S. (2019). Development of a spatial database and web-GIS for the climate of Greece. Earth Science Informatics, 12(1), 97–115.
  • Amin, M., Romney, G. W., Dey, P., & Sinha, B. (2019). Teaching relational database normalization in an innovative way. Journal of Computing Sciences in Colleges, 35(2), 48–56.
  • Alqithami, S. (2021). A serious-gamification blueprint towards a normalized attention. Brain Informatics, 8(1), 1–13.
  • Oditis, I., Bicevska, Z., Bicevskis, J., & Karnitis, G. (2018). Implementation of NoSQL-based data warehouse. Baltic Journal of Modern Computing, 6(1), 45–55.
  • Hrubaru, I., Talabă, G., & Fotache, M. (2019). A basic testbed for JSON data processing in SQL data servers. In Proceedings of the 20th International Conference on Computer Systems and Technologies (pp. 278–283).
  • Chung, Y. G., Haldoupis, E., Bucior, B. J., Haranczyk, M., Lee, S., Zhang, H., Vogiatzis, K. D., Milisavljevic, M., Ling, S., Camp, J. S., et al. (2019). Advances, updates, and analytics for the computation-ready, experimental metal–organic framework database: Core MOF 2019. Journal of Chemical & Engineering Data, 64(12), 5985–5998.
  • Bouros, P., & Mamoulis, N. (2019). Spatial joins: What’s next? SIGSPATIAL Special, 11(1), 13–21.
  • Myalapalli, V. K., Totakura, T. P., & Geloth, S. (2015). Augmenting database performance via SQL tuning. In 2015 International Conference on Energy Systems and Applications (pp. 13–18). IEEE.
  • Pedrozo, W. G., & Vaz, M. S. M. G. (2014). A tool for automatic index selection in database management systems. In 2014 International Symposium on Computer, Consumer and Control (pp. 1061–1064). IEEE.
  • Correia, J., Santos, M. Y., Costa, C., & Andrade, C. (2018). Fast online analytical processing for big data warehousing. In 2018 International Conference on Intelligent Systems (IS) (pp. 435–442). IEEE.
  • Sulistiani, H., Setiawansyah, S., & Darwis, D. (2020). Penerapan metode agile untuk pengembangan online analytical processing (OLAP) pada data penjualan (Studi kasus: CV Adilia Lestari). Jurnal CoreIT: Jurnal Hasil Penelitian Ilmu Komputer dan Teknologi Informasi, 6(1), 50–56.
  • Erdinç, H. N., Buluş, N., & Erdoğan, C. (2018). Veritabanı tasarımının yazılım performansına etkisi: Normalizasyona karşı denormalizasyon. Süleyman Demirel Üniversitesi Fen Bilimleri Enstitüsü Dergisi, 22(2), 887–895.
  • Alshemaimri, B., Elmasri, R., Alsahfi, T., & Almotairi, M. (2021). A survey of problematic database code fragments in software systems. Engineering Reports, 3(10), e12441.
  • Milicev, D. (2021). Hyper-relations: A model for denormalization of transactional relational databases. IEEE Transactions on Knowledge and Data Engineering.
  • Chaparro-Cruz, I. N., & Montoya-Zegarra, J. A. (2021). Borde: Boundary and sub-region denormalization for semantic brain image synthesis. In 2021 34th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) (pp. 81–88). IEEE.
  • Costa, R. L. D. C., Moreira, J., Pintor, P., dos Santos, V., & Lifschitz, S. (2021). A survey on data-driven performance tuning for big data analytics platforms. Big Data Research, 25, 100206.
  • Chillón, A. H., Ruiz, D. S., & Molina, J. G. (2021). Towards a taxonomy of schema changes for NoSQL databases: The Orion language. In International Conference on Conceptual Modeling (pp. 176–185). Springer.
  • Gupta, E., Sural, S., Vaidya, J., & Atluri, V. (2021). Attribute-based access control for NoSQL databases. In Proceedings of the Eleventh ACM Conference on Data and Application Security and Privacy (pp. 317–319).
  • Santos, M. Y., Costa, C., Galvão, J., Andrade, C., Martinho, B. A., Lima, F. V., & Costa, E. (2017, July). Evaluating SQL-on-Hadoop for big data warehousing on not-so-good hardware. In Proceedings of the 21st International Database Engineering & Applications Symposium (pp. 242-252).
  • Yang, F., Tschetter, E., Léauté, X., Ray, N., Merlino, G., & Ganguli, D. (2014, June). Druid: A real-time analytical data store. In Proceedings of the 2014 ACM SIGMOD international conference on Management of data (pp. 157-168).
  • Silva, Y. N., Almeida, I., & Queiroz, M. (2016, February). SQL: From traditional databases to big data. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education (pp. 413-418).
  • Rahman, M. M., Islam, S., Kamruzzaman, M., & Joy, Z. H. (2024). Advanced Query Optimization in SQL Databases For Real-Time Big Data Analytics. Academic Journal on Business Administration, Innovation & Sustainability, 4(3), 1-14.
  • Uzzaman, A., Jim, M. M. I., Nishat, N., & Nahar, J. (2024). Optimizing SQL databases for big data workloads: techniques and best practices. Academic Journal on Business Administration, Innovation & Sustainability, 4(3), 15-29.
  • Pirzadeh, P., Carey, M., & Westmann, T. (2017, December). A performance study of big data analytics platforms. In 2017 IEEE international conference on big data (big data) (pp. 2911-2920). IEEE.
  • Panwar, V. (2024). Optimizing Big Data Processing in SQL Server through Advanced Utilization of Stored Procedures. Journal Homepage: http://www. ijmra. us, 14(02).
  • Ordonez, C. (2013, October). Can we analyze big data inside a DBMS?. In Proceedings of the sixteenth international workshop on Data warehousing and OLAP (pp. 85-92)

Optimizing Big Data Management on Microsoft SQL Server: Enhancing Performance through Normalization and Advanced Analytical Techniques

Year 2025, Volume: 9 Issue: 1, 23 - 36, 30.06.2025

Süleyman Burak Altinişik , Turgay Tugay Bilgin

https://doi.org/10.46460/ijiea.1563777

Abstract

This study investigates Big Data management challenges and solutions in cable manufacturing using Microsoft SQL Server (MSSQL), focusing on performance optimization, normalization, and advanced analytical techniques. Addressing the 4Vs of Big Data, our case study collects data from 45 TAGs at one-minute intervals, generating approximately 56 million daily records. We employ OPC technology for data acquisition, strategic normalization processes, and advanced MSSQL optimization techniques. Normalization significantly reduced data redundancy, decreasing the dataset from 56 million to 283 rows per day and improving query execution times from over 40 minutes to less than 0.1 seconds for complex analytical queries. We also propose a database-independent software development approach to balance cost and performance. This research contributes practical insights into performance optimization, scalability, and cost-effective solutions for organizations managing large-scale data processing challenges in industrial settings, offering a blueprint for efficient Big Data management that balances technical performance with economic considerations.

Keywords

Big Data Management Data Normalization Cable Manufacturing Performance Optimization

References

  • Malik, P. K., Sharma, R., Singh, R., Gehlot, A., Satapathy, S. C., Alnumay, W. S., Pelusi, D., Ghosh, U., & Nayak, J. (2021). Industrial internet of things and its applications in industry 4.0: State of the art. Computer Communications, 166, 125–139.
  • Ghasemaghaei, M. (2021). Understanding the impact of big data on firm performance: The necessity of conceptually differentiating among big data characteristics. International Journal of Information Management, 57, 102055
  • Fan, C., Yan, D., Xiao, F., Li, A., An, J., & Kang, X. (2021). Advanced data analytics for enhancing building performances: From data-driven to big data-driven approaches. Building Simulation, 14(1), 3–24.
  • Naeem, M., Jamal, T., Diaz-Martinez, J., Butt, S. A., Montesano, N., Tariq, M. I., De-la Hoz-Franco, E., & De-La-Hoz-Valdiris, E. (2022). Trends and future perspective challenges in big data. In Advances in Intelligent Data Analysis and Applications (pp. 309–325). Springer.
  • Ranjan, J., & Foropon, C. (2021). Big data analytics in building the competitive intelligence of organizations. International Journal of Information Management, 56, 102231.
  • Larrea, M. L., & Urribarri, D. K. (2021). Visualization technique for comparison of time-based large data sets. In Conference on Cloud Computing, Big Data & Emerging Topics (pp. 179–187). Springer.
  • Dinneen, J. D., & Brauner, C. (2017). Information-not-thing: Further problems with and alternatives to the belief that information is physical.
  • Vaitis, M., Feidas, H., Symeonidis, P., Kopsachilis, V., Dalaperas, D., Koukourouvli, N., Simos, D., & Taskaris, S. (2019). Development of a spatial database and web-GIS for the climate of Greece. Earth Science Informatics, 12(1), 97–115.
  • Amin, M., Romney, G. W., Dey, P., & Sinha, B. (2019). Teaching relational database normalization in an innovative way. Journal of Computing Sciences in Colleges, 35(2), 48–56.
  • Alqithami, S. (2021). A serious-gamification blueprint towards a normalized attention. Brain Informatics, 8(1), 1–13.
  • Oditis, I., Bicevska, Z., Bicevskis, J., & Karnitis, G. (2018). Implementation of NoSQL-based data warehouse. Baltic Journal of Modern Computing, 6(1), 45–55.
  • Hrubaru, I., Talabă, G., & Fotache, M. (2019). A basic testbed for JSON data processing in SQL data servers. In Proceedings of the 20th International Conference on Computer Systems and Technologies (pp. 278–283).
  • Chung, Y. G., Haldoupis, E., Bucior, B. J., Haranczyk, M., Lee, S., Zhang, H., Vogiatzis, K. D., Milisavljevic, M., Ling, S., Camp, J. S., et al. (2019). Advances, updates, and analytics for the computation-ready, experimental metal–organic framework database: Core MOF 2019. Journal of Chemical & Engineering Data, 64(12), 5985–5998.
  • Bouros, P., & Mamoulis, N. (2019). Spatial joins: What’s next? SIGSPATIAL Special, 11(1), 13–21.
  • Myalapalli, V. K., Totakura, T. P., & Geloth, S. (2015). Augmenting database performance via SQL tuning. In 2015 International Conference on Energy Systems and Applications (pp. 13–18). IEEE.
  • Pedrozo, W. G., & Vaz, M. S. M. G. (2014). A tool for automatic index selection in database management systems. In 2014 International Symposium on Computer, Consumer and Control (pp. 1061–1064). IEEE.
  • Correia, J., Santos, M. Y., Costa, C., & Andrade, C. (2018). Fast online analytical processing for big data warehousing. In 2018 International Conference on Intelligent Systems (IS) (pp. 435–442). IEEE.
  • Sulistiani, H., Setiawansyah, S., & Darwis, D. (2020). Penerapan metode agile untuk pengembangan online analytical processing (OLAP) pada data penjualan (Studi kasus: CV Adilia Lestari). Jurnal CoreIT: Jurnal Hasil Penelitian Ilmu Komputer dan Teknologi Informasi, 6(1), 50–56.
  • Erdinç, H. N., Buluş, N., & Erdoğan, C. (2018). Veritabanı tasarımının yazılım performansına etkisi: Normalizasyona karşı denormalizasyon. Süleyman Demirel Üniversitesi Fen Bilimleri Enstitüsü Dergisi, 22(2), 887–895.
  • Alshemaimri, B., Elmasri, R., Alsahfi, T., & Almotairi, M. (2021). A survey of problematic database code fragments in software systems. Engineering Reports, 3(10), e12441.
  • Milicev, D. (2021). Hyper-relations: A model for denormalization of transactional relational databases. IEEE Transactions on Knowledge and Data Engineering.
  • Chaparro-Cruz, I. N., & Montoya-Zegarra, J. A. (2021). Borde: Boundary and sub-region denormalization for semantic brain image synthesis. In 2021 34th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) (pp. 81–88). IEEE.
  • Costa, R. L. D. C., Moreira, J., Pintor, P., dos Santos, V., & Lifschitz, S. (2021). A survey on data-driven performance tuning for big data analytics platforms. Big Data Research, 25, 100206.
  • Chillón, A. H., Ruiz, D. S., & Molina, J. G. (2021). Towards a taxonomy of schema changes for NoSQL databases: The Orion language. In International Conference on Conceptual Modeling (pp. 176–185). Springer.
  • Gupta, E., Sural, S., Vaidya, J., & Atluri, V. (2021). Attribute-based access control for NoSQL databases. In Proceedings of the Eleventh ACM Conference on Data and Application Security and Privacy (pp. 317–319).
  • Santos, M. Y., Costa, C., Galvão, J., Andrade, C., Martinho, B. A., Lima, F. V., & Costa, E. (2017, July). Evaluating SQL-on-Hadoop for big data warehousing on not-so-good hardware. In Proceedings of the 21st International Database Engineering & Applications Symposium (pp. 242-252).
  • Yang, F., Tschetter, E., Léauté, X., Ray, N., Merlino, G., & Ganguli, D. (2014, June). Druid: A real-time analytical data store. In Proceedings of the 2014 ACM SIGMOD international conference on Management of data (pp. 157-168).
  • Silva, Y. N., Almeida, I., & Queiroz, M. (2016, February). SQL: From traditional databases to big data. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education (pp. 413-418).
  • Rahman, M. M., Islam, S., Kamruzzaman, M., & Joy, Z. H. (2024). Advanced Query Optimization in SQL Databases For Real-Time Big Data Analytics. Academic Journal on Business Administration, Innovation & Sustainability, 4(3), 1-14.
  • Uzzaman, A., Jim, M. M. I., Nishat, N., & Nahar, J. (2024). Optimizing SQL databases for big data workloads: techniques and best practices. Academic Journal on Business Administration, Innovation & Sustainability, 4(3), 15-29.
  • Pirzadeh, P., Carey, M., & Westmann, T. (2017, December). A performance study of big data analytics platforms. In 2017 IEEE international conference on big data (big data) (pp. 2911-2920). IEEE.
  • Panwar, V. (2024). Optimizing Big Data Processing in SQL Server through Advanced Utilization of Stored Procedures. Journal Homepage: http://www. ijmra. us, 14(02).
  • Ordonez, C. (2013, October). Can we analyze big data inside a DBMS?. In Proceedings of the sixteenth international workshop on Data warehousing and OLAP (pp. 85-92)
There are 33 citations in total.

Details

Primary Language English
Subjects Computer Software
Journal Section Articles
Authors

Süleyman Burak Altinişik 0009-0005-0987-1798

Turgay Tugay Bilgin 0000-0002-9245-5728

Early Pub Date June 30, 2025
Publication Date June 30, 2025
Submission Date October 9, 2024
Acceptance Date February 10, 2025
Published in Issue Year 2025 Volume: 9 Issue: 1

Cite

APA Altinişik, S. B., & Bilgin, T. T. (2025). Optimizing Big Data Management on Microsoft SQL Server: Enhancing Performance through Normalization and Advanced Analytical Techniques. International Journal of Innovative Engineering Applications, 9(1), 23-36. https://doi.org/10.46460/ijiea.1563777
AMA Altinişik SB, Bilgin TT. Optimizing Big Data Management on Microsoft SQL Server: Enhancing Performance through Normalization and Advanced Analytical Techniques. IJIEA. June 2025;9(1):23-36. doi:10.46460/ijiea.1563777
Chicago Altinişik, Süleyman Burak, and Turgay Tugay Bilgin. “Optimizing Big Data Management on Microsoft SQL Server: Enhancing Performance through Normalization and Advanced Analytical Techniques”. International Journal of Innovative Engineering Applications 9, no. 1 (June 2025): 23-36. https://doi.org/10.46460/ijiea.1563777.
EndNote Altinişik SB, Bilgin TT (June 1, 2025) Optimizing Big Data Management on Microsoft SQL Server: Enhancing Performance through Normalization and Advanced Analytical Techniques. International Journal of Innovative Engineering Applications 9 1 23–36.
IEEE S. B. Altinişik and T. T. Bilgin, “Optimizing Big Data Management on Microsoft SQL Server: Enhancing Performance through Normalization and Advanced Analytical Techniques”, IJIEA, vol. 9, no. 1, pp. 23–36, 2025, doi: 10.46460/ijiea.1563777.
ISNAD Altinişik, Süleyman Burak - Bilgin, Turgay Tugay. “Optimizing Big Data Management on Microsoft SQL Server: Enhancing Performance through Normalization and Advanced Analytical Techniques”. International Journal of Innovative Engineering Applications 9/1 (June 2025), 23-36. https://doi.org/10.46460/ijiea.1563777.
JAMA Altinişik SB, Bilgin TT. Optimizing Big Data Management on Microsoft SQL Server: Enhancing Performance through Normalization and Advanced Analytical Techniques. IJIEA. 2025;9:23–36.
MLA Altinişik, Süleyman Burak and Turgay Tugay Bilgin. “Optimizing Big Data Management on Microsoft SQL Server: Enhancing Performance through Normalization and Advanced Analytical Techniques”. International Journal of Innovative Engineering Applications, vol. 9, no. 1, 2025, pp. 23-36, doi:10.46460/ijiea.1563777.
Vancouver Altinişik SB, Bilgin TT. Optimizing Big Data Management on Microsoft SQL Server: Enhancing Performance through Normalization and Advanced Analytical Techniques. IJIEA. 2025;9(1):23-36.
 Download Cover Image

This work is licensed under CC BY-NC 4.0