Strategies and Considerations in Constructing Data Warehouses with NoSQL Database Technologies
Keywords:
NoSQL, Data Warehouse, Data Analytics, Semi-structured data, Structured dataAbstract
The evolution of data warehousing methodologies has witnessed a significant transformation with the integration of NoSQL database technologies. Traditionally, data warehousing heavily relied on relational databases, but the surge in diverse, high-velocity, and voluminous data necessitated a paradigm shift. NoSQL databases emerged as a versatile solution, offering unprecedented flexibility, scalability, and adaptability in managing modern data challenges. Contemporary data warehousing landscapes increasingly gravitate towards NoSQL databases, prompting a pivotal shift in designing robust and scalable data warehouses.
By capturing the essence of designing data warehouses using NoSQL databases, this paper presents an informed overview of the pivotal aspects, challenges, and advancements in this dynamic domain. It serves as a guide for organizations aiming to harness the transformative potential of NoSQL technologies within their data warehousing architectures. The research explores the diverse landscape of NoSQL databases, encompassing document-oriented, key-value stores, columnar databases, and graph databases. These databases offer a dynamic framework capable of accommodating diverse and complex data structures encountered in today's data ecosystems. Embracing this paradigm shift, organizations seek innovative approaches to adapt and optimize their data warehousing structures. By outlining critical strategies and considerations, this research aims to provide a comprehensive roadmap for organizations navigating the integration of NoSQL database technologies in constructing robust, scalable, and adaptable data warehouses.
References
[1] H. Mallek, F. Ghozzi, O. Teste, and F. Gargouri, “BigDimETL with NoSQL Database,” Procedia Comput. Sci., vol. 126, pp. 798–807, 2018, doi: 10.1016/j.procS.2018.08.014.
[2] R. Yangui, A. Nabli, and F. Gargouri, “ETL based framework for NoSQL warehousing,” Lect. Notes Bus. Inf. Process., vol. 299, no. August, pp. 40–53, 2017, doi: 10.1007/978-3-319-65930-5_4.
[3] A. Sabtu et al., “The challenges of Extract, Transform and Loading (ETL) system implementation for near real-time environment,” Int. Conf. Res. Innov. Inf. Syst. ICRIIS, pp. 3–7, 2017, doi: 10.1109/ICRIIS.2017.8002467.
[4] S. R. Sandro Bimonte, Enrico Gallinucci, Patrick Marcel, “Data Variety, Come As You Are in Multi-model Data Warehouses Sandro,” Elsevier, 2022.
[5] S. Bouaziz, A. Nabli, and F. Gargouri, “From traditional data warehouse to real time data warehouse,” Adv. Intell. Syst. Comput., vol. 557, pp. 467–477, 2017, doi: 10.1007/978-3-319-53480-0_46.
[6] M. Patel and D. B. Patel, “Progressive Growth of ETL Tools: A Literature Review of Past to Equip Future,” Adv. Intell. Syst. Comput., vol. 1187, pp. 389–398, 2021, doi: 10.1007/978-981-15-6014-9_45.
[7] M. Patel and D. B. Patel, “Data Warehouse Modernization Using Document-Oriented ETL Framework for Real Time Analytics,” Lect. Notes Networks Syst., vol. 434, pp. 33–41, 2022, doi: 10.1007/978-981-19-1122-4_5.
[8] A. N. Istiqamah and K. R. S. Wiharja, “a Schema Extraction of Document-Oriented Database for Data Warehouse,” Int. J. Inf. Commun. Technol., vol. 7, no. 2, pp. 36–47, 2021, doi: 10.21108/ijoict.v7i2.584.
[9] L. Petricioli, L. Humski, and B. Vrdoljak, “The Challenges of NoSQL Data Warehousing,” E-bus. Technol. Conf. Proc., vol. 1, no. 1, pp. 44–48, 2021, [Online]. Available: https://ebt.rs/journals/index.php/conf-proc/article/view/86
[10] Panda, S. K., Bhatt, A., & Satapathy, A. (2024). ChatGPT and Its Role in Academic Libraries: A Discussion. New Review of Academic Librarianship, 30(4), 422–436. https://doi.org/10.1080/13614533.2024.2381510
[11] H. Dabbèchi, N. Z. Haddar, H. Elghazel, and K. Haddar, “Social Media Data Integration: From Data Lake to NoSQL Data Warehouse,” no. June, pp. 701–710, 2021, doi: 10.1007/978-3-030-71187-0_64.
[12] F. Abdelhedi, R. Jemmali, and G. Zurfluh, “Relational Databases Ingestion into a NoSQL Data Warehouse,” arXiv Prepr. arXiv2203.06949, 2022.
[13] F. Abdelhedi, R. Jemmali, and G. Zurfluh, “Ingestion of a Data Lake into a NoSQL Data Warehouse: The Case of Relational Databases,” Int. Jt. Conf. Knowl. Discov. Knowl. Eng. Knowl. Manag. IC3K - Proc., vol. 3, no. Ic3k, pp. 64–72, 2021, doi: 10.5220/0010690600003064.
[14] B. F. P. de Oliveira, M. d. C. Victorino, and M. Holanda, “Data Warehouse Based on NoSQL: a literature mapping,” in 2021 16th Iberian Conference on Information Systems and Technologies (CISTI), 2021, pp. 1–6. doi: 10.23919/CISTI52073.2021.9476293.
[15] I. Ben Messaoud, A. A. Alshdadi, and J. Feki, “Building a Document-Oriented Warehouse Using NoSQL,” Int. J. Oper. Res. Inf. Syst., vol. 12, no. 2, pp. 33–54, 2021, doi: 10.4018/ijoris.20210401.oa3.
[16] M. V.-T. J. of C. and Mathematics and undefined 2021, “Novel Solution for Real Time Challenges of ETL in Big Data,” Turcomat.Org, vol. 12, no. 10, pp. 3661–3674, 2021, [Online]. Available: https://www.turcomat.org/index.php/turkbilmat/article/view/5054
[17] S. Bouaziz, A. Nabli, and F. Gargouri, “NoSQL Big Data Warehouse: Review and Comparison BT - Intelligent Systems Design and Applications,” A. Abraham, V. Piuri, N. Gandhi, P. Siarry, A. Kaklauskas, and A. Madureira, Eds., Cham: Springer International Publishing, 2021, pp. 392–401.
