Dataset Creation from Job Portals using Web Scraping to enhance the Employability
Keywords:
Web Content Mining, Web Scraping, Job Portal, Data Preprocessing, Employability, Key skillsAbstract
The Computer Science is a highly sought-after field, offering some of the best job opportunities in the Information Technology sector through various job portals. Students often find that their skills in computing can open many doors in the rapidly evolving tech industry. There's a significant gap between the skills taught in educational settings and those demanded by the industry, largely due to the rapid evolution of technology. To address this, educational institutions need to frequently update their curricula collaborate with industry leaders to ensure students acquire relevant, up-to-date skills for the workforce. Thus, it's essential to analyze the skills that are currently in demand within the IT industry. Recently, a multitude of data including text, images, audio, and video has become accessible online, and extracting useful information from these diverse web contents is a key application of Web Content Mining. A job portal is an online platform where companies can post job listings, offering a quick, reliable, and precise way to connect with potential employees. Web scraping has evolved into a crucial tool for obtaining important data from a variety of sources like Job portals. In this Research Paper, the Web Scrapping procedures and methods have been developed in Python to scrape Information Technology employment details from renowned job portals like Naukri.com, Monster.com, TimesJobs.com, Internshala.com and Myamcat.com.
References
[1] A. Richlin, S. Jebakumari, and N. J. Goldena, “A Survey on Web Content Mining Methods and Applications for Perfect Catch Responses,” Int. Res. J. Eng. Technol., p. 407, 2008.
[2] Fatmasari, Y. N. Kunang, and S. D. Purnamasari, “Web Scraping Techniques to Collect Weather Data in South Sumatera,” Proc. 2018 Int. Conf. Electr. Eng. Comput. Sci. ICECOS 2018, vol. 17, pp. 385–390, 2019, doi: 10.1109/ICECOS.2018.8605202.
[3] P. Thota and E. Ramez, “Web Scraping of COVID-19 News Stories to Create Datasets for Sentiment and Emotion Analysis,” ACM Int. Conf. Proceeding Ser., pp. 306–314, 2021, doi: 10.1145/3453892.3461333.
[4] M. A. Khder, “Web scraping or web crawling: State of art, techniques, approaches and application,” Int. J. Adv. Soft Comput. its Appl., vol. 13, no. 3, pp. 144–168, 2021, doi: 10.15849/ijasca.211128.11.
[5] Panda, S. K., Bhatt, A., & Satapathy, A. (2024). ChatGPT and Its Role in Academic Libraries: A Discussion. New Review of Academic Librarianship, 30(4), 422–436. https://doi.org/10.1080/13614533.2024.2381510
[6] M. Pejic-bach, T. Bertoncel, M. Meško, and Ž. Krstić, “International Journal of Information Management Text mining of industry 4 . 0 job advertisements,” Int. J. Inf. Manage., vol. 50, no. July, pp. 416–431, 2020.
[7] F. García-Peñalvo, J. Cruz-Benito, M. Martín-González, A. Vázquez-Ingelmo, J. C. Sánchez-Prieto, and R. Therón, “Proposing a Machine Learning Approach to Analyze and Predict Employment and its Factors,” Int. J. Interact. Multimed. Artif. Intell., vol. 5, no. 2, p. 39, 2018, doi: 10.9781/ijimai.2018.02.002.
[8] M. P. Ang, “EVALUATING the responsiveness of bs is curriculum to industry skill requirements Aileen L . Rillon , MBA , MIM A Research Report Submitted to the Ateneo De Naga University Research Council October 2017,” no. October, 2017.
[9] D. T Smith and A. Ali, “Analyzing Computer Programming Job Trend Using Web Data Mining,” Issues Informing Sci. Inf. Technol., vol. 11, pp. 203–214, 2014, doi: 10.28945/1989.
[10] K. T. Selvi and R. Ramya, “an Impact on Electronic-Recruitment and Its Perception Towards Job Portal Function Through Search Engines Among Job Seekers Using Knime Data Mining Tool,” Int. J. Res. Dev. Technol., no. 5, pp. 10–18, 2016.
[11] P. Ramya, S. G. Balakrishnan, and M. Kannan, “Recommendation system to improve students performance using machine learning,” IOP Conf. Ser. Mater. Sci. Eng., vol. 872, no. 1, 2020, doi: 10.1088/1757-899X/872/1/012038.
[12] W. Shalaby et al., “Help me find a job: A graph-based approach for job recommendation at scale,” Proc. - 2017 IEEE Int. Conf. Big Data, Big Data 2017, vol. 2018-Janua, pp. 1544–1553, 2017, doi: 10.1109/BigData.2017.8258088.
