With the TÜİK Big Data Advanced Analytics Project, the storage and processing of daily price information and job postings, labeled with category and sub-category information, collected from websites, as bulk and streaming data, in the big data ecosystem within the body of the Turkish Republic Ministry of Treasury and Finance Turkish Statistical Institute (TÜİK). It is aimed to design a system that provides analysis and analysis. Thanks to the system, it will be possible to classify positions and talents from job postings, visualize the results, track prices for plane-bus-packet tour prices, and perform lag analysis.
Lambda architecture is used in order to transfer the data collected from the websites to the big data environment in the form of streaming data and to analyze the transferred data as bulk and streaming data. The system architecture is developed using open source tools in the big data ecosystem and the Cloud Computing and Big Data Research Lab (B3LAB) Data Quality Tool (B3DataQuality). The small-scale demo installation of the system, which is under development, is carried out at the B3LAB Prototype Data Center located in TÜBİTAK BİLGEM Gebze Campus.
Within the scope of the project, machine learning and deep learning methods, job posting position and talent classification models and lag analysis models will be created by using mass data in the mass data processing infrastructure in the big data environment. The results to be obtained by processing the flowing data using machine learning models will be visualized in the business intelligence tool compatible with the big data environment.