Automatic collection and storage of smart city data with semantic data model discovery and sample data analysis
Master thesis
Permanent lenke
http://hdl.handle.net/11250/2455296Utgivelsesdato
2017-06Metadata
Vis full innførselSamlinger
- Studentoppgaver (TN-IDE) [934]
Sammendrag
Collecting and storing smart city data is a task that requires thorough data exploration, configuring and testing to be of value. Configuring a data collection pipeline for data from a new data provider needs to take into account what the various fields represent, what parts of the data is of interest, which data fields should be stored, and more. In some cases the data follows a predefined, and known schema, in other cases the data may be undocumented.
This thesis presents a framework and a software for automating the process of collecting and storing smart city data, and other event based data sets. The problem, and solution is illustrated in this thesis by a use case where the task consist of storing public transportation data in a structured way in a storage system that can handle big data.
Beskrivelse
Master's thesis in Computer Science