Poenostavitev ETL procesa z uporabo platforme Talend

diplomsko delo

Tomaž Čufer (Author), Dejan Lavbič (Mentor)

Abstract

ETL proces predstavlja širok pojem pridobivanja, preoblikovanja in nalaganja podatkov. Vsaka izmed faz potrebuje podrobno definiran postopek, ki bo podatke prenesel na drugo lokacijo ali preoblikoval v potrebno obliko. Nestrukturirana oblika podatkov in njihova velika količina, ki sta pogosti danes, proces še dodatno otežujeta, kar podaljšuje njegovo izvedbo. S primernim ETL orodjem lahko poenostavimo implementacijo procesa in zagotovimo boljši nadzor nad izvajanjem. V diplomski nalogi se je s takim orodjem hotelo pokazati, kako to poenostavitev narediti v praksi. Primerjani sta dve komercialni in odprtokodni orodji. Izbrano je bilo orodje Talend in nato podrobneje predstavljeno njeno delovanje. Opisano je reševanje problemov z obvladovanjem ter integracijo podatkov pridobljenih s spletnim luščenjem in iz socialnega omrežja Twitter. Za orodje je na koncu opravljena še SWOT analiza.

Keywords

opravilo;proces;integracija podatkov;Talend;orodje;podatkovno skladišče;računalništvo;računalništvo in informatika;univerzitetni študij;diplomske naloge;

Data

Language:	Slovenian
Year of publishing:	2015
Typology:	2.11 - Undergraduate Thesis
Organization:	UL FRI - Faculty of Computer and Information Science
Publisher:	[T. Čufer]
UDC:	004.6(043.2)
COBISS:	1536264131
Views:	2518
Downloads:	318
Average score:	0 (0 votes)
Metadata:

Other data

Secondary language:	English
Secondary title:	Simplification of ETL processes using Talend Platform
Secondary abstract:	The ETL process presents a broad concept of extracting, transforming and loading data. Each of these phases needs to be well defined to transfer the data efficiently to a different location or transform it into the demanded form. Unstructured forms of data along with its huge volume, which is common nowadays, makes this process even more difficult, and is reflected in the longer execution time. With a suitable ETL tool it is possible to simplify the implementation process and assure better control over it. The thesis describes how to complete such simplifications using an appropriate tool in practice. Two commercial and open source tools were compared. Talend tool was chosen and its workflow was later presented in detail. Handling management and integration problems of data is described, where the used data came from web scraping and the Twitter social network. At the end, a SWOT analysis was made for Talend tool.
Secondary keywords:	job;process;data integration;Talend;tool;data warehouse;computer science;computer and information science;diploma;
File type:	application/pdf
Type (COBISS):	Bachelor thesis/paper
Study programme:	1000468
Thesis comment:	Univ. v Ljubljani, Fak. za računalništvo in informatiko
Pages:	42 str.
ID:	8739705