Feedback

Faculté des Sciences appliquées
Faculté des Sciences appliquées
MASTER THESIS
VIEW 17 | DOWNLOAD 1

Use of Flink Hybrid sources to initialise transformations

Download
Azoury, Elie Junior ULiège
Promotor(s) : Debruyne, Christophe ULiège
Date of defense : 24-Jun-2024/25-Jun-2024 • Permalink : http://hdl.handle.net/2268.2/20383
Details
Title : Use of Flink Hybrid sources to initialise transformations
Author : Azoury, Elie Junior ULiège
Date of defense  : 24-Jun-2024/25-Jun-2024
Advisor(s) : Debruyne, Christophe ULiège
Committee's member(s) : Fontaine, Pascal ULiège
Leduc, Guy ULiège
Bruggeman, Jehan 
Language : English
Keywords : [en] Apache Flink, Flink Hybrid Sources, Data transformation, Digazu, Stream processing, Batch processing
Discipline(s) : Engineering, computing & technology > Computer science
Target public : Researchers
Professionals of domain
Student
General public
Institution(s) : Université de Liège, Liège, Belgique
Degree: Master en sciences informatiques, à finalité spécialisée en "computer systems security"
Faculty: Master thesis of the Faculté des Sciences appliquées

Abstract

[en] This master's thesis explores using Apache Flink's HybridSource technology to enhance the data transformation processes within the Digazu framework, a data processing platform developed by EuraNova. The thesis addresses the complexities of transitioning from batch to stream processing, traditionally managed through custom orchestration solutions. By leveraging HybridSource, which facilitates seamless data ingestion from multiple heterogeneous sources, the study aims to reduce orchestration code complexity, optimize resource usage, and improve latency performance.

The research involves a comprehensive literature review on streaming frameworks, a technical analysis of the HybridSource implementation in Apache Flink, and empirical validation through proof-of-concept experiments. By practically implementing HybridSource within Digazu's infrastructure, the study evaluates its feasibility and effectiveness in streamlining data transformation processes. This investigation demonstrates significant reductions in orchestration code complexity and optimized resource usage and highlights improvements in latency performance.


File(s)

Document(s)

File
Access main.pdf
Description:
Size: 1.34 MB
Format: Adobe PDF
File
Access Abstract.pdf
Description:
Size: 74.37 kB
Format: Adobe PDF

Author

  • Azoury, Elie Junior ULiège Université de Liège > Master sc. inform. fin. spéc. comput. syst. secur.

Promotor(s)

Committee's member(s)

  • Fontaine, Pascal ULiège Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Systèmes informatiques distribués
    ORBi View his publications on ORBi
  • Leduc, Guy ULiège Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Réseaux informatiques
    ORBi View his publications on ORBi
  • Bruggeman, Jehan
  • Total number of views 17
  • Total number of downloads 1










All documents available on MatheO are protected by copyright and subject to the usual rules for fair use.
The University of Liège does not guarantee the scientific quality of these students' works or the accuracy of all the information they contain.