Feedback

Faculté des Sciences appliquées
Faculté des Sciences appliquées
MASTER THESIS
VIEW 129 | DOWNLOAD 134

Towards a distributed implementation of a data extraction system for monitoring the processing of video clips

Download
Vasbinder, Thomas ULiège
Promotor(s) : Wolper, Pierre ULiège
Date of defense : 27-Jun-2016/28-Jun-2016 • Permalink : http://hdl.handle.net/2268.2/1306
Details
Title : Towards a distributed implementation of a data extraction system for monitoring the processing of video clips
Author : Vasbinder, Thomas ULiège
Date of defense  : 27-Jun-2016/28-Jun-2016
Advisor(s) : Wolper, Pierre ULiège
Committee's member(s) : Boigelot, Bernard ULiège
Donnet, Benoît ULiège
Latour, Louis 
Language : English
Discipline(s) : Engineering, computing & technology > Computer science
Institution(s) : Université de Liège, Liège, Belgique
Degree: Master en ingénieur civil en informatique, à finalité approfondie
Faculty: Master thesis of the Faculté des Sciences appliquées

Abstract

[en] The objective of this work was to review the data handling of an application monitoring the processing of video clips and to remove any barrier preventing the concurrent execution of multiple instances of the service, while maintaining or improving the overall performance.

The main obstacle to concurrent execution was that the caching infrastructure didn’t provide synchronisation between the cache and the database. We showed that the database could perform queries in memory like the in-memory cache at the server. The database executed most of the queries in a more efficient and predictable way than the cache, thanks to its query optimiser.

Based on those results, we decided to remove the cache and perform queries directly on the database. This removal simplified and shortened the code of the application. However, there was no mechanism of translation of the query contained in the URI to the database. We thus used OData, a tool providing a query language for RESTful APIs, that can translate automatically query parameters from the URI to SQL queries. Yet, OData came with some performance flaws. To restore performances, we used indexes to speed up queries execution and we restricted query parameters combinations.

This work resulted in a solution allowing the concurrent execution of multiple instances of the service. Moreover, queries are in most cases faster and thus the service can now support more clients.

So, this work showed that the cache was not better than the database when executing the queries and could be removed.


File(s)

Document(s)

File
Access report.pdf
Description: Report
Size: 1.16 MB
Format: Adobe PDF
File
Access summary.pdf
Description: Summary
Size: 67.24 kB
Format: Adobe PDF

Annexe(s)

File
Access Parsing.zip
Description: Program parsing measured times (client, middleware, controller) from logs
Size: 2.2 kB
Format: Unknown
File
Access MultiMonitoring (modified version).zip
Description: Modified version of the MultiMonitoring application
Size: 1.71 MB
Format: Unknown
File
Access MultiMonitoring (initial version).zip
Description: Initial version of the MultiMonitoring application
Size: 1.72 MB
Format: Unknown
File
Access ClientMM.zip
Description: Program allowing to send queries to the server a given number of times.
Size: 1.35 kB
Format: Unknown
File
Access database_max_clients.scala
Description: Gatling script used to measure the number of clients (OData version)
Size: 3.51 kB
Format: Unknown
File
Access cache_max_clients.scala
Description: Gatling script used to measure the number of clients (cache version)
Size: 3.29 kB
Format: Unknown
File
Access create_indexes.sql
Description: SQL script creating 132 indexes on the Jobs table
Size: 14.62 kB
Format: Unknown
File
Access fill_db.ps1
Description: PowerShell script used to fill the database
Size: 2.15 kB
Format: Unknown
File
Access images.zip
Description: Illustrations
Size: 221.8 kB
Format: Unknown

Author

  • Vasbinder, Thomas ULiège Université de Liège > Master ingé. civ. info., fin. appr. (ex 2e master)

Promotor(s)

Committee's member(s)

  • Boigelot, Bernard ULiège Université de Liège - ULg > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Informatique
    ORBi View his publications on ORBi
  • Donnet, Benoît ULiège Université de Liège - ULg > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Algorithmique des grands systèmes
    ORBi View his publications on ORBi
  • Latour, Louis EVS - Avenue Bois saint-jean 13, 4102 Seraing
  • Total number of views 129
  • Total number of downloads 134










All documents available on MatheO are protected by copyright and subject to the usual rules for fair use.
The University of Liège does not guarantee the scientific quality of these students' works or the accuracy of all the information they contain.