Transport protocol for machine/deep learning
Chavet, Thibaut
Promotor(s) : Mathy, Laurent
Date of defense : 6-Sep-2018/7-Sep-2018 • Permalink : http://hdl.handle.net/2268.2/5513
Details
Title : | Transport protocol for machine/deep learning |
Author : | Chavet, Thibaut |
Date of defense : | 6-Sep-2018/7-Sep-2018 |
Advisor(s) : | Mathy, Laurent |
Committee's member(s) : | Donnet, Benoît
Geurts, Pierre Louppe, Gilles |
Language : | English |
Discipline(s) : | Engineering, computing & technology > Computer science |
Institution(s) : | Université de Liège, Liège, Belgique |
Degree: | Master en ingénieur civil en informatique, à finalité spécialisée en "computer systems and networks" |
Faculty: | Master thesis of the Faculté des Sciences appliquées |
Abstract
[en] Datasets for machine learning can be very large in size, and are usually transferred to dedicated servers that have the computing power necessary for complex model training.
This transfer is done losslessly by TCP, but could allow some losses without affecting too much the quality of the trained model, which would make the transfer faster.
The first part of this work looks at how packet loss on different datasets affect the accuracy of the corresponding model, and how the different ways to deal with packet loss compare to one another. It finds that it is indeed possible to lose packets during the transfer without affecting the model too much, but it requires some data preparation to repair the files if parts of it
are lost, and some parts cannot be lost and are absolutely necessary to the files.
Using those results, a protocol is designed and implemented to transfer datasets partly by TCP and partly by UDP depending on which parts of the files are required or can be lost without too many consequences. Focus is mainly set on JPEG files.
Tests then show that the protocol can be indeed be faster, but parameters need to be set to the right values or the ratio speed/loss becomes too bad.
File(s)
Document(s)
Annexe(s)
Cite this master thesis
The University of Liège does not guarantee the scientific quality of these students' works or the accuracy of all the information they contain.