Metoda prilagodljivega uglaševanja slojev konvolucijskih nevronskih mrež pri strojnem učenju s prenosom znanja

doktorska disertacija

Grega Vrbančič (Author), Vili Podgorelec (Mentor)

Abstract

V doktorski disertaciji predstavimo problematiko izbire uglaševanih slojev konvolucijskih nevronskih mrež pri strojnem učenju s prenosom znanja. Z izvedeno analizo vpliva izbire uglaševanih slojev konvolucijske nevronske mreže na uspešnost učenja potrdimo domnevo, da je primerna izbira uglaševanih slojev s ciljem doseganja visoke klasifikacijske uspešnosti odvisna od izbrane arhitekture konvolucijske nevronske mreže ter ciljnega problema oz. izbrane podatkovne zbirke. Z namenom naslovitve problema izbire uglaševanih slojev razvijemo in predlagamo prilagodljivo metodo DEFT, ki temelji na algoritmu diferencialne evolucije in deluje popolnoma samodejno, ne glede na uporabljeno arhitekturo konvolucijske nevronske mreže ali ciljni problem. Zaradi velike časovne kompleksnosti predlagane metode v nadaljevanju razvijemo in predlagamo na funkciji izgube temelječo metriko LDM, ki v zgodnji fazi učenja uspešno zaznava manj primerne izbire uglaševanih slojev, kar nam omogoča, da za zaznane manj primerne izbire uglaševanih slojev predčasno zaključimo učenje in na tak način zmanjšamo časovno zahtevnost predlagane metode. Uspešnost predlagane metode ovrednotimo z uporabo treh različnih arhitektur globokih konvolucijskih mrež nad tremi raznolikimi slikovnimi podatkovnimi zbirkami. Klasifikacijsko uspešnost predlagane metode z in brez uporabe metrike LDM smo primerjali s klasičnimi pristopi učenja globokih konvolucijskih nevronskih mrež. Primerjavo izvedemo z uporabo najpogostejših klasifikacijskih metrik, časom, potrebnim za učenje, ter porabljenim številom epoh. Rezultate smo preverili z uporabo klasičnih metod statistične analize kot tudi z naprednim pristopom Bayesove analize. Izsledki slednje so potrdili tezo, da je mogoče z uporabo metode prilagodljivega uglaševanja slojev konvolucijske nevronske mreže uspešno nasloviti problem izbire slojev ter da lahko z uporabo metrike LDM za zaznavo manj primernih izbir uglaševanih slojev učinkovito zmanjšamo število epoh, potrebnih za učenje, ob doseganju primerljivih rezultatov.

Keywords

strojno učenje;globoko učenje;učenje s prenosom znanja;klasifikacija;uglaševanje slojev konvolucijskih nevronskih mrež;konvolucijske nevronske mreže;optimizacija;doktorske disertacije;

Data

Language:	Slovenian
Year of publishing:	2021
Typology:	2.08 - Doctoral Dissertation
Organization:	UM FERI - Faculty of Electrical Engineering and Computer Science
Publisher:	[G. Vrbančič]
UDC:	004.032.26:004.85(043.3)
COBISS:	82430723
Views:	671
Downloads:	129
Average score:	0 (0 votes)
Metadata:

Other data

Secondary language:	English
Secondary title:	Method for adaptive fine-tuning of convolutional neural network layers using transfer learning
Secondary abstract:	In this Doctoral Dissertation, we present the problem of selecting fine-tunable layers when utilizing transfer learning with the fine-tuning approach for training deep convolutional neural networks. With the conducted empirical analysis of layer selection impact on the training performance, we confirmed the assumption that the most suitable selection of fine-tuned layers depends on the chosen convolutional neural network architecture, as well as on the target problem. In order to address the problem of selecting the most suitable combination of fine-tunable layers, we developed and proposed an adaptive method, DEFT, based on a differential evolution algorithm, which works in a straightforward automatic manner using different convolutional neural network architectures. Due to the high time complexity of the proposed method, we developed and proposed a metric derived from the loss value, which is capable of detecting less suitable selections of fine-tunable layers at an early stage of training, which allows us to terminate training early, and, thus, reduce the time complexity of the proposed method. The performance of the proposed method was evaluated by utilizing three different convolutional neural network architectures against three different image datasets. Classification performance of the proposed DEFT method, with or without the proposed metric LDM, was compared against conventional approaches for training convolutional neural networks. The performance comparison was conducted using the most common classification metrics, consumed time for training, and consumed number of epochs. The statistical analysis of the obtained results was conducted using conventional statistical methods, as well as modern Bayesian analysis based approaches. The results confirmed the initial thesis that the problem of layer selection when utilizing transfer learning with fine-tuning, can be addressed successfully using the proposed adaptive DEFT method, and that utilization of the proposed LDM metric reduces the number of epochs needed for training effectively, while achieving comparable results.
Secondary keywords:	machine learning;deep learning;transfer learning;classification;fine-tuning;optimization;Globoko učenje (strojno učenje);Univerzitetna in visokošolska dela;
Type (COBISS):	Doctoral dissertation
Thesis comment:	Univ. v Mariboru, Fak. za elektrotehniko, računalništvo in informatiko
Pages:	XXV, 166 str.
ID:	13003169