magistrsko delo
Gašper Jelovčan (Author), Kristína Malinovská (Mentor)

Abstract

Univerzalno dvosmerno aktivacijsko učenje (UBAL) je nov učni algoritem za umetne nevronske mreže, ki temelji na delovanju pravih bioloških nevronov. Namesto razširjanja napak za posodabljanje uteži uporablja lokalne aktivacijske vrednosti. Razširi kontrastno hebbijsko učenje, ki za posodobitve uporablja presinaptične in postsinaptične aktivacijske vrednosti. Je tudi dvosmerno, ker uporablja dve različni matriki uteži za vsako smer aktivacije. Zamisli za UBAL izvirajo iz predhodnih algoritmov recirkulacije in GeneRec. UBAL še nikoli ni bil izveden v konvolucijski različici, kar je glavni cilj magistrskega dela. Konvolucijske nevronske mreže so običajno primernejše za obdelavo slik, zato je bila naša hipoteza, da bo konvolucijski UBAL dal boljše rezultate tudi pri nalogi klasifikacije slik v primerjavi s sedanjo, popolnoma povezano različico. Konvolucijski UBAL smo implementirali in izvedli poskuse v programskem jeziku Python, in sicer v knjižnici Pytorch. Izvedbo smo preizkusili na znanem naboru podatkov MNIST. Rezultati so pokazali da smo bili uspešni in sicer z najboljšo klasifikacijsko točnostjo 89.678% v primerjavi z 91.05% točnostjo za nevronsko mrežo z algoritmom vzvratnega širjenja napake. Poleg konvolucijskega UBAL smo raziskali tudi vpliv ciljnega kodiranja na uspešnost mreže. Namesto eničnega kodiranja smo v izhodnih nevronih uporabili poenostavljene binarne slike števk. Naši rezultati kažejo, da nismo uspeli doseči boljše klasifikacije.

Keywords

UBAL;neural networks;convolution;MNIST;

Data

Language: English
Year of publishing:
Typology: 2.09 - Master's Thesis
Organization: UL PEF - Faculty of Education
Publisher: [G. Jelovčan]
UDC: 165.194(043.2)
COBISS: 183335171 Link will open in a new window
Views: 17
Downloads: 2
Average score: 0 (0 votes)
Metadata: JSON JSON-RDF JSON-LD TURTLE N-TRIPLES XML RDFA MICRODATA DC-XML DC-RDF RDF

Other data

Secondary language: Slovenian
Secondary title: Biologically Motivated Learning in Neural Networks with Convolutional Architectures
Secondary abstract: Universal Bidirectional Activation-based learning (UBAL) is a novel learning algorithm for artificial neural networks, which is based on the workings of real biological neurons. Instead of propagating error derivatives it uses local activation values for updating its weights. It extends contrastive Hebbian learning, which uses presynaptic and postsynaptic activation values for the updates. It is also bidirectional for which it uses two different weight matrices for each activation direction. The ideas for UBAL comes from its predecessor algorithms recirculation and GeneRec. UBAL has never before been implemented in a convolutional version, which was the main aim of the master thesis. Convolutional neural networks are usually better suited for processing images, therefore our hypothesis was that convolutional UBAL also yields better results in the image classification task compared to the current, fully connected version. We implemented convolutional UBAL and make experiments in the programming language Python, namely the Pytorch library. We tested the implementation on the famous MNIST dataset. The results show that we were reasonably successful with our best classification accuracy of 89.678% compared to 91.05% for backpropagation neural network. Apart from convolutional UBAL, we explored the influence of target encoding on success of UBAL network. Namely, we used simplified binary images of digits instead of one-hot encoding in the output neurons. Our results show, we did not achieve better results with this approach.
Secondary keywords: UBAL;nevronske mreže;konvolucija;MNIST;Kognitivna znanost;Univerzitetna in visokošolska dela;
Type (COBISS): Master's thesis/paper
Study programme: 0
Embargo end date (OpenAIRE): 1970-01-01
Thesis comment: Univ. of Ljubljana joint interdisciplinary program in Cognitive science in collaboration with Universität Wien, Univerzita Komenského v Bratislave and Eötvös Loránd Tudományegyetem
Pages: 1 spletni vir (1 datoteka PDF (83 str.))
DOI: 20.500.12556/RUL-154178
ID: 22737183