bachelor thesis
Lazar Đoković (Author), Marko Robnik Šikonja (Mentor)

Abstract

Sarcasm detection is a natural language processing task of classifying whether an utterance is sarcastic or not. It is closely related to sentiment analysis since it often inverts surface sentiment. Despite the great interest and research done by the sentiment analysis community, it remains a challenging problem. This is because sarcastic sentences are highly dependent on context, and they are often accompanied by various non-verbal cues. Recent work in sarcasm detection mostly focuses on the Transformer architecture of neural networks and its application in high-resourced languages like English. To build a sarcasm detection dataset for Slovene, we leverage two modern techniques in machine translation and language modeling. The first approach uses a medium-size Transformer model trained specifically for neural machine translation, while the second method utilizes a very large generative model. We explore the viability of such datasets and how the size of a pretrained Transformer affects its ability to detect sarcasm. We use this data to train ensembles of Transformer-based models. We evaluate model performance using established methodologies. Our results show that larger models generally outperform smaller ones, and that ensembling can slightly improve sarcasm detection performance. Our best ensemble approach achieves an $\text{F}_1$-score of 0.765.

Keywords

natural language processing;large language models;sarcasm detection;neural machine translation;BERT model; GPT model;Llama model;computer and information science;diploma thesis;

Data

Language: English
Year of publishing:
Typology: 2.11 - Undergraduate Thesis
Organization: UL FRI - Faculty of Computer and Information Science
Publisher: [L. Đoković]
UDC: 004.85:81'322(043.2)
COBISS: 208260867 Link will open in a new window
Views: 204
Downloads: 77
Average score: 0 (0 votes)
Metadata: JSON JSON-RDF JSON-LD TURTLE N-TRIPLES XML RDFA MICRODATA DC-XML DC-RDF RDF

Other data

Secondary language: Slovenian
Secondary title: Zaznavanje sarkazma s prenosom znanja iz več virov
Secondary abstract: Zaznavanje sarkazma je naloga obdelave naravnega jezika, pri kateri ugotavljamo, ali je izjava sarkastična ali ne. Tesno je povezana z analizo mnenj, saj pogosto spremeni površinsko razumljeno mnenje. Kljub mnogim raziskavam ostaja sarkazem izziv za avtomatsko detekcijo, saj so sarkastični stavki odvisni od konteksta in jih pogosto spremljajo neverbalni znaki. Nedavni pristopi k zaznavanju sarkazma večinoma uporabljajo arhitekturo nevronskih mrež transformer v jezikih z veliko viri, predvsem v angleščini. Za izdelavo učne množice za zaznavanje sarkazma v slovenščini smo uporabili dve sodobni tehniki strojnega prevajanja in jezikovnega modeliranja. Prvi pristop uporablja srednje velik model transformer, učen posebej za nevronsko strojno prevajanje, medtem ko druga metoda uporablja zelo velik generativni jezikovni model. Raziskali smo uporabnost teh učnih množic in kako velikost modelov vpliva na njihovo sposobnost zaznavanja sarkazma. Z generiranimi podatki smo ustvarili več modelov in napovedni ansambel, sestavljen iz več jezikovnih modelov. Pristope smo ovrednotili z uporabo uveljavljenih metod. Rezultati kažejo, da večji modeli presegajo manjše, ansambli pa nekoliko izboljšajo uspešnost zaznavanja sarkazma. Naš najboljši ansambel doseže $\text{F}_1$-oceno 0,765.
Secondary keywords: veliki jezikovni modeli; zaznavanje sarkazma;strojno prevajanje;model BERT;model GPT;model Llama;univerzitetni študij;diplomske naloge;Obdelava naravnega jezika (računalništvo);Računalniško jezikoslovje;Posmeh;Računalništvo;Univerzitetna in visokošolska dela;
Type (COBISS): Bachelor thesis/paper
Study programme: 1000468
Embargo end date (OpenAIRE): 1970-01-01
Thesis comment: Univ. v Ljubljani, Fak. za računalništvo in informatiko
Pages: 1 spletni vir (1 datoteka PDF (59 str.))
ID: 24824457