SloBench

Slovenian Natural Language Processing Benchmark

Frenk Dragar (Author), Slavko Žitnik (Mentor)

Abstract

Evaluation of natural language processing (NLP) tasks is an essential part of research and progress in the field. It provides an objective standard for comparison and performance on a specific task. We give an overview of recent public benchmarks and evaluation trends, with focus on the automatic evaluation of systems. We then propose, implement and document a general and extendable model-agnostic evaluation framework, along with the first online platform for the automatic evaluation of Slovene language NLP tasks with public leaderboards, showing the performance of submitted systems.

Keywords

natural language processing;benchmarking;leaderboard;machine learning;web platform;computer science;computer science and mathematics;interdisciplinary studies;diploma;

Data

Language:	English
Year of publishing:	2022
Typology:	2.11 - Undergraduate Thesis
Organization:	UL FRI - Faculty of Computer and Information Science
Publisher:	[F. Dragar]
UDC:	004.9:81'322(043.2)
COBISS:	105616643
Views:	576
Downloads:	149
Average score:	0 (0 votes)
Metadata:

Other data

Secondary language:	Slovenian
Secondary title:	SloBench
Secondary abstract:	Evalvacija nalog procesiranja naravnega jezika (NLP) je bistven del raziskav in napredka na tem področju. Zagotavlja objektiven standard za uspešnost in primerjavo sistemov pri določeni nalogi. Podamo pregled nedavnih javnih lestvic za najboljše sisteme in trendov njihovega ocenjevanja s poudarkom na avtomatskem vrednotenju sistemov. Nato predlagamo, implementiramo in dokumentiramo splošno, razširljivo in od sistemske arhitekture neodvisno ogrodje za evalvacijo sistemov, skupaj s prvo spletno platformo za avtomatsko vrednotenje NLP nalog v slovenščini z javnimi lestvicami, ki prikazujejo rezultate objavljenih sistemov.
Secondary keywords:	procesiranje naravnega jezika;vrednotenje;lestvica najboljših;spletna platforma;računalništvo in matematika;interdisciplinarni študij;univerzitetni študij;diplomske naloge;Obdelava naravnega jezika (računalništvo);Računalniško jezikoslovje;Strojno učenje;Računalništvo;Univerzitetna in visokošolska dela;
Type (COBISS):	Bachelor thesis/paper
Study programme:	1000407
Thesis comment:	Univ. v Ljubljani, Fak. za računalništvo in informatiko
Pages:	45 str.
ID:	15098308