Generative LLMs for Named Entity Recognition (NER) in Swedish

Goal: Test and evaluate the performance of generative LLMs for Named Entity Recognition on Swedish parliamentary debates.

Background: Named Entity Recognition (NER) is the process of automatically identifying and classifying entity types, such as people and places, in text. The current state of the art in NER for Swedish are two BERT (encoder-only tranformer) models.

The data consists of 450 speeches from the Swedish parliamentary debates. They have been manually annotated with eight different types of named entities.

Project description: In this project, the student(s) will test and evaluate a number of generative LLMs, at least some of which are open source, on their abilities to perform NER on the data. Evaluation should be done both in comparison to the human annotators (using Krippendorff's Alpha) and in overall performance (using precision and recall). Along with the annotated data, the student(s) will receive the same guidelines as the human annotators, which should be used for zero shot experiments.

Recommended knowledge and skills: The project requires knowledge of some programming language, as well as familiarity with generative LLMs. Knowledge of Swedish is recommended.