| Naslov: | Evaluating Proprietary and Open-Weight Large Language Models as Universal Decimal Classification Recommender Systems |
|---|
| Avtorji: | ID Borovič, Mladen, Faculty of Electrical Engineering and Computer Science, University of Maribor, 2000 Maribor, Slovenia (Avtor)
ID Tomovski, Eftimije, Faculty of Electrical Engineering and Computer Science, University of Maribor, 2000 Maribor, Slovenia (Avtor)
ID Li Dobnik, Tom, Faculty of Electrical Engineering and Computer Science, University of Maribor, 2000 Maribor, Slovenia (Avtor)
ID Majninger, Sandi, Faculty of Electrical Engineering and Computer Science, University of Maribor, 2000 Maribor, Slovenia (Avtor) |
|---|
| Datoteke: |  applsci-15-07666-v2.pdf (447,50 KB)
MD5: 72ACD70840709FDBA57806D0FE92EBAF
 https://www.mdpi.com/2076-3417/15/14/7666/pdf
|
|---|
| Jezik: | Angleški jezik |
|---|
| Vrsta gradiva: | Članek v reviji |
|---|
| Tipologija: | 1.01 - Izvirni znanstveni članek |
|---|
| Organizacija: | FERI - Fakulteta za elektrotehniko, računalništvo in informatiko
|
|---|
| Opis: | Manual assignment of Universal Decimal Classification (UDC) codes is time-consuming and inconsistent as digital library collections expand. This study evaluates 17 large language models (LLMs) as UDC classification recommender systems, including ChatGPT variants (GPT-3.5, GPT-4o, and o1-mini), Claude models (3-Haiku and 3.5-Haiku), Gemini series (1.0-Pro, 1.5-Flash, and 2.0-Flash), and Llama, Gemma, Mixtral, and DeepSeek architectures. Models were evaluated zero-shot on 900 English and Slovenian academic theses manually classified by professional librarians. Classification prompts utilized the RISEN framework, with evaluation using Levenshtein and Jaro–Winkler similarity, and a novel adjusted hierarchical similarity metric capturing UDC’s faceted structure. Proprietary systems consistently outperformed open-weight alternatives by 5–10% across metrics. GPT-4o achieved the highest hierarchical alignment, while open-weight models showed progressive improvements but remained behind commercial systems. Performance was comparable between languages, demonstrating robust multilingual capabilities. The results indicate that LLM-powered recommender systems can enhance library classification workflows. Future research incorporating fine-tuning and retrieval-augmented approaches may enable fully automated, high-precision UDC assignment systems. |
|---|
| Ključne besede: | universal decimal classification, large language models, conversational systems, recommender systems, prompt engineering, zero-shot classification, hierarchical similarity |
|---|
| Status publikacije: | Objavljeno |
|---|
| Verzija publikacije: | Objavljena publikacija |
|---|
| Poslano v recenzijo: | 20.06.2025 |
|---|
| Datum sprejetja članka: | 07.07.2025 |
|---|
| Datum objave: | 08.07.2025 |
|---|
| Založnik: | MDPI AG |
|---|
| Leto izida: | 2025 |
|---|
| Št. strani: | 7666-7689 |
|---|
| Številčenje: | let. 15, št. 14 |
|---|
| PID: | 20.500.12556/DKUM-93644  |
|---|
| UDK: | 004.8 |
|---|
| eISSN: | 2076-3417 |
|---|
| COBISS.SI-ID: | 243245571 |
|---|
| DOI: | 10.3390/app15147666 |
|---|
| ISSN pri članku: | 2076-3417 |
|---|
| Avtorske pravice: | © 2025 by the authors |
|---|
| Datum objave v DKUM: | 21.07.2025 |
|---|
| Število ogledov: | 220 |
|---|
| Število prenosov: | 18 |
|---|
| Metapodatki: |    |
|---|
| Področja: | Ostalo
|
|---|
|
:
|
Kopiraj citat |
|---|
| | | | Skupna ocena: | (0 glasov) |
|---|
| Vaša ocena: | Ocenjevanje je dovoljeno samo prijavljenim uporabnikom. |
|---|
| Objavi na: |  |
|---|
Postavite miškin kazalec na naslov za izpis povzetka. Klik na naslov izpiše
podrobnosti ali sproži prenos. |
