| Naslov: | Human-led and artificial intelligence-automated critical appraisal of systematic reviews : comparative evaluation |
|---|
| Avtorji: | ID Gosak, Lucija (Avtor)
ID Štiglic, Gregor (Avtor)
ID Tam, Wilson (Avtor)
ID Vrbnjak, Dominika (Avtor) |
|---|
| Datoteke: |  1-s2.0-S1471595325003713-main.pdf (651,99 KB)
MD5: 4F774DD00C5B15851F78DA6A546E2FDF
 https://www.sciencedirect.com/science/article/pii/S1471595325003713
|
|---|
| Jezik: | Angleški jezik |
|---|
| Vrsta gradiva: | Znanstveno delo |
|---|
| Tipologija: | 1.01 - Izvirni znanstveni članek |
|---|
| Organizacija: | FZV - Fakulteta za zdravstvene vede
FERI - Fakulteta za elektrotehniko, računalništvo in informatiko
|
|---|
| Opis: | Aim To evaluate and compare human-led and artificial intelligence-automated critical appraisal of evidence. Background Critical appraisal is essential in evidence-based practice, yet many nurses lack the skills to perform it. Large language models offer potential support, but their role in critical appraisal remains underexplored. Design We conducted a comparative study to evaluate the performance of five commonly used large language models versus two human reviewers in appraising four systematic reviews on interventions to reduce medication administration errors. Methods We compared large language models and two human reviewers in independently appraising four systematic reviews using the JBI Critical Appraisal Checklist. These models were Perplexity Sonar (Pro), Claude 3.7 Sonnet, Gemini 2.0 Flash, GPT-4.5 and Grok-2. All models received identical full texts and standardized prompts. Responses were analyzed descriptively and agreement was assessed using Cohen’s Kappa. Results Large language models showed full agreement with human reviewers on five of 11 JBI items. Most disagreements occurred in appraising search strategy, inclusion criteria and publication bias. The agreement between human reviewers and large language models ranged from slight to moderate. The highest level of agreement was observed with Claude (κ = 0.732), while the lowest level was observed with Gemini (κ = 0.394). Conclusion Large language models can support aspects of critical appraisal evidence but lack contextual reasoning and methodological insight required for complex judgments. While Claude 3.7 Sonnet aligned most closely with human reviewers, human oversight remains essential. Large language models should serve as adjuncts and not substitutes for evidence-based practice. |
|---|
| Ključne besede: | artificial intelligence in healthcare, multimodal large language models, nursing, evidence-based practice |
|---|
| Status publikacije: | Objavljeno |
|---|
| Verzija publikacije: | Objavljena publikacija |
|---|
| Poslano v recenzijo: | 30.05.2025 |
|---|
| Datum sprejetja članka: | 27.10.2025 |
|---|
| Datum objave: | 28.10.2025 |
|---|
| Založnik: | Elsevier |
|---|
| Leto izida: | 2025 |
|---|
| Št. strani: | str. 1-6 |
|---|
| Številčenje: | Letn. 89, št. članka 104614 |
|---|
| PID: | 20.500.12556/DKUM-95932  |
|---|
| UDK: | 616-083:004.8 |
|---|
| COBISS.SI-ID: | 256810243 |
|---|
| DOI: | 10.1016/j.nepr.2025.104614 |
|---|
| ISSN pri članku: | 1873-5223 |
|---|
| Avtorske pravice: | © 2025 The Authors. Published by Elsevier Ltd. |
|---|
| Datum objave v DKUM: | 12.11.2025 |
|---|
| Število ogledov: | 202 |
|---|
| Število prenosov: | 9 |
|---|
| Metapodatki: |    |
|---|
| Področja: | Ostalo
|
|---|
|
:
|
Kopiraj citat |
|---|
| | | | Skupna ocena: | (0 glasov) |
|---|
| Vaša ocena: | Ocenjevanje je dovoljeno samo prijavljenim uporabnikom. |
|---|
| Objavi na: |  |
|---|
Postavite miškin kazalec na naslov za izpis povzetka. Klik na naslov izpiše
podrobnosti ali sproži prenos. |
