| Naslov: | PICOT questions and search strategies formulation: a novel approach using artificial intelligence automation |
|---|
| Avtorji: | ID Gosak, Lucija (Avtor)
ID Štiglic, Gregor (Avtor)
ID Pruinelli, Lisiane (Avtor)
ID Vrbnjak, Dominika (Avtor) |
|---|
| Datoteke: |  J_of_Nursing_Scholarship_-_2024_-_Gosak_-_PICOT_questions_and_search_strategies_formulation__A_novel_approach_using.pdf (527,46 KB)
MD5: 8CAAD71B117D326B198DB5D794A9709A
 https://sigmapubs.onlinelibrary.wiley.com/doi/epdf/10.1111/jnu.13036
|
|---|
| Jezik: | Angleški jezik |
|---|
| Vrsta gradiva: | Znanstveno delo |
|---|
| Tipologija: | 1.01 - Izvirni znanstveni članek |
|---|
| Organizacija: | FZV - Fakulteta za zdravstvene vede
FERI - Fakulteta za elektrotehniko, računalništvo in informatiko
|
|---|
| Opis: | Aim
The aim of this study was to evaluate and compare artificial intelligence (AI)-based large language models (LLMs) (ChatGPT-3.5, Bing, and Bard) with human-based formulations in generating relevant clinical queries, using comprehensive methodological evaluations.
Methods
To interact with the major LLMs ChatGPT-3.5, Bing Chat, and Google Bard, scripts and prompts were designed to formulate PICOT (population, intervention, comparison, outcome, time) clinical questions and search strategies. Quality of the LLMs responses was assessed using a descriptive approach and independent assessment by two researchers. To determine the number of hits, PubMed, Web of Science, Cochrane Library, and CINAHL Ultimate search results were imported separately, without search restrictions, with the search strings generated by the three LLMs and an additional one by the expert. Hits from one of the scenarios were also exported for relevance evaluation. The use of a single scenario was chosen to provide a focused analysis. Cronbach's alpha and intraclass correlation coefficient (ICC) were also calculated.
Results
In five different scenarios, ChatGPT-3.5 generated 11,859 hits, Bing 1,376,854, Bard 16,583, and an expert 5919 hits. We then used the first scenario to assess the relevance of the obtained results. The human expert search approach resulted in 65.22% (56/105) relevant articles. Bing was the most accurate AI-based LLM with 70.79% (63/89), followed by ChatGPT-3.5 with 21.05% (12/45), and Bard with 13.29% (42/316) relevant hits. Based on the assessment of two evaluators, ChatGPT-3.5 received the highest score (M = 48.50; SD = 0.71). Results showed a high level of agreement between the two evaluators. Although ChatGPT-3.5 showed a lower percentage of relevant hits compared to Bing, this reflects the nuanced evaluation criteria, where the subjective evaluation prioritized contextual accuracy and quality over mere relevance.
Conclusion
This study provides valuable insights into the ability of LLMs to formulate PICOT clinical questions and search strategies. AI-based LLMs, such as ChatGPT-3.5, demonstrate significant potential for augmenting clinical workflows, improving clinical query development, and supporting search strategies. However, the findings also highlight limitations that necessitate further refinement and continued human oversight.
Clinical Relevance
AI could assist nurses in formulating PICOT clinical questions and search strategies. AI-based LLMs offer valuable support to healthcare professionals by improving the structure of clinical questions and enhancing search strategies, thereby significantly increasing the efficiency of information retrieval. |
|---|
| Ključne besede: | PICOT question, search strategies, artificial intelligence |
|---|
| Status publikacije: | Objavljeno |
|---|
| Verzija publikacije: | Objavljena publikacija |
|---|
| Poslano v recenzijo: | 31.01.2024 |
|---|
| Datum sprejetja članka: | 07.11.2024 |
|---|
| Datum objave: | 27.01.2025 |
|---|
| Leto izida: | 2025 |
|---|
| Št. strani: | str. 5-16 |
|---|
| Številčenje: | Letn. 57, št. 1 |
|---|
| PID: | 20.500.12556/DKUM-93712  |
|---|
| UDK: | 616-083:004.8 |
|---|
| COBISS.SI-ID: | 222504195 |
|---|
| DOI: | 10.1111/jnu.13036 |
|---|
| ISSN pri članku: | 1547-5069 |
|---|
| Avtorske pravice: | © 2024 The Author(s) |
|---|
| Datum objave v DKUM: | 21.07.2025 |
|---|
| Število ogledov: | 231 |
|---|
| Število prenosov: | 11 |
|---|
| Metapodatki: |    |
|---|
| Področja: | Ostalo
|
|---|
|
:
|
Kopiraj citat |
|---|
| | | | Skupna ocena: | (0 glasov) |
|---|
| Vaša ocena: | Ocenjevanje je dovoljeno samo prijavljenim uporabnikom. |
|---|
| Objavi na: |  |
|---|
Postavite miškin kazalec na naslov za izpis povzetka. Klik na naslov izpiše
podrobnosti ali sproži prenos. |
