| Title: | A brief review on benchmarking for large language models evaluation in healthcare |
|---|
| Authors: | ID Cilar Budler, Leona (Author)
ID Chen, Hongyu (Author)
ID Chen, Aokun (Author)
ID Topaz, Maxim (Author)
ID Tam, Wilson (Author)
ID Bian, Jiang (Author)
ID Štiglic, Gregor (Author) |
|---|
| Files: |  https://wires.onlinelibrary.wiley.com/doi/10.1002/widm.70010
 WIREs-Data-Min-Knowl---2025---Budler---A-Brief-Review-on-Ben.pdf (943,83 KB)
MD5: 132DFD2ED2C2E99251CEF9B5CEE42FF8
|
|---|
| Language: | English |
|---|
| Work type: | Scientific work |
|---|
| Typology: | 1.02 - Review Article |
|---|
| Organization: | FZV - Faculty of Health Sciences
FERI - Faculty of Electrical Engineering and Computer Science
|
|---|
| Abstract: | This paper reviews benchmarking methods for evaluating large language models (LLMs) in healthcare settings. It highlights the importance of rigorous benchmarking to ensure LLMs' safety, accuracy, and effectiveness in clinical applications. The review also discusses the challenges of developing standardized benchmarks and metrics tailored to healthcare-specific tasks such as medical text generation, disease diagnosis, and patient management. Ethical considerations, including privacy, data security, and bias, are also addressed, underscoring the need for multidisciplinary collaboration to establish robust benchmarking frameworks that facilitate LLMs' reliable and ethical use in healthcare. Evaluation of LLMs remains challenging due to the lack of standardized healthcare-specific benchmarks and comprehensive datasets. Key concerns include patient safety, data privacy, model bias, and better explainability, all of which impact the overall trustworthiness of LLMs in clinical settings. |
|---|
| Keywords: | artificial intelligence, benchmarking, chatbots, healthcare, large language models, natural language processing |
|---|
| Publication status: | Published |
|---|
| Publication version: | Version of Record |
|---|
| Submitted for review: | 17.02.2025 |
|---|
| Article acceptance date: | 11.03.2025 |
|---|
| Publication date: | 01.01.2025 |
|---|
| Publisher: | Wiley |
|---|
| Year of publishing: | 2025 |
|---|
| Number of pages: | str. 1-16 |
|---|
| Numbering: | Letn. 15, išt. 2, št. članka e70010 |
|---|
| PID: | 20.500.12556/DKUM-92741  |
|---|
| UDC: | 004.89:614 |
|---|
| ISSN on article: | 1942-4795 |
|---|
| COBISS.SI-ID: | 234990851 |
|---|
| DOI: | 10.1002/widm.70010 |
|---|
| Publication date in DKUM: | 12.05.2025 |
|---|
| Views: | 184 |
|---|
| Downloads: | 21 |
|---|
| Metadata: |    |
|---|
| Categories: | Misc.
|
|---|
|
:
|
Copy citation |
|---|
| | | | Average score: | (0 votes) |
|---|
| Your score: | Voting is allowed only for logged in users. |
|---|
| Share: |  |
|---|
Hover the mouse pointer over a document title to show the abstract or click
on the title to get all document metadata. |
