| | SLO | ENG | Piškotki in zasebnost

Večja pisava | Manjša pisava

Izpis gradiva Pomoč

Naslov:Exploration of augmentation strategies in multi-modal retrieval-augmented generation for the biomedical domain : a case study evaluating question answering in glycobiology
Avtorji:ID Kocbek, Primož (Avtor)
ID Frkatović-Hodžić, Azra (Avtor)
ID Lalić, Dora (Avtor)
ID Hui, Vivian (Avtor)
ID Lauc, Gordan (Avtor)
ID Štiglic, Gregor (Avtor)
ID IEEE (Lastnik avtorskih pravic)
Datoteke:.pdf IEEE_BigData25_MMAI_Kocbek_etal.pdf (232,82 KB)
MD5: 33355984EC6223B01395A2EA13F9D69C
 
URL https://ieeexplore.ieee.org/document/11401037
 
Jezik:Angleški jezik
Vrsta gradiva:Znanstveno delo
Tipologija:1.08 - Objavljeni znanstveni prispevek na konferenci
Organizacija:FZV - Fakulteta za zdravstvene vede
Opis:Multi-modal retrieval-augmented generation (MMRAG) promises grounded biomedical QA, but it is unclear when to (i) convert figures/tables into text versus (ii) use optical character recognition (OCR)-free visual retrieval that returns page images and leaves interpretation to the generator. We study this trade-off in glycobiology, a visually dense domain. We built a benchmark of 120 multiple-choice questions (MCQs) from 25 papers, stratified by retrieval difficulty (easy text, medium figures/tables, hard cross-evidence). We implemented four augmentations—None, Text RAG, Multi-modal conversion, and late-interaction visual retrieval (ColPali)—using Docling parsing and Qdrant indexing. We evaluated mid-size opensource and frontier proprietary models (e.g., Gemma-3-27BIT, GPT-4o family). Additional testing used the GPT-5 family and multiple visual retrievers (ColPali/ColQwen/ColFlor). Accuracy with Agresti–Coull 95% confidence intervals (CIs) was computed over 5 runs per configuration. With Gemma-3-27BIT, Text and Multi-modal augmentation outperformed OCR-free retrieval (0.722-0.740 vs. 0.510 average accuracy). With GPT-4o, Multi-modal achieved 0.808, with Text 0.782 and ColPali 0.745 close behind; within-model differences were small. In follow-on experiments with the GPT-5 family, the best results with ColPali and ColFlor improved by 2% to 0.828 in both cases. In general across the GPT-5 family, ColPali, ColQwen, and ColFlor were statistically indistinguishable; ColFlor matched ColPali while being far smaller. GPT-5-nano trailed larger GPT-5 variants by roughly 8-10%. Pipeline choice is capacity-dependent: converting visuals to text lowers the reader burden and is more reliable for mid-size models, whereas OCR-free visual retrieval becomes competitive under frontier models. Among retrievers, ColFlor offers parity with heavier options at a smaller footprint, making it an efficient default when strong generators are available.
Ključne besede:multimodal retrieval-augmentation generation, evaluation, large language models
Status publikacije:Posredovano založniku
Leto izida:2026
Št. strani:Str. 5156-5165
PID:20.500.12556/DKUM-97620 Novo okno
UDK:004.89+37.011.22
COBISS.SI-ID:272777987 Novo okno
DOI:10.1109/BigData66926.2025.11401037 Novo okno
Datum objave v DKUM:25.03.2026
Število ogledov:219
Število prenosov:11
Metapodatki:XML DC-XML DC-RDF
Področja:Ostalo
:
Kopiraj citat
  
Skupna ocena:(0 glasov)
Vaša ocena:Ocenjevanje je dovoljeno samo prijavljenim uporabnikom.
Objavi na:Bookmark and Share



Postavite miškin kazalec na naslov za izpis povzetka. Klik na naslov izpiše podrobnosti ali sproži prenos.

Gradivo je del monografije

Naslov:Proceedings : 2025 IEEE International Conference on Big Data (BigData)
Kraj izida:[Piscataway]
Založnik:IEEE
ISBN:979-8-3315-9447-3
COBISS.SI-ID:272757763 Novo okno

Gradivo je financirano iz projekta

Financer:EC - European Commission
Program financ.:HE
Številka projekta:101159018
Naslov:Synergy for Healthy Longevity
Akronim:SynHealth

Financer:EC - European Commission
Številka projekta:101101903
Naslov:Slovenian AI Factory
Akronim:SLAIF

Financer:ARIS - Javna agencija za znanstvenoraziskovalno in inovacijsko dejavnost Republike Slovenije
Številka projekta:GC-0001
Naslov:Umetna inteligenca za znanost

Licence

Licenca:Drugo
Opis:COPYRIGHT TRANSFER The undersigned hereby assigns to The Institute of Electrical and Electronics Engineers, Incorporated (the "IEEE") all rights under copyright that may exist in and to: (a) the Work, including any revised or expanded derivative works submitted to the IEEE by the undersigned based on the Work; and (b) any associated written or multimedia components or other enhancements accompanying the Work. GENERAL TERMS The undersigned represents that he/she has the power and authority to make and execute this form.1. The undersigned agrees to indemnify and hold harmless the IEEE from any damage or expense that may arise in the event of a breach of any of the warranties set forth above. 2. The undersigned agrees that publication with IEEE is subject to the policies and procedures of the IEEE PSPB Operations Manual.3. In the event the above work is not accepted and published by the IEEE or is withdrawn by the author(s) before acceptance by the IEEE, the foregoing copyright transfer shall be null and void. In this case, IEEE will retain a copy of the manuscript for internal administrative/record-keeping purposes. 4. For jointly authored Works, all joint authors should sign, or one of the authors should sign as authorized agent for the others.5. The author hereby warrants that the Work and Presentation (collectively, the "Materials") are original and that he/she is the author of the Materials. To the extent the Materials incorporate text passages, figures, data or other material from the works of others, the author has obtained any necessary permissions. Where necessary, the author has obtained all third party permissions and consents to grant the license above and has provided copies of such permissions and consents to IEEE 6. You have indicated that you DO wish to have video/audio recordings made of your conference presentation under terms and conditions set forth in "Consent and Release." CONSENT AND RELEASE ln the event the author makes a presentation based upon the Work at a conference hosted or sponsored in whole or in part by the IEEE, the author, in consideration for his/her participation in the conference, hereby grants the IEEE the unlimited, worldwide, irrevocable permission to use, distribute, publish, license, exhibit, record, digitize, broadcast, reproduce and archive, in any format or medium, whether now known or hereafter developed: (a) his/her presentation and comments at the conference; (b) any written materials or multimedia files used in connection with his/her presentation; and (c) any recorded interviews of him/her (collectively, the "Presentation"). The permission granted includes the transcription and reproduction of the Presentation for inclusion in products sold or distributed by IEEE and live or recorded broadcast of the Presentation during or after the conference. In connection with the permission granted in Section 1, the author hereby grants IEEE the unlimited, worldwide, irrevocable right to use his/her name, picture, likeness, voice and biographical information as part of the advertisement, distribution and sale of products incorporating the Work or Presentation, and releases IEEE from any claim based on right of privacy or publicity.
Začetek licenciranja:20.11.2025
Vezano na:P. Kocbek, A. Frkatović-Hodžić, D. Lalić, V. Hui, G. Lauc and G. Štiglic, "Exploration of Augmentation Strategies in Multi-Modal Retrieval-Augmented Generation for the Biomedical Domain*: *A Case Study Evaluating Question Answering in Glycobiology," 2025 IEEE International Conference on Big Data (BigData), Macau, China, 2025, pp. 5156-5165, doi: 10.1109/BigData66926.2025.11401037.

Sekundarni jezik

Jezik:Slovenski jezik
Ključne besede:večmodalno pridobivanje-povečanje generacije, vrednotenje, veliki jezikovni modeli


Komentarji

Dodaj komentar

Za komentiranje se morate prijaviti.

Komentarji (0)
0 - 0 / 0
 
Ni komentarjev!

Nazaj
Logotipi partnerjev Univerza v Mariboru Univerza v Ljubljani Univerza na Primorskem Univerza v Novi Gorici