20.500.12556/DKUM-97620 Exploration of augmentation strategies in multi-modal retrieval-augmented generation for the biomedical domain a case study evaluating question answering in glycobiology Multi-modal retrieval-augmented generation (MMRAG) promises grounded biomedical QA, but it is unclear when to (i) convert figures/tables into text versus (ii) use optical character recognition (OCR)-free visual retrieval that returns page images and leaves interpretation to the generator. We study this trade-off in glycobiology, a visually dense domain. We built a benchmark of 120 multiple-choice questions (MCQs) from 25 papers, stratified by retrieval difficulty (easy text, medium figures/tables, hard cross-evidence). We implemented four augmentations—None, Text RAG, Multi-modal conversion, and late-interaction visual retrieval (ColPali)—using Docling parsing and Qdrant indexing. We evaluated mid-size opensource and frontier proprietary models (e.g., Gemma-3-27BIT, GPT-4o family). Additional testing used the GPT-5 family and multiple visual retrievers (ColPali/ColQwen/ColFlor). Accuracy with Agresti–Coull 95% confidence intervals (CIs) was computed over 5 runs per configuration. With Gemma-3-27BIT, Text and Multi-modal augmentation outperformed OCR-free retrieval (0.722-0.740 vs. 0.510 average accuracy). With GPT-4o, Multi-modal achieved 0.808, with Text 0.782 and ColPali 0.745 close behind; within-model differences were small. In follow-on experiments with the GPT-5 family, the best results with ColPali and ColFlor improved by 2% to 0.828 in both cases. In general across the GPT-5 family, ColPali, ColQwen, and ColFlor were statistically indistinguishable; ColFlor matched ColPali while being far smaller. GPT-5-nano trailed larger GPT-5 variants by roughly 8-10%. Pipeline choice is capacity-dependent: converting visuals to text lowers the reader burden and is more reliable for mid-size models, whereas OCR-free visual retrieval becomes competitive under frontier models. Among retrievers, ColFlor offers parity with heavier options at a smaller footprint, making it an efficient default when strong generators are available. multimodal retrieval-augmentation generation evaluation large language models večmodalno pridobivanje-povečanje generacije vrednotenje veliki jezikovni modeli true true true Angleški jezik Slovenski jezik Znanstveno delo 2026-03-25 14:21:27 2026-03-25 15:15:46 2026-03-26 03:08:06 0000-00-00 00:00:00 2026 0 0 Str. 5156-5165 0000-00-00 NiDoloceno PosredovanoVObjavo NiDoloceno 0000-00-00 0000-00-00 0000-00-00 004.89+37.011.22 272757763 272777987 10.1109/BigData66926.2025.11401037 IEEE_BigData25_MMAI_Kocbek_etal.pdf IEEE_BigData25_MMAI_Kocbek_etal.pdf 1 33355984EC6223B01395A2EA13F9D69C 73dce105f75c8e080f048cef9869f928ceba086bdd35bdad3abe75493b40ddac bfdd956b-284d-11f1-bbad-00155d0001a1 https://dk.um.si/Dokument.php?lang=slv&id=204786 https://ieeexplore.ieee.org/document/11401037 1 04546afc-284e-11f1-bbad-00155d0001a1 https://dk.um.si/Dokument.php?lang=slv&id=204788 Fakulteta za zdravstvene vede 0 0 0