20.500.12556/DKUM-92856 LLM in the loop: a framework for contextualizing counterfactual segment perturbations in point clouds Point Cloud Data analysis has seen a major leap forward with the introduction of PointNet algorithms, revolutionizing how we process 3D environments. Yet, despite these advancements, key challenges remain, particularly in optimizing segment perturbations to influence model outcomes in a controlled and meaningful way. Traditional methods struggle to generate realistic and contextually appropriate perturbations, limiting their effectiveness in critical applications like autonomous systems and urban planning. This paper takes a bold step by integrating Large Language Models into the counterfactual reasoning process, unlocking a new level of automation and intelligence in segment perturbation. Our approach begins with semantic segmentation, after which LLMs intelligently select optimal replacement segments based on features such as class label, color, area, and height. By leveraging the reasoning capabilities of LLMs, we generate perturbations that are not only computationally efficient but also semantically meaningful. The proposed framework undergoes rigorous evaluation, combining human inspection of LLM-generated suggestions with quantitative analysis of semantic classification model performance across different LLM variants. By bridging the gap between geometric transformations and high-level semantic reasoning, this research redefines how we approach perturbation generation in Point Cloud Data analysis. The results pave the way for more interpretable, adaptable, and intelligent AI-driven solutions, bringing us closer to realworld applications where explainability and robustness are paramount. explainable AI point cloud data counterfactual reasoning LiDAR 3D point cloud data interpretability human-centered AI large language models K-nearest neighbors umetna inteligenca podatki v oblaku veliki jezikovni modeli true true false IEEE XPLORE Angleški jezik Slovenski jezik https://ieeexplore.ieee.org/document/10993377 Članek v reviji 2025-05-19 15:56:54 2025-05-19 15:56:54 2025-05-20 03:15:23 0000-00-00 00:00:00 2025 0 0 17 str. 2025 0000-00-00 Zaloznikova Objavljeno NiDoloceno 0000-00-00 0000-00-00 2025-05-08 004.7 236009987 10.1109/ACCESS.2025.3568052 2169-3536 LLM_in_the_Loop_A_Framework_for_Contextualizing_Counterfactual_Segment_Perturbations_in_Point_Clouds.pdf LLM_in_the_Loop_A_Framework_for_Contextualizing_Counterfactual_Segment_Perturbations_in_Point_Clouds.pdf 1 19B58061989BF92CB7C80EDBA041C477 7977b6833764282488e9e6fa6f9680b9cf9fe739ad64e13d0ee9dbc6bc120ad9 db527a01-34b9-11f0-ad3e-00155d000105 https://dk.um.si/Dokument.php?lang=slv&id=191115 Fakulteta za elektrotehniko, računalništvo in informatiko 0 0 0