| Title: | Differences in user perception of artificial intelligence-driven chatbots and traditional tools in qualitative data analysis |
|---|
| Authors: | ID Šumak, Boštjan (Author)
ID Pušnik, Maja (Author)
ID Kožuh, Ines (Author)
ID Šorgo, Andrej (Author)
ID Brdnik, Saša (Author) |
|---|
| Files: |  applsci-15-00631-v2.pdf (1,51 MB)
MD5: 23D038F57EA23A9D23395C6DAD18E6E5
|
|---|
| Language: | English |
|---|
| Work type: | Article |
|---|
| Typology: | 1.01 - Original Scientific Article |
|---|
| Organization: | FERI - Faculty of Electrical Engineering and Computer Science
|
|---|
| Abstract: | Qualitative data analysis (QDA) tools are essential for extracting insights from complex datasets. This study investigates researchers’ perceptions of the usability, user experience (UX), mental workload, trust, task complexity, and emotional impact of three tools: Taguette 1.4.1 (a traditional QDA tool), ChatGPT (GPT-4, December 2023 version), and Gemini (formerly Google Bard, December 2023 version). Participants (N = 85), Master’s students from the Faculty of Electrical Engineering and Computer Science with prior experience in UX evaluations and familiarity with AI-based chatbots, performed sentiment analysis and data annotation tasks using these tools, enabling a comparative evaluation. The results show that AI tools were associated with lower cognitive effort and more positive emotional responses compared to Taguette, which caused higher frustration and workload, especially during cognitively demanding tasks. Among the tools, ChatGPT achieved the highest usability score (SUS = 79.03) and was rated positively for emotional engagement. Trust levels varied, with Taguette preferred for task accuracy and ChatGPT rated highest in user confidence. Despite these differences, all tools performed consistently in identifying qualitative patterns. These findings suggest that AI-driven tools can enhance researchers’ experiences in QDA while emphasizing the need to align tool selection with specific tasks and user preferences. |
|---|
| Keywords: | user experience, UX, usability, qualitative data analysis, QDA, chatbots |
|---|
| Publication status: | Published |
|---|
| Publication version: | Version of Record |
|---|
| Submitted for review: | 29.12.2024 |
|---|
| Article acceptance date: | 06.01.2025 |
|---|
| Publication date: | 10.01.2025 |
|---|
| Publisher: | MDPI |
|---|
| Year of publishing: | 2025 |
|---|
| Number of pages: | 37 str. |
|---|
| Numbering: | Vol. 15, iss. 2, [article no.] 631 |
|---|
| PID: | 20.500.12556/DKUM-91797  |
|---|
| UDC: | 004.5 |
|---|
| ISSN on article: | 2076-3417 |
|---|
| COBISS.SI-ID: | 222358787 |
|---|
| DOI: | 10.3390/app15020631 |
|---|
| Copyright: | © 2025 by the authors |
|---|
| Publication date in DKUM: | 07.02.2025 |
|---|
| Views: | 206 |
|---|
| Downloads: | 29 |
|---|
| Metadata: |    |
|---|
| Categories: | Misc.
|
|---|
|
:
|
Copy citation |
|---|
| | | | Average score: | (0 votes) |
|---|
| Your score: | Voting is allowed only for logged in users. |
|---|
| Share: |  |
|---|
Hover the mouse pointer over a document title to show the abstract or click
on the title to get all document metadata. |
