Peer Reviewed Article

Usability and Adoption in a Randomized Trial of GutGPT a GenAI Tool for Gastrointestinal Bleeding

Authors
  • Colleen Chan
  • Sunny Chung
  • Leigh Evans
  • Mauro Giuffre
  • Allen Hsiao
  • Rene F. Kizilcec
  • Simone Kresevic
  • Loren Laine
  • Terika McCall
  • Shinpei Nakamura-Sakai
  • Yuan Pu
  • Niroop Rajashekar
  • Theo Saarinen
  • Jasjeet Sekhon
  • Yeo Eun Shin
  • Dennis L. Shung
  • Ambrose H. Wong
  • Kisung You
Published
August 18, 2025
Publication
npj Digital Medicine
Discipline
Areas of Study
Document Control Number(s)
  • ISPS 25-47
Citation

Chung, S., Giuffrè, M., Rajashekar, N. et al. Usability and adoption in a randomized trial of GutGPT a GenAI tool for gastrointestinal bleeding. npj Digit. Med. 8, 527 (2025).

Abstract

Generative AI (GenAI) may enhance clinical decision support systems (CDSS), but its impact on adoption remains unclear. We conducted a simulation-based randomized trial to evaluate whether a GenAI-enhanced CDSS, “GutGPT,” improves adoption compared to an AI dashboard in acute upper gastrointestinal bleeding management. Clinical trainees were randomized to either GutGPT or a comparator dashboard across three cases. The primary outcome was Behavioral Intention, from the Unified Theory of Acceptance and Use of Technology (UTAUT). Secondary measures included additional constructs and decision accuracy. A total of 106 participants participated (52 GutGPT, 54 comparator). GutGPT users reported higher Effort Expectancy. Behavioral Intention had no significant difference. Qualitative analysis highlighted trust and workflow concerns. These findings suggest that usability alone is insufficient to drive adoption. As this study was conducted in a simulation without real-world integration or patient outcomes, further studies are needed. (Trial Registration: ClinicalTrials.gov; Identifier: NCT05816473; Registered March 6, 2023).

Description

Supplemental:

Full original article

Related Data:

The datasets generated and/or analyzed during the current study are not publicly available due to sensitive educational history data of participants. However, the minimal dataset necessary to interpret, replicate, and build upon the findings are available from the corresponding authors on request. The underlying code for this study is not publicly available per the IRB as individual level patient data is required for the code, but are available to qualified researchers on request to the corresponding author. All analyses were conducted using Python (v3.11) and the scipy 1.14.1, scikit-learn 1.6.1, pandas 2.2.3, and numpy 1.26.4 packages.