Can LLMs like GPT-4 outperform traditional AI tools in dementia diagnosis? Maybe, but not today

by   Zhuo Wang, et al.

Recent investigations show that large language models (LLMs), specifically GPT-4, not only have remarkable capabilities in common Natural Language Processing (NLP) tasks but also exhibit human-level performance on various professional and academic benchmarks. However, whether GPT-4 can be directly used in practical applications and replace traditional artificial intelligence (AI) tools in specialized domains requires further experimental validation. In this paper, we explore the potential of LLMs such as GPT-4 to outperform traditional AI tools in dementia diagnosis. Comprehensive comparisons between GPT-4 and traditional AI tools are conducted to examine their diagnostic accuracy in a clinical setting. Experimental results on two real clinical datasets show that, although LLMs like GPT-4 demonstrate potential for future advancements in dementia diagnosis, they currently do not surpass the performance of traditional AI tools. The interpretability and faithfulness of GPT-4 are also evaluated by comparison with real doctors. We discuss the limitations of GPT-4 in its current state and propose future research directions to enhance GPT-4 in dementia diagnosis.


page 1

page 2

page 3

page 4


Lessons Learned from Designing an AI-Enabled Diagnosis Tool for Pathologists

Despite the promises of data-driven artificial intelligence (AI), little...

The Case Records of ChatGPT: Language Models and Complex Clinical Questions

Background: Artificial intelligence language models have shown promise i...

Sparks of Artificial General Intelligence: Early experiments with GPT-4

Artificial intelligence (AI) researchers have been developing and refini...

Investigating ChatGPT's Potential to Assist in Requirements Elicitation Processes

Natural Language Processing (NLP) for Requirements Engineering (RE) (NLP...

Benchmark datasets driving artificial intelligence development fail to capture the needs of medical professionals

Publicly accessible benchmarks that allow for assessing and comparing mo...

Please sign up or login with your details

Forgot password? Click here to reset