Related Experiment Video
Updated: Jan 13, 2026
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
1.0K
An autonomous agentic workflow for clinical detection of cognitive concerns using large language models.
Jiazi Tian1, Pedram Fard1, Cameron Cagan1
1Department of Medicine, Massachusetts General Hospital, Boston, MA, USA.
NPJ Digital Medicine
|January 7, 2026
Summary
We developed AI workflows to detect cognitive impairment from clinical notes. Autonomous AI agents achieved expert-level performance, offering scalable clinical decision support for early detection.
Area of Science:
- Artificial Intelligence
- Clinical Informatics
- Cognitive Science
Background:
- Traditional cognitive impairment screening faces limitations in accessibility and resource allocation.
- Early detection is crucial for timely intervention and management of cognitive decline.
- Large language models (LLMs) offer potential for analyzing clinical notes to identify cognitive concerns.
Purpose of the Study:
- To develop and compare two LLM workflows for identifying cognitive concerns in clinical notes.
- To evaluate the performance of an expert-driven workflow versus an autonomous agentic workflow.
- To assess the impact of prevalence shifts on model generalizability and the clinical relevance of discrepancies.
Main Methods:
- Developed an expert-driven LLM workflow using iterative prompt refinement across LLaMA 3.1, LLaMA 3.2, and Med42 v2.
- Created an autonomous agentic workflow with five specialized agents for prompt optimization.
- Optimized models on a balanced dataset and validated on an independent dataset reflecting real-world prevalence.
- Conducted expert re-adjudication to analyze false negatives and assess clinical reasoning.
Main Results:
- The autonomous agentic workflow achieved comparable validation performance (F1=0.74) to the expert-driven workflow (F1=0.81).
- The agentic workflow demonstrated superior refinement results (0.93) compared to the expert-driven workflow (0.87).
- Model sensitivity decreased from 0.91 to 0.62 between the refinement and validation datasets due to prevalence shifts.
- Expert re-adjudication found 44% of apparent false negatives represented clinically appropriate reasoning.
Conclusions:
- Autonomous agentic systems can achieve expert-level performance in identifying cognitive concerns from clinical notes.
- These AI systems offer scalable and interpretable clinical decision support for early cognitive impairment detection.
- Model generalizability is significantly impacted by shifts in data prevalence, necessitating careful validation strategies.
More Related Videos
Related Concept Videos
01:10Higher Mental Functions of the Brain: Language
3.4K
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
3.4K
01:27Language and Cognition
704
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
704
01:21Cognitive Learning
997
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
997
01:25Cognitive Therapy
877
Cognitive therapy, pioneered by Aaron T. Beck in the 1960s, is a structured approach to addressing psychological distress by focusing on the influence of thoughts on emotions and behaviors. All cognitive therapies involve the basic assumption that human beings have control over their feelings, and that how individuals feel about something depends on how they think about it. Unlike psychoanalytic methods that delve into unconscious processes or humanistic approaches emphasizing...
877
01:22Language Development
831
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
831
01:19High-Level and Low-Level Awareness
623
Controlled processes in human consciousness represent high-alert mental states where individuals deliberately focus their attention on achieving specific goals. Controlled processes can be seen in situations like mastering new technology, where a person might become so absorbed that they ignore surrounding distractions. Such processes involve selective attention, requiring one to concentrate on particular elements of experience while disregarding others. These are governed by executive...
623