Related Experiment Video
Updated: Feb 24, 2026
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
1.2K
To what Degree can LLMs Support Medical Informatics Research? Examining the Interplay of Research Support LLMs with
Naren Khatwani1, Lijing Wang1, James Geller1
1New Jersey Institute of Technology, Newark, New Jersey, USA.
AMIA ... Annual Symposium Proceedings. AMIA Symposium
|February 23, 2026
Summary
Large Language Models (LLMs) can generate research plans and critiques in Medical Informatics. LLM-generated feedback improves these plans, demonstrating their potential to support scientific research.
Area of Science:
- Medical Informatics
- Artificial Intelligence in Research
Background:
- Large Language Models (LLMs) are rapidly advancing.
- LLMs present new opportunities for research support.
Purpose of the Study:
- To assess LLM capability in generating "thoughtful" research plans.
- To evaluate LLM-generated critiques for improving research plans.
Main Methods:
- Four LLMs generated primary research plans.
- Plans underwent mutual critique and LLM-based refinement.
- Human evaluators assessed original and improved responses.
- ROUGE scores and cosine similarity quantified response similarities.
Main Results:
- LLMs produced varied outputs, with differences between primary and refined plans.
- All LLMs generated coherent plans and critiques.
- LLMs effectively integrated feedback for improved outputs.
- Human evaluators could distinguish between initial and refined plans.
Conclusions:
- LLMs demonstrate potential in generating and refining research plans.
- LLM-generated critiques can enhance research plan quality.
- Further research is needed to explore LLM applications in scientific research.
