GrantCheck-an AI Solution for Guiding Grant Language to New Policy Requirements: Development Study

Area of Science:

• Health Policy
• Biomedical Research
• Artificial Intelligence in Research

Background:

• Academic institutions face grant writing challenges due to evolving language policies.
• Manual review is labor-intensive and can delay critical research submissions.
• Need for scalable, secure solutions to ensure compliance without compromising scientific integrity.

Purpose of the Study:

• Develop a secure, AI-powered tool to aid researchers in grant writing.
• Ensure grant proposals align with current federal and state policy requirements.
• Maintain scientific integrity and inclusivity during the grant writing process.

Main Methods:

• GrantCheck developed on a private AWS virtual private cloud.
• Integrated rule-based NLP with large language models via Amazon Bedrock.
• Hybrid pipeline detects flagged terms, suggests alternatives, and includes validation to prevent hallucinations.
• Assessed usability using the System Usability Scale.

Main Results:

• GrantCheck achieved high performance (F1=0.84) in detecting sensitive terms, outperforming general-purpose models (e.g., GPT-4o F1=0.43, Gemini 2.5 Pro F1=0.72).
• Usability testing with 25 faculty/staff yielded a high mean System Usability Scale score of 85.9 (SD 13.4).
• Demonstrated strong user satisfaction and workflow integration.

Conclusions:

• Institutionally hosted AI systems can support compliant, researcher-friendly grant writing.
• Safeguards public health research continuity by minimizing delays and funding losses from policy changes.
• Protects the research pipeline advancing biomedical discovery and health equity, especially in sensitive research areas.