A local, privacy-oriented multi-agent LLM framework for framework-grounded manuscript editing: A proof-of-concept

Area of Science:

• Artificial Intelligence
• Computational Linguistics
• Scientific Publishing

Background:

• Manuscript preparation is a significant bottleneck in scientific publishing.
• Cloud-based AI tools pose confidentiality risks for sensitive clinical research data.
• There is a need for privacy-preserving AI solutions for manuscript auditing.

Purpose of the Study:

• To develop and evaluate the Paper Analysis Tool (PAT), a free, multi-agent framework for local manuscript auditing.
• To assess PAT's ability to enhance manuscript quality using a local open-weight language model.
• To compare PAT's performance against a generic prompt and a frontier model.

Main Methods:

• PAT utilizes 31 components, including a deterministic text-metrics module and 30 language-model agents, all running locally.
• Six manuscripts were audited using PAT, the same local model with a generic prompt, and a frontier model (Claude Sonnet 5).
• Suggestions were pooled, anonymized, and blindly scored by co-authors for actionability and usefulness across 15 text and 5 figure quality domains.

Main Results:

• PAT, orchestrated with a local 27B model, covered 11.3/15 useful domains, outperforming the same model with a generic prompt (5.5) and approaching the frontier benchmark (9.3).
• The local model within PAT exceeded its one-prompt counterpart in every paper and after matching.
• PAT's rewrites significantly reduced passive-voice sentences (51% to 6%), long sentences (73%), and word count (22%).

Conclusions:

• The Paper Analysis Tool (PAT) demonstrates the feasibility of local, privacy-preserving multi-agent manuscript review.
• PAT reliably broadens the quality-relevant domains addressed by local open-weight language models.
• While PAT enhances breadth, suggestion usefulness is model-dependent; future iterations can pair PAT with stronger base models.