Toward advanced management of sludge bulking: An explainable evaluation framework to quantify and improve large...

Activated sludge bulking is a globally prevalent operational failure in wastewater treatment plants (WWTPs), which demands timely, accurate human intervention, but the scarcity of specialist wastewater expertise poses a considerable challenge. Large language models (LLMs) potentially support bulking management by providing human-like cognitive capabilities. However, there is no explainable evaluation framework to quantify LLMs' cognitive capabilities' strengths and limits, and, crucially, inform how these capabilities can be improved for advanced sludge bulking management. Here, we developed an explainable evaluation framework, by establishing an open-access evaluation suite (280 tasks including real-world cases) with an LLM-rater (with ground truth in prompts to ensure evaluation reliability), to diagnose LLMs' cognitive capabilities of understanding, analysis, and practice. Under this framework, we found that performance was dominated by task type rather than LLM identity (DeepSeek-R1, GPT-4o and Gemini-2.0). Concretely, LLMs generally showed limited understanding capability on microscopy-based recognition (accuracy<54%) and microbial identification (normalized-scores:28-42%), and weak analysis capability for process-microbe interactions (normalized-scores:36-52%). Practice tasks were also challenging, with deficits across all cognitive capabilities, and had highest token demand among all task types. Guided by this explainable framework, we integrated a vision module with LLMs to improve visual understanding of key filaments, including Beggiatoa, fungi, and Nocardia (accuracy from 51% to 78%). In parallel, retrieval-augmented generation (RAG) was further employed to selectively enhance practice capability, with DeepSeek-R1, a thinking model, showing the largest gains in practice capability (69%). The RAG-enhanced DeepSeek-R1 (i.e., FilamentGPT) achieved the best overall performance (normalized-scores>80% for all, except microbial identification), and demonstrated practical feasibility for WWTP decision-support based on perspectives of 20 WWTP engineers. We attribute this advantage to its stronger ability to integrate retrieved knowledge into reasonable, plant-specific workflows. Collectively, this work established an explainable evaluation framework for LLMs in sludge bulking management, constituting a science foundation for LLMs evaluation. Informed by explainable framework, we also outlined (post-)training pathways (e.g., RAG, vision module) to selectively enhance cognitive capabilities of LLMs, which could democratize subspecialist-level expertise for wastewater treatment in WWTPs.