Related Experiment Videos
SiBot: A hybrid framework for fine-grained semantic attribution of AI-generated text and its production deployment
Md Sadiq Iqbal1, Mohammod Abul Kashem1
1Department of Computer Science and Engineering, Dhaka University of Engineering and Technology, Gazipur, Bangladesh.
Abstract:
Large language models such as ChatGPT, Gemini, Claude and DeepSeek now generate text that is routinely indistinguishable from human writing, yet virtually every deployed detector answers only one binary question: human or AI. This is the wrong question for the settings that matter most. Digital forensics needs to know which model produced a text; academic-integrity adjudication needs to know whether a human was involved at all; and neither can be answered by a single probability, least of all for the hybrid human-AI drafts that now dominate real-world writing. This paper addresses that gap with SiBot, an end-to-end framework that reformulates AI-text detection as fine-grained multi-class source attribution and carries it through to public deployment. We construct two purpose-built corpora using a controlled bank of 3,000 prompts spanning six subject domains and five task types: a 66,000-sample corpus covering 22 AI sources, and a 30,000-sample corpus covering 10 classes that explicitly models five pure sources and four human-AI collaboration workflows; both are publicly released as the TXD-22 benchmark dataset. On the 22-class corpus we benchmark twelve feature-classifier combinations and find that TF-IDF with Random Forest attains 68.2% accuracy and an AUC of 0.742. On the 10-class corpus we benchmark fifteen classical, deep-learning and fine-tuned transformer baselines, and propose a hybrid architecture that fuses sparse TF-IDF lexical features with DeBERTa contextual embeddings at the feature level before Random Forest classification. The proposed hybrid attains 92.38% accuracy, 92.41% precision, 92.38% recall and 92.31% F1 under prompt-aware 10-fold cross-validation, a 17.73-point gain over the strongest fine-tuned transformer (DeBERTa, 74.65%) and a 34.30-point gain over TF-IDF with Random Forest alone, with non-overlapping 95% confidence intervals. An ablation confirms that both branches are necessary, and SHAP and LIME analyses expose the lexical evidence behind individual decisions. Because the 10-class hybrid is both substantially more accurate and interpretable, it is the model served by the live SiBot platform; the 22-class model is reported here as a research benchmark, and extending the deployment to all 22 classes is left to future work. In a controlled three-way benchmark against QuillBot, Turnitin, ZeroGPT and TextGuard, SiBot is the only system that names the generating model of AI text (Gemini, 96.7% confidence) and the only one that recognises hybrid provenance (97.7% confidence, against verdicts ranging from 0% to 76% among the commercial tools on the identical input).