Weaponizing cognitive bias in autonomous systems: a framework for black-box inference attacks.

Area of Science:

• Artificial Intelligence
• Cognitive Science
• Autonomous Systems

Background:

• Autonomous systems in high-dimensional environments use prioritization heuristics for attention and risk assessment.
• These heuristics can introduce cognitive biases (salience, spatial framing, temporal familiarity) affecting decision-making.
• Limited accessibility of embodied vision systems necessitates alternative diagnostic methods.

Purpose of the Study:

• To introduce Priority Inversion via Operational Reasoning (PRIOR), a black-box diagnostic framework.
• To probe inference-level vulnerabilities in autonomous systems using structurally biased, semantically neutral cues.
• To evaluate PRIOR using large language models (LLMs) as proxies for cognitive prioritization in simulated surveillance scenarios.

Main Methods:

• Developed PRIOR, a non-perturbative diagnostic framework.
• Employed structurally biased but semantically neutral scenario cues.
• Utilized LLMs as abstract reasoning proxies for simulated Unmanned Aerial Vehicle (UAV) surveillance.

Main Results:

• Minimal structural cues consistently induced priority inversions across multiple LLMs.
• Analysis of justifications and confidence estimates revealed systematic distortions in inferred threat relevance.
• Symmetrical inputs did not prevent these distortions in prioritization logic.

Conclusions:

• Inference-level reasoning in black-box systems is fragile.
• Evaluation strategies must interrogate internal prioritization logic, not just output correctness.
• Findings have implications for dynamic, embodied, and visually grounded agents in real-world deployments.