Interactive and inspectable LLM planning for human-in-the-loop EV battery disassembly

Intent-driven robotic disassembly requires translating operator language and noisy RGB-D perception into grounded, precedence-consistent removal plans. This work extends a perception-grounded large language model (LLM) planning framework for two-arm electric vehicle (EV) battery disassembly and evaluates the pipeline's effectiveness after replacing the LLM backbone.

The system converts live RGB-D detections into structured scene text, rewrites coordinates into a frame-consistent LUZD representation, generates an ordered removal sequence, validates it with an LLM verifier, and applies a deterministic anti-hallucination filter that removes labels or coordinates not present in the perceived scene. The replacement LLM is first evaluated within the prior ensemble-based pipeline. The upgraded system then combines a single-checkpoint generator-verifier workflow, revised generator and verifier prompts, and an interactive, inspectable human-in-the-loop planning interface that displays intermediate outputs and final filtered plans.

On the same 200-scene, 600-intent benchmark used in earlier work, mean full-list correctness was 0.824 for the prior six-sample final pipeline, 0.896 when the replacement LLM was evaluated within that prior pipeline, 0.935 with the upgraded generator alone, and 0.959 with verification and filtering. The final upgraded configuration reduced the LLM-call count from seven to two. In a 12-participant within-subject pilot, mean time to accepted answer was lower on both tasks, the mean raw NASA-TLX score was lower, and all participants preferred the new interface.

These results suggest that structured scene text, clear prompts, verifier-based correction, deterministic grounding, and inspectable interfaces enhance human-in-the-loop disassembly planning.