Adaptive concept rule learning for interpretable and generalizable medical image classification

The pursuit of decision safety in clinical applications calls for medical image classification models that are not only accurate but also transparent enough for clinicians to audit the visual evidence and decision logic behind each diagnosis. Existing concept-based models ground predictions in human-interpretable concepts (i.e., diagnostic visual signs), but they typically provide only instance-level explanations without exposing the global classification logic, and their soft concept representations are prone to concept leakage, where unintended information undermines both interpretability and generalizability. We present Concept Rule Learner (CRL), a framework for medical image classification that learns Boolean logical rules over binary visual concepts, providing faithful local explanations for individual predictions and global rules for the diagnosis task. To flexibly adapt the rule structure to the data, we extend the stratified AND/OR layout in CRL with an adaptive logical layer, forming Ada-CRL, which jointly learns operator, connectivity, and polarity at the node level. While Ada-CRL targets faithful rule learning with dense concept supervision, two practical questions remain: how much flexibility is gained by relaxing Boolean reasoning, and whether rule learning remains feasible without concept annotations. To answer these questions, we introduce two controlled variants: Fuzzy-CRL, which probes the trade-off between continuous reasoning flexibility and rule faithfulness, and SAE-CRL, which replaces manually annotated concepts with sparse features to examine rule learning without concept supervision. Results from four clinical image classification tasks spanning dermatology, hematology, ophthalmology, and breast ultrasound show that the CRL family achieves competitive diagnostic performance under both in-distribution and out-of-distribution evaluation, and that the learned rules remain valid under severe distribution shift. The code of our work is available at https://github.com/obiyoag/crl.