Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Learning Compositional Sparse Bimodal Models.

Suren Kumar, Vikas Dhiman, Parker A Koch

    IEEE Transactions on Pattern Analysis and Machine Intelligence
    |April 20, 2017
    PubMed
    Summary
    This summary is machine-generated.

    Related Concept Videos

    You might also read

    Related Articles

    Articles linked to this work by shared authors, journal, and citation graph.

    Sort by
    Same author

    Adaptation and mitigation approaches to reduce heat-related health risks: An overview of systematic reviews.

    Environmental research·2026
    Same author

    Measuring Physical Plausibility of 3D Human Poses Using Physics Simulation.

    BMVC : proceedings of the British Machine Vision Conference. British Machine Vision Conference·2026
    Same author

    Learning to Estimate External Forces of Human Motion in Video.

    Proceedings of the ... ACM International Conference on Multimedia, with co-located Symposium & Workshops. ACM International Conference on Multimedia·2026
    Same author

    Clinico-genetic Analysis of Adenosine Signaling Pathway in Drug-Resistant Epilepsy.

    Annals of Indian Academy of Neurology·2025
    Same author

    Microwave assisted green synthesized copper- carrageenan bionanocomposite for efficient removal of cefixime from defile water.

    International journal of biological macromolecules·2024
    Same author

    A systematic review and meta-analysis of prevalence of seven psychiatric disorders in India.

    Indian journal of psychiatry·2024
    Same journal

    Relation DETR+: Exploring Explicit Position Relation Prior for Dense Prediction.

    IEEE transactions on pattern analysis and machine intelligence·2026
    Same journal

    RBF++: Quantifying and Optimizing Reasoning Boundaries across Measurable and Unmeasurable Capabilities for Chain-of-Thought Reasoning.

    IEEE transactions on pattern analysis and machine intelligence·2026
    Same journal

    CAFE: Cross-View Adaptive Fusion and Cluster Center Enhancement for Robust Multi-View Clustering.

    IEEE transactions on pattern analysis and machine intelligence·2026
    Same journal

    DIVER: Reinforced Diffusion Breaks Imitation Bottlenecks in End-to-End Autonomous Driving.

    IEEE transactions on pattern analysis and machine intelligence·2026
    Same journal

    Ethics-Aware Safe Reinforcement Learning for Rare-Event Risk Control in Interactive Urban Driving.

    IEEE transactions on pattern analysis and machine intelligence·2026
    Same journal

    Learning Shape Anchors for Holistic Indoor Scene Understanding.

    IEEE transactions on pattern analysis and machine intelligence·2026
    See all related articles

    This study introduces a novel method for modeling bimodal perception by learning compositional structures across different sensory inputs. The approach enables generalization to new combinations of features, enhancing AI

    Area of Science:

    • Artificial Intelligence
    • Cognitive Science
    • Machine Learning

    Background:

    • Current AI models struggle to capture compositional semantics in perceptual domains.
    • Learning compositional structure directly has been a significant challenge for existing models.

    Purpose of the Study:

    • To propose a novel approach for modeling bimodal perceptual domains by explicitly learning compositional structures.
    • To enable AI models to generalize to unobserved percepts by grounding compositional semantics in separate modalities.

    Main Methods:

    • Developed a bimodal sparse representation model that relates distinct projections across modalities.
    • Jointly learned the compositional structure and projection basis automatically, without prior assumptions.
    • Focused on a tabletop building-blocks scenario with a new dataset of images and spoken utterances.

    Related Experiment Videos

    Main Results:

    • The model successfully learned compositional semantics, enabling generalization to novel combinations (e.g., learning 'red squares' and 'blue triangles' from 'red triangles' and 'blue squares').
    • Demonstrated significant benefits of the approach through quantitative experiments.
    • Validated the model's effectiveness in human evaluation studies.

    Conclusions:

    • Explicitly modeling compositional semantics across bimodal perceptual domains is crucial for generalization.
    • The proposed method offers a robust framework for learning and leveraging compositionality in AI.
    • This approach has implications for developing more sophisticated and human-like perceptual AI systems.