Perceptually Aware Image Retargeting for Mobile Devices.

Area of Science:

• Computer Vision
• Image Processing
• Human-Computer Interaction

Background:

• Conventional image and video retargeting methods often fail to capture human visual perception, relying on low-level features and high computational resources unsuitable for mobile platforms.
• Existing approaches struggle with arbitrary aspect ratios and may not produce aesthetically pleasing results, especially for video content.

Purpose of the Study:

• To develop a novel, efficient retargeting framework for photographs and videos that leverages human gaze behavior to improve visual quality on diverse screen resolutions and aspect ratios.
• To enable rapid retargeting on mobile devices by optimizing computational processes and incorporating human visual perception.

Main Methods:

• A geometry-preserving graph ranking algorithm was developed to identify salient object patches mimicking the human gaze shifting path (GSP).
• An aggregation-based Convolutional Neural Network (CNN) was employed to learn hierarchical deep representations of GSPs.
• A probabilistic model was utilized to learn aesthetic priors from professional photographer-selected images, guiding the retargeting process.

Main Results:

• The proposed method achieves retargeting in under 35 ms per photograph or video frame on mobile devices, significantly outperforming conventional algorithms.
• User studies demonstrated that the retargeted images and videos produced by this framework are visually superior to those from competing methods.
• Human eye-tracking experiments confirmed that the learned GSPs accurately reflect human visual attention.

Conclusions:

• The novel retargeting framework effectively adapts visual content for various screens by mimicking human gaze behavior, offering significant speed and quality improvements.
• This approach represents a substantial advancement in mobile multimedia adaptation, enhancing user experience through perceptually optimized retargeting.