Specifying Visual Parameters for Haptic-Visual Sequential Matching of Material Softness

When shopping online, a customer cannot directly touch the products but may sometimes make judgments about the haptic properties of a product based only on visual information, before making a purchase decision. In this scenario, a customer may be dissatisfied if there is an inconsistency in the judgment of the product's haptic properties they made before purchasing, and their actual experience of those haptic properties once they have received the product. Thus, it is necessary for online sellers to appropriately optimize visual information for materials so that perceived softness is consistent between haptic and visual modalities presented in different locations and at different moments in time. Focusing on visual indentation depth and speed, we examined the visual parameters used to sequentially match haptic and visual softness from haptic and visual information made available in different locations and at different times. Participants performed a two-alternative forced choice task to determine which of two video clips contained an elastic material with a softness impression most similar to the haptic softness of an actual material that the participants indented with their index finger. Based on a sequence of 25 repeated judgments for each material, our algorithm optimized each visual parameter based on a Gaussian process. The optimized visual indentation depth varied consistently with material compliance, while the optimized visual indentation speed did not, suggesting that visual indentation depth was critical for softness matching. The optimized visual indentation depth was highly correlated with the haptic indentation depth. Subjective rating scores for the softness matching increased significantly after the optimization process. The results indicate that participants are able to successfully match the haptic and visual softness of materials by checking the relationship between indentation depths detected haptically, and those detected visually.