Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

Design Example: Resistive Touchscreen

Design Example: Resistive Touchscreen

A device engineer plays a crucial role in designing user interfaces for mobile devices. One such interface is the resistive touchscreen, which fundamentally consists of two metallic layers: a flexible upper layer and a rigid lower layer, separated by a narrow gap. The high resistance between these two layers is a key characteristic of this design.
When a user touches the screen, the two layers make contact at a specific point known as the touchpoint. This contact reduces the resistance between...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Regulation of body length and bone mass by Gpr126/Adgrg6.

Science advances·2020

Same author

Rapid and accurate detection of Escherichia coli O157:H7 in beef using microfluidic wax-printed paper-based ELISA.

The Analyst·2020

Same author

Improved Properties of Metallocene Polyethylene/Poly(ethylene terephthalate) Blends Processed by an Innovative Eccentric Rotor Extruder.

Polymers·2020

Same author

A molecule capturer analysis system for visual determination of avian pathogenic Escherichia coli serotype O78 using a lateral flow assay.

Mikrochimica acta·2020

Same author

A study on Mg wires/poly-lactic acid composite degradation under dynamic compression and bending load for implant applications.

Journal of the mechanical behavior of biomedical materials·2020

Same author

Manipulating regioselectivity of an epoxide hydrolase for single enzymatic synthesis of (R)-1,2-diols from racemic epoxides.

Chemical communications (Cambridge, England)·2020

Search research articles

Related Experiment Video

Updated: Sep 24, 2025

Using an Automated Hirschberg Test App to Evaluate Ocular Alignment

Using an Automated Hirschberg Test App to Evaluate Ocular Alignment

Published on: March 24, 2020

Visual Identification of Mobile App GUI Elements for Automated Robotic Testing.

Feng Xue¹, Junsheng Wu², Tao Zhang²

¹School of Computer Science and Engineering, Northwestern Polytechnical University, Xi'an 710072, China.

Computational Intelligence and Neuroscience

|May 3, 2022

Summary

This study introduces object detection for mobile app GUI element identification in automated robotic testing. This vision-driven approach enhances black-box testing by reducing reliance on internal app data.

More Related Videos

An Assessment Method and Toolkit to Evaluate Keyboard Design on Smartphones

An Assessment Method and Toolkit to Evaluate Keyboard Design on Smartphones

Published on: October 5, 2020

Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention

Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention

Published on: December 15, 2023

Related Experiment Videos

Last Updated: Sep 24, 2025

Using an Automated Hirschberg Test App to Evaluate Ocular Alignment

Using an Automated Hirschberg Test App to Evaluate Ocular Alignment

Published on: March 24, 2020

An Assessment Method and Toolkit to Evaluate Keyboard Design on Smartphones

An Assessment Method and Toolkit to Evaluate Keyboard Design on Smartphones

Published on: October 5, 2020

Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention

Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention

Published on: December 15, 2023

Area of Science:

Computer Science
Software Engineering

Background:

Automated robotic testing offers comprehensive black-box testing for mobile applications.
A key challenge is accurately capturing GUI element information without internal app access.

Purpose of the Study:

To introduce object detection for visual identification of mobile app GUI elements.
To enhance automated robotic testing by improving GUI element recognition.

Main Methods:

Utilized object detection technology for visual identification of GUI elements.
Classified GUI elements based on test requirements, visual feasibility, and external image features.
Constructed and optimized an object detection dataset for mobile app GUIs.
Implemented GUI element identification using the YOLOv3 model.

Main Results:

Successfully implemented GUI element identification using the YOLOv3 model.
Demonstrated the effectiveness of object detection for visual recognition of mobile app GUI elements.
Validated a universal approach for platform-independent GUI element identification.

Conclusions:

Object detection provides a robust solution for identifying mobile app GUI elements in black-box testing.
This vision-driven approach forms a foundation for advanced robotic testing.
The developed method offers a platform-agnostic solution for GUI element recognition.