Related Experiment Video
Updated: Jun 12, 2026
05:43
Usability Evaluation of Augmented Reality: A Neuro-Information-Systems Study
Published on: November 30, 2022
Complementary methods of system usability evaluation: surveys and observations during software design and development
Jan Horsky1, Kerry McColgan, Justine E Pang
1Clinical Informatics Research and Development, Partners HealthCare, Boston, USA. jhorsky@partners.org
Journal of Biomedical Informatics
|June 16, 2010
Summary
No single usability evaluation method is best for clinical information systems. Combining methods like surveys early and observations later improves software adoption and care quality.
Area of Science:
- Health Informatics
- Human-Computer Interaction
- Software Engineering
Background:
- Poor usability of clinical information systems (CIS) hinders clinician adoption and limits improvements in healthcare efficiency and safety.
- Regular usability evaluations are crucial for effective CIS design and development.
Purpose of the Study:
- To compare the effectiveness of four usability evaluation methods during outpatient clinical documentation software development.
- To determine the optimal stage and usability aspects addressed by each method.
Main Methods:
- Clinician email response
- Online surveys
- Direct observations of clinical work
- Clinician interviews
Main Results:
- No single method identified all usability issues; each method excels at different design stages and identifies different problem types.
- Email responses and surveys effectively identified technical, terminology, and control issues late in development (post-prototype).
- Observations and interviews were most effective early in the design cycle for identifying conceptual and workflow-related problems.
Conclusions:
- Integrating multiple usability evaluation methods throughout the software development lifecycle is essential for improving CIS usability.
- Consistent application of diverse evaluation techniques can lead to higher clinician adoption rates and enhanced quality of care.
Related Concept Videos
01:07Data Collection by Survey
The systematic method of obtaining and analyzing accurate information of a population is called data collection. A survey is a standard method of data collection that involves collecting information from a target human population about their experience, opinion, or knowledge of a product, service, or process. The responses are recorded and interpreted. The most common survey examples are written questionnaires, face-to-face or telephonic conversations, focus groups, and electronic (e-mail or...
01:16Response Surface Methodology
Response Surface Methodology (RSM) is a collection of statistical and mathematical techniques used to develop, improve, and optimize processes. It is particularly valuable when many input variables or factors potentially influence a response variable.
The process of RSM involves several key steps:
The process of RSM involves several key steps:
02:16Surveys
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
01:27Types of Surveys
Surveys are essential for marking property boundaries near water bodies. Different types of surveys are defined, each with its own function. Land surveys mark the property boundaries, while route surveys determine the position of properties on nearby highways. Topographic surveys create maps by capturing the three-dimensional features of the land. Hydrographic surveys focus on the shapes of underwater areas and the movement of streams through the properties. Mine surveys determine the relative...
01:22Design Consideration
Designing a structure involves a series of considerations, primarily the material's ultimate strength, calculated through tests that measure changes under increased force until the material reaches its breaking point or limit. The ultimate load, where the material breaks, is divided by its original cross-sectional area, resulting in the ultimate normal stress or strength. The ultimate shearing stress is another significant factor taken into account.
The factor of safety is another key aspect...
The factor of safety is another key aspect...
01:16Experimental Designs
An experimental design is a systematic process that allows researchers to evaluate the relationship between dependent and independent variables. There are three widely used types of experimental design - pre-experimental design, true experimental design, and quasi-experimental design. In pre-experimental design, the researcher compares the data before and after some interventions or treatments. The true-experimental design has more than one purposefully created group, a commonly measured...