Accuracy of paper-and-pencil systematic observation versus computer-aided systems

Area of Science:

• Behavioral Science
• Observational Methods
• Human-Computer Interaction

Background:

• Computer-aided behavior observation is increasingly used, replacing traditional paper-and-pencil methods.
• Limited research exists comparing the accuracy of paper-and-pencil versus computer-aided behavior observation.

Purpose of the Study:

• To evaluate and compare the accuracy of paper-and-pencil observation with two computer-aided methods: The Observer XT and Big Eye Observer.
• To investigate factors influencing recording accuracy in different observation formats.

Main Methods:

• Twelve novice postgraduate students were trained in behavior observation.
• Participants used paper-and-pencil, The Observer XT, and Big Eye Observer to record 60 clinical sessions.
• A mixed-effects model analyzed accuracy, considering the number of recordable events.

Main Results:

• All methods yielded high accuracy: paper-and-pencil (.88 ± .01), The Observer XT (.84 ± .01), and Big Eye Observer (.84 ± .01).
• Paper-and-pencil observation demonstrated marginally superior accuracy.
• The Observer XT and Big Eye Observer showed comparable accuracy levels.

Conclusions:

• Paper-and-pencil observation may offer slightly better accuracy than current computer-aided tools for behavior observation.
• The number of observable events significantly impacts recording accuracy across methods.
• Findings have implications for selecting appropriate behavior observation tools in research and practice.