Data journeys: Identifying social and technical barriers to data movement in large, complex organisations

Area of Science:

• Information Systems Engineering
• Health Informatics
• Organizational Behavior

Background:

• Enterprise-level software development decisions are complex, with costs and risks often underestimated.
• Existing software engineering methodologies frequently overlook social aspects influencing project outcomes.
• Large organizations, such as the UK's National Health Service (NHS), require better tools for assessing new software system investments.

Purpose of the Study:

• To aid managers and stakeholders in complex organizations in making informed decisions about the costs and risks of new software systems.
• To develop a method that predicts high-cost and risk areas in software development, particularly concerning data movement.
• To integrate both technical and social factors into risk and cost prediction for enterprise software projects.

Main Methods:

• Analysis of case studies from UK healthcare providers to identify common risk and cost factors in software development.
• Development of a novel 'data journey modelling' method.
• Retrospective evaluation of the data journey modelling method within an NHS Foundation Trust hospital.

Main Results:

• Data movement within and between organizations was identified as a critical factor for cost and risk.
• The proposed data journey modelling method effectively predicted key areas of high cost and risk.
• The method also identified potential risks not initially recognized by hospital staff.

Conclusions:

• Data journey modelling offers a lightweight approach to predict costs and risks in software development by integrating technical and social aspects.
• The method is valuable for complex organizations, especially in the healthcare sector, for better decision-making on IT infrastructure projects.
• This approach enhances existing methodologies by incorporating crucial social dynamics often overlooked in traditional software engineering.