Related Experiment Video
Updated: Nov 19, 2025
05:21
Characterization of the Sense of Agency over the Actions of Neural-machine Interface-operated Prostheses
Published on: January 7, 2019
8.1K
Meaningful Human Control over Autonomous Systems: A Philosophical Account
Filippo Santoni de Sio1, Jeroen van den Hoven1
1Section Ethics/Philosophy of Technology, Faculty Technology Policy and Management, Delft University of Technology, Delft, Netherlands.
Frontiers in Robotics and AI
|January 27, 2021
Summary
This paper defines meaningful human control for autonomous weapon systems, proposing two conditions: tracking environmental and moral factors, and tracing outcomes to human operators. This ensures ethical AI development and deployment.
Area of Science:
- Ethics of Artificial Intelligence
- Robotics and Autonomous Systems
- Philosophy of Technology
Background:
- Proliferation of lethal autonomous weapon systems (LAWS) raises ethical concerns regarding increased military wrongs and a potential responsibility gap.
- The principle of meaningful human control (MHC) is central to legal and political debates on LAWS, emphasizing human oversight.
- Existing policy and design frameworks lack a precise definition of MHC, hindering its practical implementation.
Purpose of the Study:
- To establish a foundational philosophical account of meaningful human control (MHC) for autonomous systems.
- To translate MHC principles into actionable design requirements aligned with Responsible Innovation and Value-sensitive Design.
- To explore the broader implications of the MHC framework for both military and non-military AI applications.
Main Methods:
- Developed a philosophical account of MHC grounded in the concept of 'guidance control' from free will and moral responsibility debates.
- Formulated MHC as a set of design requirements, focusing on necessary conditions for maintaining human control over autonomous systems.
- Proposed two core conditions for MHC: 'tracking' (system responsiveness to reasons and environment) and 'tracing' (accountability for outcomes).
Main Results:
- Defined MHC through two necessary conditions: 'tracking' and 'tracing'.
- The 'tracking' condition requires autonomous systems to respond to relevant human moral reasons and environmental facts.
- The 'tracing' condition mandates that system outcomes must be traceable to a human in the design or operational chain.
Conclusions:
- The proposed account of MHC provides a theoretical basis and practical design requirements for ethically developing and deploying autonomous systems.
- Meaningful human control, defined by tracking and tracing conditions, is crucial for addressing ethical concerns in AI and robotics.
- The MHC framework has implications for the design and use of diverse autonomous systems, including self-driving cars and military applications.
Keywords:
AI ethicsautonomous weapon systemsethics of autonomous systemsethics of roboticsmeaningful human controlresponsibility gapresponsible innovation in roboticsvalue-sensitive design in roboticsMore Related Videos
Related Concept Videos
01:10Control Systems
1.6K
Control systems are everywhere in contemporary society, influencing diverse applications from aerospace to automated manufacturing. These systems can be found naturally within biological processes, such as blood sugar regulation and heart rate adjustment in response to stress, as well as in man-made systems like elevators and automated vehicles. A control system is essentially a network of subsystems and processes that collaboratively convert specific inputs into desired outputs.
At the heart...
At the heart...
1.6K
01:25Control Systems: Applications
936
Electrical engineering plays a pivotal role in our daily lives, with control systems at the heart of many applications, from home appliances to sophisticated space shuttles. Control systems manage and regulate the behavior of devices and processes, ensuring they function safely, correctly, and efficiently.
In modern vehicles, control systems manage various functions to enhance performance and safety. The steering wheel and accelerator are primary inputs in a car's control system. The...
In modern vehicles, control systems manage various functions to enhance performance and safety. The steering wheel and accelerator are primary inputs in a car's control system. The...
936
01:17Open and closed-loop control systems
1.3K
Control systems are foundational elements in automation and engineering. They are broadly categorized into open-loop and closed-loop systems. These classifications hinge on the presence or absence of feedback mechanisms, significantly influencing the system's performance, complexity, and application.
An open-loop control system operates without feedback from the output. It consists of two primary elements: the controller and the controlled process. The controller receives an input signal...
An open-loop control system operates without feedback from the output. It consists of two primary elements: the controller and the controlled process. The controller receives an input signal...
1.3K
01:26Feedback control systems
556
Feedback control systems are categorized in various ways based on their design, analysis, and signal types.
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
Linear feedback systems are theoretical models that simplify analysis and design. These systems operate under the principle that their output is directly proportional to their input within certain ranges. For instance, an amplifier in a control system behaves linearly as long as the input signal remains within a specific range. However, most physical systems exhibit inherent nonlinearity...
556
01:18Hierarchy of Motor Control
5.1K
The hierarchy of motor control refers to the different levels of organization and processing involved in controlling movement in the body. These levels range from higher cortical areas involved in planning and decision-making to lower spinal cord reflexes that respond automatically to external stimuli.
5.1K
01:23Carl Rogers' Humanistic Perspective on Personality
1.7K
Carl Rogers, a key figure in humanistic psychology, believed that individuals possess an innate potential for growth and fulfillment. According to his model of personality, three significant components define an individual: the organism, the self, and conditions of worth.
The organism refers to an individual's inherent blueprint, which Rogers saw as innately positive and directed toward helping others, unlike Freud's view of the id as driven by base impulses. The self is a person's...
The organism refers to an individual's inherent blueprint, which Rogers saw as innately positive and directed toward helping others, unlike Freud's view of the id as driven by base impulses. The self is a person's...
1.7K