当明天到来时:对未来基于人工通用智能的无人战斗飞行器系统进行前性风险评估
Paul M Salmon1, Scott McLean1, Tony Carden2
1Centre for Human Factors and Sociotechnical Systems, University of the Sunshine Coast, Australia.
Applied ergonomics
|February 6, 2024
概括
使用WDA-BN和EAST-BL方法分析了人工通用智能 (AGI) 的风险. 目标对齐和超级智能等新风险需要超越当前系统的新型安全控制.
科学领域:
- 人工智能安全问题 人工智能安全问题
- 系统安全工程系统安全工程
- 未来战争研究 未来战争研究
背景情况:
- 人工通用智能 (AGI) 存在存在威胁的担忧.
- 鉴定和控制潜在的AGI风险是具有挑战性的,因为其不存在的状态.
- 对于先进的人工智能系统,现有的安全框架可能不足.
研究的目的:
- 识别与未来基于AGI的无人战斗飞行器 (UCAV) 系统相关的潜在风险.
- 评估当前安全方法的适用于预想的AGI系统的适用性.
- 为新的AGI相关风险提出必要的控制措施.
主要方法:
- 应用工作领域分析破碎节点 (WDA-BN) 来建模系统约束和故障.
- 使用系统团队合作-断裂链路事件分析 (EAST-BL) 来分析团队合作和协调失败.
- 基于场景的风险评估在一个假设的"设想的世界"AGI-UCAV系统.
主要成果:
- 确定了五个主要的风险类别:低于最佳的表现,目标对齐,超级智能,过度控制和削弱.
- 目标调整和超级情报风险构成了传统安全管理无法解决的新挑战.
- 低于最佳性能的风险在很大程度上可以通过现有的防御设计流程来管理.
结论:
- 与AGI相关的新风险,特别是目标对齐和超级智能,需要开发新的安全控制.
- 拟议的控制包括对AGI开发人员的规定,内部AGI保障措施和全面的社会技术系统监督.
- 积极开发安全框架对于减轻未来AGI系统潜在的生存威胁至关重要.
更多相关视频
06:20Author Spotlight: Development of an Automated Camera-Based System for Real-Time Blast Overpressure Monitoring and TBI Risk Assessment in Military Training
Published on: December 6, 2024
2.8K
07:48Eye Tracking During A Complex Aviation Task For Insights Into Information Processing
Published on: April 4, 2025
281
相关概念视频
00:59Uncertainty: Overview
In analytical chemistry, we often perform repetitive measurements to detect and minimize inaccuracies caused by both determinate and indeterminate errors. Despite the cares we take, the presence of random errors means that repeated measurements almost never have exactly the same magnitude. The collective difference between these measurements - observed values - and the estimated or expected value is called uncertainty. Uncertainty is conventionally written after the estimated or expected value.
01:24Absolute Motion Analysis- General Plane Motion
Visualize a drone, with its propellers spinning rapidly, hovering mid-air. The fascinating movements and operations of this drone can be comprehended by applying the principle of general plane motion.
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the drone...
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the drone...
01:30Unrealistic Optimism Bias
Unrealistic optimism bias is the tendency to overestimate the likelihood of positive outcomes. This cognitive bias makes individuals believe they are less likely to experience failures, setbacks, or risks and more likely to succeed than others. For example, people may assume they are less prone to health issues, accidents, or financial struggles than their peers, even when they share similar risk factors.One key component of this bias is the above-average effect, where individuals perceive...