AI as a Force Multiplier for Harm: Governing Catastrophic Misuse Scenarios

Executive Summary

Much of the public discourse around AI risk focuses on misalignment — AI systems that develop goals at odds with human values and act on them autonomously. But a parallel and more immediate risk requires urgent attention: deliberate misuse of AI by humans as a force multiplier for targeted harm. Cyberattacks, bioweapon design, autonomous weapons, and influence operations powered by AI represent near-term catastrophic risks that existing governance frameworks are entirely unprepared to handle.

Threat Vectors

1. Cyberattacks on Critical Infrastructure

AI has already been demonstrated to accelerate vulnerability discovery, automate exploit generation, and enable adaptive malware that evades detection. Red-team evaluations at major AI labs show 10–100x improvement in exploit discovery speed with AI assistance. Energy grids, water treatment systems, financial networks, and hospital systems represent high-value targets where AI-enabled attacks could cause mass casualties.

2. Bioweapon Design Assistance

Early research suggests that large language models can provide meaningful uplift to individuals seeking to create dangerous pathogens — suggesting dangerous mutation pathways, synthesis routes, and evasion strategies. This threat vector intersects with the engineered pandemic risk described in a companion document.

3. Autonomous Weapons

Lethal autonomous weapons systems (LAWS) that can select and engage targets without human authorization are being developed by multiple state actors. The combination of reduced decision latency, potential for mass deployment, and unclear accountability creates novel escalation risks in conflict scenarios.

4. AI-Powered Influence Operations

Synthetic media, persona networks, and hyper-personalized persuasion at scale already undermine democratic discourse. As AI-generated content becomes indistinguishable from authentic human output, the information ecosystem faces structural degradation.

Governance Gaps

• Export controls on AI hardware (GPUs) exist but are porous and do not address model weights or algorithmic knowledge.
• Frontier model evaluations for dangerous capabilities are voluntary and inconsistently applied across labs.
• International coordination on AI weapons is nascent — the UN process on LAWS has produced no binding agreement in a decade of deliberation.
• Dual-use research norms in AI lag far behind those in biosecurity.

Tractable Interventions

1. Mandatory dangerous capability evaluations before model deployment above defined capability thresholds (CBRN uplift, cyberoffense, etc.).
2. Compute governance: Strengthen and extend export controls; develop international frameworks for monitoring training runs above critical thresholds.
3. LAWS moratorium: Advocate for a binding international agreement prohibiting fully autonomous lethal weapons targeting.
4. Content provenance standards: Adopt C2PA (Coalition for Content Provenance and Authenticity) or equivalent technical standards for AI-generated media.

Recommendations

1. Require independent third-party evaluations of frontier AI models for misuse potential before public release.
2. Establish an AI Safety Information Sharing and Analysis Center (AI-ISAC) for threat intelligence sharing among governments and labs.
3. Fund research into AI-specific deterrence frameworks for state-level misuse.
4. Engage the synthetic biology and cybersecurity communities in joint governance development.

Further Reading

• Center for AI Safety: State of AI Safety (2024)
• RAND Corporation: Autonomous Weapons and the Laws of War
• Anthropic: Responsible Scaling Policy (anthropic.com)
• UK AI Safety Institute: Frontier AI Safety Commitments