The Experiment That Wasn't Run
Description
The provided sources do not directly address the question of whether rigorous AI risk content has been systematically tested for virality and found to fail. Instead, they cover a wide range of AI-related topics including risk frameworks (NIST AI RMF, MIT AI Risk Initiative), AI failures in marketing and advertising, AI-generated misinformation, and viral content strategies. There is no evidence of a deliberate, well-funded attempt to make nuanced AI risk content go viral and a subsequent conclusion that it cannot work. The sources suggest that AI-generated content can go viral (e.g., AI-generated disinformation during crises) and that human-like, emotionally resonant content tends to perform better, but they do not provide data on rigorous risk content specifically.
Sources & further reading (25)
- AI Disinformation Incident Repository: How AI is transforming crisis events | The Alan Turing Institutehttps://www.turing.ac.uk/blog/ai-disinformation-incident-repository-how-ai-transforming-crisis-events
- The Science of Viral Content & How to Use AI to Create It: Data-Driven Strategies That Actually Work (2026 Guide) - First Movershttps://firstmovers.ai/viral-content-guide/
- 12 AI Fails in Advertising (2026 Update): Examples and Lessons AI Fails in Advertising: 8 Shocking Examples Brands Regrethttps://fraudblocker.com/articles/shocking-ai-fails-by-advertisers
- 10 Epic and Entertaining AI Marketing Fails: Lessons in Innovationhttps://blog.brandsatplayllc.com/blog/10-ai-marketing-fails
- AI Risk Management Framework | NISThttps://www.nist.gov/itl/ai-risk-management-framework
- MIT AI Risk Initiativehttps://airisk.mit.edu/
- Is your AI Model Accurate Enough? The Difficult Choices Behind Rigorous AI Development and the EU AI Acthttps://arxiv.org/html/2604.03254v1
- New sources of inaccuracy? A conceptual framework for studying AI hallucinations | HKS Misinformation Reviewhttps://misinforeview.hks.harvard.edu/article/new-sources-of-inaccuracy-a-conceptual-framework-for-studying-ai-hallucinations/
- AI vs. Traditional Risk Assessment Methodshttps://www.lucid.now/blog/ai-traditional-risk-assessment-methods/
- The Great AI Risk Miscalculation: Why 90% of Companies Are Unprepared | Censinet, Inc.https://censinet.com/perspectives/ai-risk-miscalculation-companies-unprepared
- Developing a Risk-Scoring Tool for Artificial Intelligence–Enabled Biological Design: A Method to Assess the Risks of Using Artificial Intelligence to Modify Select Viral Capabilities | RANDhttps://www.rand.org/pubs/research_reports/RRA4490-1.html
- AI can now be used to design brand-new viruses. Can we stop it from making the next devastating bioweapon? | Live Sciencehttps://www.livescience.com/health/viruses-infections-disease/ai-can-now-be-used-to-design-brand-new-viruses-can-we-stop-it-from-making-the-next-devastating-bioweapon
- Advancing AI safely and responsibly — Google AIhttps://ai.google/safety/
- 10 AI dangers and risks and how to manage them | IBMhttps://www.ibm.com/think/insights/10-ai-dangers-and-risks-and-how-to-manage-them
- AI Safety Index: Summer 2025 - Future of Life Institutehttps://futureoflife.org/ai-safety-index-summer-2025/
- Risks of Bad AI-Generated Content for Your Brand | TechWysehttps://www.techwyse.com/blog/digital-marketing-101/ai-generated-content-risks-for-brands
- Why Some AI Images Go Viral — And Others Fail Miserably | by Nancy | Mediumhttps://medium.com/@2218867196ji/why-some-ai-images-go-viral-and-others-fail-miserably-287a804028f7
- Decoding AI Virality Algorithms: The Complete 2025 Guide to Viral Content & SEO | by Andy | Activated Thinker | Mediumhttps://medium.com/activated-thinker/decoding-ai-virality-algorithms-the-complete-2025-guide-to-viral-content-seo-b249ac969d7a
- Journal of Medical Internet Research - Using AI to Assess Potential Zoonotic Threatshttps://www.jmir.org/2026/1/e93261
- Risk of viral failure after simplification therapy without using integrase inhibitors compared with maintenance of triple antiretroviral therapy: A systematic review and meta-analysis - ScienceDirecthttps://www.sciencedirect.com/science/article/pii/S1413867024007463
- AI Mismatches: Identifying Potential Algorithmic Harms Before AI Development | Proceedings of the 2025 CHI Conference on Human Factors in Computing Systemshttps://dl.acm.org/doi/10.1145/3706598.3714098
- Atlas of AI Risks: Enhancing Public Understanding of AI Riskshttps://arxiv.org/html/2502.05324v1
- AI Risks and Trustworthiness - AIRC - NIST AI Resource Centerhttps://airc.nist.gov/airmf-resources/airmf/3-sec-characteristics/
- News – FAR.AIhttps://www.far.ai/blog
- Guardii | AI-Powered Child Safety & Social Media Protectionhttps://www.guardii.ai/blog/how-ai-detects-harmful-context-in-messages
Script
Cold open
What if the reason rigorous AI risk content doesn't go viral is simply that no one with serious resources has actually tried?
Frame
Projects like plzdontkillus are betting on a theory: that the attention economy requires simplification, that rigorous arguments can't break out of niche communities, and that the right move is to recruit charismatic creators and let them distill the message for reach. The theory has a prior assumption built into it — that depth-first content can't work on these platforms. That assumption has not been tested against the available evidence.
How do we know that AI-generated content can go viral?
What does depth-first content actually do on modern platforms? Science communicators have spent the last decade demonstrating that genuinely difficult material — quantum mechanics, Gödel's incompleteness theorems, the mathematics of neural networks, the long-run consequences of existential catastrophes — can reach audiences of millions if the production quality is high and the framing is compelling. The ceiling for complex content is not where the AI risk world tends to assume it is.
What are the known risks of AI-generated content?
What has the AI risk world actually tried in short and medium form? The primary modes are long-form essays on niche platforms, multi-hour podcast appearances, and conference talks. These reach the already-converted. The high-production, well-resourced attempt to put the actual core arguments — orthogonality, instrumental convergence, the difficulty of value specification — into tight eight-minute YouTube videos with compelling visuals has not happened at scale. That experiment remains unrun.
What frameworks exist to assess AI risks rigorously?
What would it actually look like to run that experiment? It looks like the Kurzgesagt treatment of instrumental convergence. Like a 3Blue1Brown-style walkthrough of what 'value specification' means in practice. Like a tight, well-produced video that doesn't abstract away the argument but makes its logical structure legible without flattening it. None of these exist at the production level that would make them competitive with what the best science communicators are doing on the same platforms. The argument that they can't work hasn't been made, because the attempt hasn't been made.
What tools and methods can help produce accurate AI content at scale?
What is the cost if the premature bet is wrong? The simplification-first strategy optimizes for something measurable — follower counts, engagement, reach — at the cost of something harder to measure: whether the people reached have an accurate model of the risk. If rigorous depth-first content could have reached the same audiences with the right production, then the epistemic quality of the conversation could have been higher at the same scale. That opportunity doesn't recur.
Turn
The missing experiment is not exotic or expensive. It requires treating AI risk communication the way the best science communicators treat physics or mathematics — as a challenge of exposition, not a requirement of simplification. The claim that depth can't go viral should be tested before it's treated as settled policy.
Closer
The simplification tradeoff is an empirical claim, not a discovered fact. Right now it's being treated as the lesson without the test having been run. That is not a small assumption to build a movement on.