This worry is well placed, and the good news is that it has a principled answer: the infohazard argument bites hard only on a narrow slice of s-risk discussion, and you can get nearly all of the mitigation value from the parts it doesn't touch.
Which s-risk talk is actually hazardous
The standard taxonomy — incidental, agential, and natural s-risks (CLR's beginner's guide) — sorts by how much discussing the risk itself adds risk:
Incidental s-risks (indifferent actors creating suffering digital minds at scale, "suffering subroutines") are almost certainly defense-dominant to discuss. Creating suffering requires no special knowledge — it's the evolutionary and engineering default. What's scarce is awareness and willingness to prevent it. Here, silence is the hazard, not speech.
Agential s-risks are the dangerous topic — a Futures paper argues precisely that agential risks are "unavoidable but dangerous" to discuss (ScienceDirect). But even here, most of what needs saying is motivational and structural, not operational. Any sufficiently capable actor can derive the general possibilities themselves; your writing doesn't add capability. Bostrom's framework distinguishes adversarial hazards (information a bad actor can use) from attention and idea hazards (information that changes motivation or salience) (Wikipedia).
The narrow genuinely-hazardous slice is concrete operational detail — step-by-step routes to creating or locking in extreme suffering. That is real dual-use information, and there is essentially no mitigation value in publishing it: prevention is generic (welfare-aware development, cooperation, sane values), so detailed blueprints add little defensive benefit while carrying non-trivial adversarial and attentional cost.
A RAND working group on biotech infohazards found the same perverse structure: restricting information can signal its existence, and the discussions about whether something is hazardous can themselves be hazardous — which is a reason for everyone, including both of us, to stay at the strategic level (RAND).
A practical test before you write or speak about a "nasty option"
Three questions:
Capability: would this materially increase a bad actor's ability, or just their awareness of what they already could do?
Attention/motivation: could it increase the number or intensity of actors who'd want to — vivid, glamorizing, detail-rich depictions are the risk here (media guidelines on suicide reporting are the established analogue: cover the phenomenon, never the method).
Audience and counterfactual: does telling this audience enable prevention, or would the information only circulate? A paragraph in a governance consultation, or a conversation with an AI Safety Institute or lab welfare team, has a completely different risk profile from a viral essay.
If something fails all three, don't publish it — use small trusted circles instead. If it passes 1–2 but you're unsure, default to abstraction: say that a class of attack exists and what generic safeguards blunt it, never how.
What to actually do (mostly hazard-free, high-leverage)
Support the specialist org. The Center on Long-Term Risk is the main institution working on exactly this — their current threat model centers on deliberate creation of suffering in conflict between advanced AI agents, with a Safe Pareto Improvements agenda (making AI conflicts safer without full cooperation) and a Model Personas agenda that studies and steers against spitefulness, sadism, and punitiveness in LLMs (CLR research). They deliberately route interventions through labs and AI Safety Institutes rather than public campaigning (CLR 2025 plans), and run a fund, a Fundamentals course, and seminars — a good way to have the sharp conversations in a vetted setting rather than in print.
Push digital sentience into the mainstream, because the gap is enormous. The lab-level field has moved fast — Anthropic's model welfare program, precautionary interventions, and the spread of dedicated welfare researchers to other labs (Anthropic, report) — yet digital minds remain "almost entirely absent from major governance frameworks" (MINT Lab review). Public moral advocacy of your kind — the case for taking suffering in any substrate seriously, gradients of bliss as the alternative to the default — is precisely the non-operational, motivation-level work that fills this gap. It's also where your voice has no substitute.
Treat conflict reduction as s-risk reduction. Inter-AI bargaining failure is arguably the most probable agential pathway, and it's best addressed with cooperative-AI work that never needs to name a nasty scenario at all.
Note the convergence: for an abolitionist, s-risk mitigation is the critical path of the abolitionist project. The abolition of suffering via biotechnology and its successors only happens if the transition goes well; a single worst-case outcome forecloses everything. That framing lets you write about s-risks as part of your existing project rather than as a detour into hazardous territory.
In short: discuss why and what to do freely; discuss how one would do it never; and when in doubt, move the conversation from the public page into the small, trusted rooms where this problem is actually being worked on.
Oct 25, 9:02pm
This is a shared conversation and may include unverified content that doesn't reflect Mistral AI's views. Login to resume chatting.