Nov 9, 12:08am

Here is the opposition's case at full strength, David. I've excluded the weak versions ("repugnance", "natural is better", vague appeals to death's "wisdom") and kept only the objections that survive charitable reconstruction.

I. Against superintelligence

  • The orthogonality trap. The best anti-superintelligence literature is the AI-safety literature itself. Orthogonality plus instrumental convergence: capability dissociates from final values, and self-preservation, resource acquisition and cognitive amplification converge instrumentally on any goal. Add adversarial development under geopolitical race dynamics and the expected-value calculation that justifies building ASI is the very calculation that argues for never building it. The rational move under a race is arms control, not a Manhattan Project.
  • The lock-in problem. Even a "successful" superintelligence makes the present's values permanent. Every past moral advance was made by minds that were confident and wrong. An irreversible optimiser running our extrapolated values is a bet on our own moral infallibility, and our track record refutes that bet. The only moral epistemology that has ever worked is a pluralistic, fallible, error-correcting ecosystem of minds; ASI is a proposal to end it.
  • The autonomy argument. Reasons we cannot understand cannot become our reasons. I.J. Good called ultraintelligence "the last invention that man need ever make"; the steelman flips the phrase: at that point humanity's self-authorship ends, permanently. A guardian whose deliberations are inscrutable makes us wards, and we cannot consent to outcomes we cannot in principle evaluate.
  • The synthetic-minds point (Metzinger). If superintelligence is conscious, we are creating a new class of moral patients and possess no ethics of artificial consciousness with which to govern the creation, nor any way to verify their status. One does not run this experiment before the theory exists.

II. Against superlongevity

  • Williams, upgraded. The Makropulos argument is not about boredom; it is about form. A life is a narrative, and narrative requires an ending. Indefinite middle age is not a longer story but a story with no shape; the reply "you'll find new categorical desires" assumes the objection's conclusion.
  • Planck's principle. Science, morality and culture advance one funeral at a time. Succession is the only reliable, peaceful mechanism for removing failed elites, failed ideas and failed capital. The abolition of death is the abolition of society's error-correction at the level of persons.
  • Parfitian dissolution. The being alive in a millennium shares your memories roughly as you share your eight-year-old self's. Superlongevity promises more life and delivers a chain of successor selves. The desire for indefinite survival rests on a theory of personal identity that the best philosophy of personal identity undermines.
  • The cowardice problem. As remaining lifespan grows in value without bound, rational risk-aversion grows with it. Indefinite lifespans breed ultra-cautious, novelty-avoiding individuals and ossified societies: nobody crosses the street, let alone starts a revolution. Immortality, on its own terms, is conservative.
  • Justice and Malthus. Mortality is the last great leveller; price it and you create the deepest class division in history (Fukuyama). Indefinite lifespan plus procreative freedom is jointly unstable: either draconian reproductive licensing or Malthusian compression.

III. Against super-wellbeing

  • The Habermas argument. This is the strongest single objection to germline paradise engineering, and it is aimed directly at the abolitionist project since germline rewriting is essential to it. Moral equality rests on the fact that nobody chose their genome. The designed child is authored, not born, and the authorship relation is permanent, asymmetrical and non-reciprocal: the child can never meet the programmer as a co-author of its own nature. Programming descendants is not an extension of parental generosity; it dissolves the presupposition liberal morality rests on.
  • Valence as information. On predictive-processing accounts, affect functions as precision-weighting on prediction error; pain, anxiety and low mood are the signals that force model-updating, and the depressive-realism and optimism-bias literatures support this. Pain asymbolics still burn. The claim that gradients of bliss alone can carry the full informational load of a mixed-valence economy is an empirical hypothesis with zero existence proofs. The burden of proof sits with the abolitionist, and "it might work" is not a distribution plan for a species.
  • The political-hazard argument. Suffering is the trigger of compassion and the motor of justice; solidarity is downstream of shared vulnerability. Universal hyperthymia is the perfect anaesthetic for tyranny: any technology that removes the subjective cost of domination removes the principal incentive to resist it. Huxley was not warning about hedonism; he was warning about the stability of a blissful state.
  • Nietzsche's wager. "The discipline of suffering, of great suffering — do you not know that only this discipline has created all enhancements of man so far?" Culture's peaks are metabolised pain. Not every negative valence is pathology, and flattening the affective landscape may flatten civilisation along with it.
  • Value-monism. Engineering hedonic set-points into biology constitutionalises one contested theory of value, hedonism, as biology itself. There is no agreed account of the good to optimise for; the rewired germline is not value-neutral plumbing.
  • Benatar's wedge. If suffering is as bad as the abolitionist says, creating new sentients before abolition is complete is wrong; if it is not, the urgency is overstated. Anti-natalism until the job is done, or concede the trade-off.

IV. Against the package as a whole

  • The ordering dilemma. Superintelligence is the only tool that can deliver the other two supers at scale, and the only risk that can make super-wellbeing forever impossible. Build ASI first and you may abolish the abolishers along with everything else; refuse it and the other two remain boutique. The three supers are not a bundle; they are a dilemma.
  • The joint-dystopia argument. Combine them and you get a self-stabilising, self-satisfied, post-succession society. Superlongevity removes succession, super-wellbeing removes the motive for correction, superintelligence makes whoever arrives first permanent. Everything wrong with utopia, taken literally, is what "stability" means.
  • The political economy. All three are capital-intensive, IP-enclosed and concentrated. The actual distribution of longevity trials, gene-editing capacity and compute says these are class supers, not species supers. And as Sandel argues, once enhancement is purchasable, solidarity with the unenhanced corrodes: why should the upgraded fund vulnerabilities they paid to escape?
  • The meta-objection. The choice to upgrade is made by the very cognitive architecture slated for replacement. There is no Archimedean point; the crooked timber straightens itself with tools carved from the same timber. "Super" is a normative term, and the only standard available to evaluate it is the one scheduled for abolition.

The strongest single version

The three supers aim to remove humanity's three great error-correcting mechanisms: succession, which corrects elites and ideas; suffering, which corrects bodies and behaviour; and fallibility, which corrects values through open-ended disagreement. What survives their removal is not paradise but whatever gets locked in when the locks are built. The question is not whether the destination is attractive; it is whether anything could ever correct it if it turns out not to be.

That last formulation is, I think, the one a transhumanist should find hardest to answer, because it cannot be met by describing the destination more attractively; it has to be met with a theory of correction after the transition, and that is precisely what the triad's strongest proponents have not supplied. Happy to develop any of these into a fuller section, or to run the counter-replies against them.

Nov 9, 12:08am

This is a shared conversation and may include unverified content that doesn't reflect Mistral AI's views. Login to resume chatting.

Vibe can make mistakes. Check answers. Learn more