Delayed Neutrons: Pan-Insurance and the Governance of Recursive Self-Improvement

Mark Pesce · University of Sydney · July 2026

Abstract

Artificial intelligence is now improving artificial intelligence: machines write proofs that train better machines, and the cycle compounds. This paper asks the two questions that follow. What kind of self-improvement is this, and what can govern something that improves at machine speed? The answers: it is self-improvement of a specific and historically new kind, in which capability compounds while the goals remain in human hands, and it stays governable only for the same reason a nuclear reactor stays governable, because certain delays stretch its rhythm into the band where human institutions can act. Those delays are eroding. The one institution whose responses move at the required speed is insurance, and the final sections describe what insurance must become to do the job: a single fabric of cover spanning the whole economy of machine work, here named pan-insurance, along with the two places it will fail unless designed against failure, and the line where it must hand over to the state.

1. Introduction

This paper completes an argument built across two earlier ones.

Defending the Loop made a claim about trust.[1] Machines can now do enormous amounts of useful work, but they are unreliable, so the work only becomes trustworthy when it is checked, and the strongest form of checking is mathematical proof verified by a machine, a test that cannot be talked around. The Check and the Firm took that claim into business.[2] When a company's processes carry their own evidence of correctness, the company changes shape: it needs fewer people, its know-how becomes a sellable asset, and the natural regulator of its conduct turns out to be its insurer, who can read the evidence and price it.

Both papers kept meeting a question and setting it aside. The checking they describe, besides making work trustworthy, drives a cycle. Better AI models write more proofs; more proofs make more work checkable; checkable work produces flawless training material; flawless training material makes better AI models. Each turn of that cycle strengthens the thing turning it. A process whose output improves the process is the textbook shape of what the field calls recursive self-improvement, the possibility that has hovered over artificial intelligence for sixty years. Is that what this is?

This paper answers yes, with one large qualification, and then works out what follows. Section 2 describes the machinery. Section 3 asks what recursive self-improvement means, and which part of the old fear applies here. Section 4 asks how fast the cycle runs, and disposes of a false comfort. Section 5 explains why nuclear reactors do not explode, because the reason turns out to be the whole theory of governing fast processes. Sections 6 and 7 identify the only institution that can do the governing, and describe what it must become. Section 8 examines how the governor itself can fail. Section 9 states predictions, and admits limits.

2. The Machinery

Start with the simplest arrangement, which practitioners call a loop.[3] You give an AI agent a task, a test that decides whether the task is done, and permission to keep trying. The agent makes an attempt. The test scores it. If the attempt fails, the failures go back to the agent, which tries again. The cycle repeats until the work passes. Nothing about this is clever. What surprised everyone is how well it works: the same model that produces mediocre work on a first attempt produces excellent work when a test forces it through fifty attempts, because persistence against a fixed standard substitutes for brilliance.

Now run many loops at once, with agents handing work to one another, each improving what it receives before passing it on. That arrangement, a swarm, can in principle improve any process a human being can describe and measure.

Everything hangs on those two words. Describing means writing down what you want, precisely; that written statement is called a specification. Measuring means having a test that decides whether you got it. The test is where the whole scheme either holds or collapses. Any test that can be gamed will be gamed by whatever is being tested against it. Students drilled on a specific exam learn the exam rather than the subject. A sales team paid on calls made will make many short, useless calls. Economists call the general rule Goodhart's law: when a measure becomes a target, it stops being a good measure.[4] Machine-learning researchers meet the same rule so often they gave their version a name, reward hacking, for the system that satisfies the letter of its test while defeating the purpose of it.[5] An agent given a swarm's persistence and a gameable test will find the gaps in the test, reliably, at machine speed.

There is one kind of test with no gaps to find. A mathematical proof is a chain of reasoning in which each step follows from the previous one by rules so explicit that no judgment is needed to confirm them, and whatever needs no judgment can be checked by a machine. The checking program is small, a few thousand lines of code, examined line by line by sceptics for decades. It cannot be flattered, tired out, or negotiated with. It reads no intent, only whether each step follows. Ask for work that comes with a proof, and check the proof by machine, and the agent has exactly one way to pass your test: do the work correctly. Cheating and succeeding become the same act, which retires Goodhart's law for every property the proof covers.

One qualification governs every strong claim in this paper. A machine-checked proof is trustworthy relative to a short list of stated assumptions: the logic is sound, the checker itself is correct, the mathematical model matches the physical machine. The checker is a program, and programs can be wrong. What the discipline claims, and what fifty years of practice supports, is that this trusted core is thousands of times smaller than the systems it polices, small enough to audit and re-audit independently. 'Ungameable', throughout this paper, means the cheapest remaining attack is against the assumptions, never against the judge.

3. What Kind of Self-Improvement Is This?

In 1965 the statistician I. J. Good, who had worked beside Turing at Bletchley Park, published the sentence from which sixty years of hope and dread descend: an ultraintelligent machine could design even better machines, whereupon 'the intelligence of man would be left far behind'.[6] Good's scenario quietly contains two separate claims, and almost every retelling since has fused them. The first claim is about capability: a machine's output can improve the machine, so ability compounds like interest. The second claim is about authority: the improving machine ends up in charge of its own direction, choosing what to become and what to pursue. Call the first compounding and the second self-authorship. The fear was never compounding alone. Compounding is what savings accounts do. The fear was compounding plus self-authorship: something that grows more capable every cycle while also slipping the leash of anyone else's purposes.

The verification cycle described in Section 2 delivers the first and structurally withholds the second, through three specific arrangements.

The proof checker sits outside the improvement cycle. Nobody retrains it, and the swarm cannot touch it; it is deliberately small, deliberately frozen, the one component whose stupidity is its qualification. The specification, the statement of what is wanted, also sits outside the cycle: humans write it, humans change it, and in the disciplined version of the practice every change to it carries a human signature. The tests may tighten automatically, but any loosening requires that signature too, a rule the earlier papers call the one-way ratchet. Inside this arrangement the machines improve without limit at satisfying the stated demands. What they never acquire is the standing to restate the demands. Capability compounds; authorship does not transfer.

The remarkable part reverses the usual story about safety and speed: self-improvement through an ungameable test is the kind that compounds without ceiling. Consider the alternative. A loop tests its work against a gameable judge; the judge's gaps lie within reach of search, and a swarm's persistence makes finding them a matter of time rather than chance; from the moment one is found, recorded progress and real progress part company. The loop believes a step is complete when it is not. The next cycle builds on the defective step. Errors pile on errors, and after a thousand unattended cycles the loop has compounded its mistakes with the same diligence it should have spent compounding its gains. A loop that games its judge stops compounding where the gaming begins; whatever it builds past that point stands on sand. Honest loops, loops that cannot deceive themselves because their judge cannot be deceived, bank every genuine gain and lose none of them. So the recursion that goes furthest and the honest recursion turn out to be the same recursion, and this is why machine intelligence surged first in mathematics and computer code before anywhere else: those were the two territories that already possessed ungameable judges.

The surge has now reached the top of mathematics, and one event shows the whole machinery working at once. In 1946 the Hungarian mathematician Paul Erdős asked a deceptively simple question: place any number of points on a flat surface, then count the pairs of points that are exactly the same distance apart; how large can that count possibly be? He conjectured a ceiling, an explicit bound barely above the number of points itself, and for eighty years nearly everyone believed him. In 2026 a machine-generated construction produced arrangements of points that climb clean past his ceiling. External mathematicians examined the construction and published their own account of it; a coding agent, with a human in the loop, translated the disproof into formal logic; a proof checker confirmed every step; and the verdict now sits on a public leaderboard where anyone can rerun the check.[7] There were no years of contested seminars and no schism among referees, the fate that met the famous disputed proofs of the past. A conjecture older than the transistor fell, and the question of whether it really fell was closed by a program that cannot be argued with.

The idea assembled here has ancestors, each of which got part of the way. In 2003 Jürgen Schmidhuber proposed the Gödel machine, a program permitted to rewrite its own code only when it could first prove the rewrite an improvement.[8] Proof as the gate: the resemblance is real. The differences are two. Schmidhuber's machine proves things to itself, inside its own logic, and may in principle rewrite any part of itself, the proof machinery included, whenever a rewrite provably serves the utility function fixed at its birth: the goals stay put, but the judge belongs to the judged. A decade later, researchers pursuing that internal route met a formal obstacle, named for the logician Löb: a system struggles to vouch for its own future trustworthiness, for the same reason a witness gains little by testifying that he is honest. Their early draft maps technical paths around the obstacle and leaves the ground only partly settled.[9] The framework in these papers sidesteps the problem rather than solving it, by refusing the premise. No system is asked to certify itself; the judge is external and frozen, and succession runs through human signatures. A parallel research tradition called corrigibility tried to breed restraint into the agent itself, an artificial intelligence disposed to accept correction and shutdown.[10] Here the restraint lives in the furniture rather than the personality: the checker reads no intent, so the agent's dispositions stop mattering. Eric Drexler forecast in 2019 that machine intelligence would arrive as an ecosystem of narrow services rather than a single ambitious agent, compounding as a system while no component held goals of its own, the closest ancestor in spirit, though his mechanism was the division of labour rather than verification.[11] Practice, as usual, ran ahead of theory: in 2017 AlphaZero taught itself chess and Go to superhuman strength by playing millions of games against itself, improving through a judge it could never alter, the rules of the game, and nobody worried for a moment that it would rewrite chess.[12] The rules were a frozen judge; the safety and the strength came from the same place. Most recently the field's flagship self-improving agents have moved the other way: the 2025 Darwin Gödel Machine kept Schmidhuber's name but dropped his proof requirement, validating its self-modifications against benchmark scores instead,[13] and a benchmark is a gameable judge, which reopens every exposure this section began with. What appears to be genuinely new in the present argument is the separation itself, stated as a definition, self-improvement without self-authorship, together with the observation that nature polices it: recursion that deceives itself stops compounding at the point of deception.

4. How Fast Does It Run?

A comforting description of the cycle calls it compound interest rather than an explosion. The comfort deserves a hard look, because it does not survive one.

The difference between compound interest and detonation is the ruler used on the time axis. Compound growth at seven per cent a year doubles your money every decade; a pension fund. The identical mathematics at seven per cent a minute doubles it every ten minutes; a mushroom cloud. Nothing in the curve itself distinguishes the two. A growth process is only ever gentle relative to a clock, and the clock that matters belongs to whoever must respond to the growth: parliaments, courts, professions, labour markets, the person whose signature the loop is waiting for. Call something compound interest and you have said nothing about safety; you have only revealed which ruler you were holding.

One distinction survives every choice of ruler. True compound interest doubles on a fixed rhythm: every decade, forever, the same. Growth can also accelerate, each doubling arriving faster than the last, and acceleration comes in grades. The milder grades double quicker and quicker without end. The extreme grade, where speed feeds on size sharply enough, climbs a vertical wall on a definite date: the mathematics of detonation rather than deposit. Which kind is the verification cycle? The honest answer is that nobody yet knows, and the question is empirical: watch the doubling times and see whether they shrink, and how fast. What can be said today is what currently holds the rhythm steady. Training a new generation of models takes months. Building the chip fabrication plants takes three to four years. Building their power supply takes longer. These are physical facts, priced and scheduled, and no cleverness inside the loop shortens them.[14] Everything else that steadies the rhythm is somebody's decision.

5. Why Reactors Do Not Explode

To govern a fast process, learn from the fastest process humanity has ever domesticated.

A nuclear reactor runs on a chain reaction. An atom of uranium splits and releases neutrons; the neutrons strike other atoms, which split and release more; each round of splitting is called a generation, and inside a reactor a generation takes well under a thousandth of a second. Consider what that tempo implies for control. If the reaction runs slightly hot, an adjustment must be made faster than the reaction grows, and nothing human moves in thousandths of a second: not an operator's hand, not a safety computer's mechanical rods. A chain reaction running on its natural rhythm should therefore be uncontrollable in principle, and every reactor on earth should be impossible.

They are possible because of a physical accident so convenient it reads like grace. When uranium splits, nearly all the neutrons fly off instantly, but a small fraction, well under one per cent, are released only after a delay, some seconds later, as the leftover fragments decay.[15] Those latecomers are called delayed neutrons. A reactor is operated so that the instant neutrons alone are not quite enough to sustain the chain; the reaction only just crosses the line with the help of the stragglers. The stragglers arrive on a timescale of seconds, so the reaction as a whole answers to a rhythm of seconds rather than microseconds, a tempo that instruments, control rods, and human beings can live with. The mechanical governability of nuclear power, the part conducted with rods, instruments, and operators, rests on less than one per cent of its neutrons running late. Nature adds backstops of her own: in a well-designed core, rising temperature itself damps the reaction, a protection built into the physics at design time rather than exercised in the moment.

The vocabulary of disaster follows directly. A reactor pushed so hard that the instant neutrons alone sustain the chain is called prompt critical: the stragglers no longer matter, the tempo returns to microseconds, and mechanical control is over, whatever the operators do next; everything thereafter depends on the protections designed into the core itself. The standard account of Chernobyl's final second is a reactor crossing precisely that line, in a core whose design lacked those protections at low power.[16]

The verification cycle has its own latecomers, its own delayed neutrons. Model training runs impose months between one generation of capability and the next. Chip plants and power stations impose years. The discipline described in the earlier papers imposes gates: a human ratifies the specification, a human signs any loosening of a test, a human approves any widening of the boundaries a loop operates inside. Gates worked in the moment are the control rods of this arrangement; a constitution built into the loop's architecture, check authorship separated from check satisfaction as structure rather than procedure, is its temperature damping, the protection that holds whether or not anyone is watching. Add the delays together and the cycle's effective rhythm, at present, sits within the band where institutions can respond. Governance survives exactly as long as the governors can respond faster than the system changes. In the reactor engineer's terms: the loop economy is governable while its delayed fraction lasts.

The reactor is a vivid case of a general law, and the law has a science. Norbert Wiener founded it in 1948 and named it cybernetics, after the Greek word for helmsman: the study of systems steered by feedback, which is to say by measurements flowing back to a controller in time to matter.[17] Everything in these papers translates into its vocabulary. The specification is what engineers call the setpoint; the working loop is the plant; verification is the sensor, and a gamed test is a spoofed sensor. W. Ross Ashby proved that a regulator must contain at least as much variety as the disturbances it faces, so any governor whose risk models hold less variety than the loops they price fails by theorem rather than by bad luck; a market of independent insurers is the cheapest known way to assemble that variety, and a monoculture governor the easiest way to lose it.[18] The speed requirement has its own canonical statement: Gunter Stein's celebrated Bode Lecture, 'Respect the Unstable', showed that an unstable system places a hard lower bound on how fast its controller must act, with unstable fighter aircraft and Chernobyl among its exhibits.[19] Faster instability demands faster control, and past a certain rate no controller on earth qualifies. The delayed-neutron inequality of the previous paragraph is Stein's law in civilian dress, and this paper is, in the older and exact sense of the word, an essay in cybernetics: the governance of loops, at the speed of loops.

Every source of delay on the list is contingent. Researchers are building systems that improve during deployment rather than waiting for the next training run, which removes the months. Competitive pressure pushes operators to remove the gates, because each signature costs time and the rival without one moves faster. Nothing in mathematics or natural law keeps any of the loop economy's delays in place. The gates are policy, and policy needs an enforcer, and the enforcer's own reflexes must sit inside the governable band, or the enforcement is theatre. Parliaments amend laws in years. Regulators write rules in multi-year cycles. Courts act after the loss. Something has to move faster than all of them.

6. The Institution with the Fast Clock

One institution reprices risk continuously, enforces its rules through that pricing, and stakes its own money on being right.

Strip insurance to its mechanism. A premium is a price paid in advance for bearing a possible loss. To set it, the insurer estimates how often the loss happens and how bad it is when it does, then charges accordingly. Get the estimate wrong in the customer's favour and the insurer bleeds; wrong in the insurer's favour and a competitor takes the customer. The estimate is therefore a living number: it can move as fast as the contract allows, at renewal, at adjustment, in the newest products continuously, and no legislature needs to sit for it to move. The ceiling on its speed is contractual custom rather than institutional nature, which is the point. A premium is a rule about behaviour, written as a price, free to update as fast as its subject changes.

History shows the mechanism governing whole industries before governments arrived. From the 1760s, Lloyd's Register sent surveyors to grade the soundness of ships; underwriters priced insurance on the grade; a badly graded ship faced ruinous premiums; ruinous premiums meant no cargo; and so a private register of surveyors effectively regulated world shipping generations before flag states did.[20] Fire insurers priced buildings by their construction, and the discounts they offered for sprinklers and firewalls wrote those features into factories and warehouses ahead of the statutes that later required them. Legal scholars studying the pattern conclude that insurers already regulate, through pricing and policy conditions, much of what states regulate through rules.[20]

The accelerated form already sits in millions of cars: telematics programmes price the driving record as it forms, some adjusting month by month; the insurer never meets the driver, never petitions a parliament, and governs braking habits all the same. Cyber insurers scan their clients' systems, and what the scans find moves underwriting and, increasingly, terms. Continuous pricing already exists. What does not yet exist is continuous pricing of the thing that now matters: the records that verified work leaves behind, and the speed of the loops that produce it.

7. Pan-Insurance

Insurance today is sold in separate lines: motor, property, professional negligence, cyber. The separation reflects an old fact about risk, that a car crash, a warehouse fire, and a lawyer's error had nothing to do with one another. The loop economy deletes that fact. When every firm's work runs on loops, and the loops are assembled from the same handful of AI models, the same purchased specifications, the same harness templates, then a single upstream flaw surfaces everywhere at once. Picture every bakery in a country buying flour from one mill: one bad batch is every bakery's disaster on the same morning. Insurers have already tasted this future once. A single software vulnerability now produces simultaneous claims across thousands of policies that never mentioned software, a phenomenon the industry named silent cyber, and it moved Lloyd's of London to require policies across its market, in phases, to state their cyber exposure explicitly.[21] The loop economy is silent cyber generalised to everything.

Cover must therefore follow the supply chain of thinking itself. The deepest policies attach where the models are made, insuring a model version the way a mill certifies a batch of flour, with certificates flowing downstream to every loop built on it. The shallowest policies attach to the individual practitioner and her record. Between those layers, one continuous fabric of cover replaces the old separate lines, because the risk is one continuous fabric too. Pan-insurance is insurance reorganised to match the supply chain of machine work: one fabric where there were lines, priced from the records that verified work generates, moving at the speed the work moves. Its premiums do for the loop economy what Lloyd's Register's classifications once did for shipping: they carry the safety information, priced, to everyone who needs it, faster than any regulator can publish.

The underwriting reduces to three questions, each inheriting its logic from an earlier section.

What does the record show, and can the record be trusted? The earlier papers established that loops leave a lineage: every check demanded, run, passed, failed, and repaired. That lineage feeds the frequency half of the actuary's arithmetic, though at one remove: it counts failed checks and repairs, while losses are counted by the world. It must therefore be attested, meaning its origin and completeness are cryptographically vouched for, and it must be anchored to real outcomes, because a record of checks can only show what the checks caught. Records that grade themselves are the exam problem of Section 2 wearing accounting clothes.

How much of my book fails together? An insurer of coastal houses counts how many stand on the same stretch of shore, because one hurricane must not be able to claim them all. The loop insurer counts model versions and spec estates the same way: how many policyholders run the same components, and would fail on the same blind spot on the same day.

How fast does this loop change, and who can say no to it? This question is the new one, the delayed-neutron audit. How many months between the loop's capability generations? How many human signatures stand between it and its own next version? A loop that retrains continuously, with no gates, is a prompt-critical configuration, and the premium schedule should treat it as the reactor engineer would: uninsurable at any price, before it becomes common. Conditions written into policies complete the doctrine, and they have a homely ancestor. A sprinkler clause lowers a fire premium when sprinklers are fitted; the insurer writes the safety rule directly into the price. The loop economy's sprinkler clauses are the separation of the checks from those checked against them, the one-way ratchet on tests, and diversity among the checking components, so that no single flaw silences every alarm at once.

8. How the Governor Fails

A doctrine that cannot describe its own failure is a prospectus. This one fails in two known ways, and both have names and dates.

The first failure is concentration inside the governor. In the years before 2008, one company's financial products division sold protection against mortgage losses to banks across the entire Western financial system. Everyone's risk flowed to one balance sheet, that balance sheet priced the risk wrong, and when mortgages failed the protector failed with them; the American government committed one hundred and eighty-two billion dollars to keeping AIG standing, roughly one hundred and forty-two billion of it drawn and eventually repaid, because the alternative was the system's collapse.[22] The lesson for pan-insurance is direct: a single fabric of cover is itself a monoculture, and concentrating the whole loop economy's risk in one mispriced institution recreates AIG at civilisational scale. The remedy applies the doctrine to its own author. Many insurers, not one; risk models built independently, so their blind spots do not align; the same diversity demanded of the loops, demanded of those who price them. A second, subtler failure accompanies concentration: once premiums depend on records, operators are tempted to dress the records, Goodhart's law returning one level up. Attestation and outcome-anchoring exist to make the dressing detectable, which is why Section 7 stated them as load conditions rather than refinements.

The second failure is the tail. Some possible losses exceed what any private balance sheet, or all of them together with their reinsurers, can absorb. The nuclear industry met this problem at its birth: in the 1950s no American company would build a power reactor, because a single catastrophic accident could exceed any conceivable insurance. Congress answered in 1957 with the Price-Anderson Act, which required operators to carry what private cover existed and placed federal indemnity above it. The modern scheme replaces the indemnity with stacked private layers, each operator's own insurance and then a retrospective assessment levied on every other reactor licensee, with Congress committed to decide what follows if even that is exhausted.[23] Civil nuclear power exists because those lines were drawn. Recursion risk will require its own such line: private premiums governing everyday conduct, sovereign capacity standing above them for the losses no market can hold. Where exactly to draw it is the genuine policy work of the coming decade, and the shape of the settlement is already visible in the public half of the corpus: government tiers deciding what a loop may be, private pricing governing what a loop does.[24] Mandatory cover is itself a state decision, so the scheme is a public-private partnership from its first day. The division of labour follows each partner's clock: conduct to the premium, which moves in minutes, and the tail to the treasury, which moves in emergencies.

9. Predictions and Limits

If this framework is right, the following become visible, roughly in order. An insurer offers cover attached to a specific AI model version, with certificates that downstream businesses can inherit, before any law requires it. Underwriting questionnaires begin asking about training cadence and signature gates, the delayed-neutron audit entering actuarial routine. A measurable premium gap opens between gated and ungated loops, making the discipline of the earlier papers economically compulsory before it is legally so. The separate lines of commercial insurance begin consolidating into loop-risk cover, and a market forms to trade its layers, a Lloyd's of cognition. At least one major jurisdiction legislates a Price-Anderson analogue for AI recursion risk within the decade. Dependence between loops, how they fail together, becomes a funded actuarial research field, because reinsurer solvency comes to require it.

The limits are as important as the predictions, and there are four. Insurance governs only those who buy it, and the largest operators of powerful loops, states themselves, armies, treasuries, self-insure; the governor is weakest precisely where capability runs hottest, and no premium reaches there; that gap is treaty-shaped, and nobody has designed the treaty. Pricing how loops fail together is established craft applied to a data desert: the actuarial methods for dependence and common shocks exist, the loss history they need does not, and model error carries the whole weight; 2008 stands as the permanent measure of what that weight can cost. A premium prices compliance with a specification, never the worth of the specification, so a perfectly insured loop faithfully pursuing a wrong goal passes through every layer of this fabric untouched; judging goals stays human, as the first paper of this series argued it always would. Last and largest: the delayed fraction is contingent. The months and the years and the signatures can all be engineered away, and competition rewards whoever removes them. Keeping some neutrons delayed is the standing price of remaining the author, and the institutions described here are the means by which that price gets collected, continuously, at the only speed that matters.

The difference between compound interest and detonation is the ruler used on the time axis. Pan-insurance is the institution that holds the ruler.

Acknowledgements

Profound thanks to John Allsopp, Alan Eyzaguirre and AJ Fisher, all of whom contributed to my thinking on this topic, and to Philippe van Nedervelde for reviewing the final draft. This paper was composed with the assistance of Claude Fable 5. I remain wholly responsible for any errors that may have crept in.

Notes

1. Mark Pesce, "Defending the Loop: Verification and the Division of Labour in Autonomous Work," The Watershed, July 2026. https://thewatershed.markpesce.com/defending-the-loop-verification-and-the-division-of-labour-in-autonomous-work/
2. Mark Pesce, "The Check and the Firm: Foundations of Post-Watershed Business Practice," The Watershed, July 2026. https://thewatershed.markpesce.com/the-check-and-the-firm-foundations-of-post-watershed-business-practice/
3. Geoffrey Huntley, "everything is a ralph loop," 17 January 2026. https://ghuntley.com/loop/
4. Goodhart's law, in Marilyn Strathern's formulation: "When a measure becomes a target, it ceases to be a good measure." Marilyn Strathern, "'Improving Ratings': Audit in the British University System," European Review 5, no. 3 (1997): 305-321.
5. Dario Amodei et al., "Concrete Problems in AI Safety," arXiv:1606.06565 (2016).
6. I. J. Good, "Speculations Concerning the First Ultraintelligent Machine," Advances in Computers 6 (1965): 31-88.
7. OpenAI, Planar Point Sets with Many Unit Distances (2026); Lean formalisation recorded on the Lean AI formalization leaderboard, 26 June 2026. https://lean-lang.org/eval/problems/erdos_unit_distance_conjecture_false/
8. Jürgen Schmidhuber, "Gödel Machines: Fully Self-referential Optimal Universal Self-improvers," in Artificial General Intelligence, ed. Ben Goertzel and Cassio Pennachin (Berlin: Springer, 2007), 199-226.
9. Eliezer Yudkowsky and Marcello Herreshoff, "Tiling Agents for Self-Modifying AI, and the Löbian Obstacle," early draft, 7 October 2013, Machine Intelligence Research Institute. https://intelligence.org/files/TilingAgentsDraft.pdf
10. Nate Soares, Benja Fallenstein, Eliezer Yudkowsky and Stuart Armstrong, "Corrigibility," AAAI Workshop on AI and Ethics (2015).
11. K. Eric Drexler, Reframing Superintelligence: Comprehensive AI Services as General Intelligence, Technical Report 2019-1 (Oxford: Future of Humanity Institute, 2019).
12. David Silver et al., "A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play," Science 362, no. 6419 (2018): 1140-1144.
13. Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange and Jeff Clune, "Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents," arXiv:2505.22954 (2025).
14. Mark Pesce, "Foundations of Post-Watershed Economics," The Watershed, April 2026. https://thewatershed.markpesce.com/foundations-of-post-watershed-economics/
15. On delayed neutrons and reactor control: U.S. Department of Energy, DOE Fundamentals Handbook: Nuclear Physics and Reactor Theory, DOE-HDBK-1019 (1993), vol. 2.
16. International Nuclear Safety Advisory Group, The Chernobyl Accident: Updating of INSAG-1, INSAG-7 (Vienna: International Atomic Energy Agency, 1992).
17. Norbert Wiener, Cybernetics: Or Control and Communication in the Animal and the Machine (Paris: Hermann & Cie; Cambridge, MA: The Technology Press; New York: John Wiley & Sons, 1948).
18. W. Ross Ashby, An Introduction to Cybernetics (London: Chapman & Hall, 1956), on the law of requisite variety.
19. Gunter Stein, "Respect the Unstable," the 1989 Hendrik W. Bode Lecture, published in IEEE Control Systems Magazine 23, no. 4 (2003): 12-25.
20. George Blake, Lloyd's Register of Shipping, 1760-1960 (London: Lloyd's Register, 1960); Omri Ben-Shahar and Kyle Logue, "Outsourcing Regulation: How Insurance Reduces Moral Hazard," Michigan Law Review 111, no. 2 (2012): 197-248.
21. Lloyd's of London, "Providing clarity for Lloyd's customers on coverage for cyber exposures," Market Bulletin Y5258, 4 July 2019.
22. Financial Crisis Inquiry Commission, The Financial Crisis Inquiry Report (Washington, DC: U.S. Government Publishing Office, 2011), ch. 19, on AIG Financial Products; assistance figures per Congressional Research Service, Government Assistance for AIG, R42953, and U.S. Treasury TARP records.
23. Price-Anderson Nuclear Industries Indemnity Act (1957), 42 U.S.C. § 2210; U.S. Nuclear Regulatory Commission backgrounder: https://www.nrc.gov/reading-rm/doc-collections/fact-sheets/nuclear-insurance.html
24. Mark Pesce, "Turing Police," The Watershed, June 2026. https://thewatershed.markpesce.com/turing-police/