Supplementary Essay 2: Deriving AISOP
A supplementary essay to AGI: How Superhuman Intelligence Reshapes Civilization. Shows how the five AISOP norms for knowledge production follow from Proposition Six, constitutive pluralism.
Last updated: August 2, 2026 (JST)
AISOP is derived from constitutive pluralism, the position advanced by Proposition Six — the book's sole normative proposal. When what constitutive pluralism seeks to protect is taken up within the domain of knowledge production, the code of conduct that follows for individuals is AISOP. The substance of constitutive pluralism itself — the derivation of its foundational layer (relational value and vulnerability) and its three pillars (non-domination, non-aggregation, and slack) — is set out in Supplementary Essay 1, “Deriving Constitutive Pluralism”.
1. The Form of the Derivation
Norms cannot be derived directly from facts. In Chapter 6 the book rejected the argument of e/acc (effective accelerationism) as a naturalistic fallacy — deriving the goodness of acceleration from the physical fact of increasing entropy. AISOP, likewise, must not be derived directly from a description of the fact that the how has been automated.
The derivation therefore takes the form of a hypothetical imperative. We first posit a goal; we then decompose the act of knowledge production into its constituent elements; and we set each element against the conditions of the AI age to derive the norms to be followed. The goal is as follows.
G: That the collective process of knowledge production continue to function in the AI age.
The substance of G divides in two. G1: the preservation of error-correcting capacity (the epistemic condition). G2: the survival of knowledge production as a human practice of asking what and why (the value condition).
2. Deriving the Goal
G cannot — and must not — be derived from Propositions One through Five (the descriptive propositions). G borrows its normative force from Proposition Six. The chain of derivation runs Proposition Six → G → AISOP, and G stands on the same normative footing as Proposition Six. Within this, G1 and G2 differ in the path and the strength of their derivation.
G1 is derived as a condition for the operability of Proposition Six's institutional skeleton (value-setting rights, veto rights, and slack). That skeleton presupposes a functioning collective process of knowledge. Veto rights rest on two grounds. A normative refusal grounded in the non-consent of those affected stands directly on vulnerability and on the standing of the affected party (Supplementary Essay 1). What is at issue here is the other ground: the demand to halt on the grounds that an objective function is in error — a demand that has force only once the error can be detected. And the detection of error cannot exist without a collective process of verification. Value-setting rights presuppose deliberation over what ought to be optimized, and deliberation requires a supply of trustworthy knowledge. The very justification of slack — precisely because we cannot predict, we build into our institutions an adaptive capacity that does not depend on prediction — presupposes the functions of exploration and learning. Chapter 11 states plainly in its concluding section that “this adaptive capacity is the very thing constitutive pluralism seeks to protect.” And the institutionalized form of civilization's adaptive capacity is nothing other than the collective process of knowledge production (Chapter 8: science is collective information compression, and mutual criticism transcends the biases of the individual). Whoever wills this skeleton, therefore, cannot but will G1 as the condition of its operability.
G1 also has a broader basis. Under bounded rationality (Chapter 5), any agent with long-term aims — whatever those aims may be — needs error-correcting capacity. As the case of CFCs shows, even the products of rational optimization can backfire in unforeseeable ways, which is exactly why a structure for learning from error (antifragility) is required. In the sense of a means demanded in common whatever the goal, this amounts to instrumental convergence within the domain of norms (Chapter 5). G1 thus carries near-universal hypothetical binding force even on those who withhold assent from Proposition Six.
G2 is derived by narrowing the object of protection that the book makes explicit down to the domain of knowledge production. The section of Chapter 11 titled “Vulnerability as the Ground” identifies what is to be protected as “the conditions under which finite, interdependent beings can hold questions of their own, encounter others, fail, and choose again.” The practice of asking what and why is nothing other than the exercise, within the domain of knowledge production, of this capacity to “hold questions of one's own.” And this capacity atrophies where it is not exercised. To protect the conditions, therefore, includes preserving the places in which the capacity is exercised. The practice of posing questions is not confined to knowledge production, but knowledge production is one of the few places where that practice enjoys institutional support. Chapter 10 likewise defines UBI as “structural infrastructure enabling human beings to keep asking questions.” There is also a second, independent path. The two-layer Human-over-the-Loop (HOL) model — the division of labor in which AI carries execution in the speed layer while human beings carry value-setting in the legitimacy layer — requires, in the legitimacy layer, humans who set values and update objective functions. If the human side loses the capacity to question, the legitimacy layer becomes a hollow shell and collapses into what the book itself calls the counterfeiting of responsibility. The limiting case is the solid phase Chapter 12 warns of: complete dependence on and indifference toward AI. G2 is the condition for HOL to be substance rather than pretense.
Proposition Six at the institutional level and AISOP at the individual level thus form two layers issuing from the same normative root. The distribution to individuals is not uniform, however. What the five principles demand is a standard to be met by the system of knowledge production as a whole, and which principles weigh most heavily on a given person depends on the role that person plays within the system. Those charged with records and preservation are asked for continuity rather than agility; research under safety constraints is asked for circumscribed independent verification rather than full disclosure.
3. Decomposing the Act
Norms regulate action. How many principles a normative system for knowledge requires is therefore determined by how many constituent elements make up the act of knowledge production. We decompose that act following the account given in the main text.
The main text describes the structure of knowledge production at three levels. First, the level of the individual. The methodology of science forms a closed loop: constructing models from observation, deriving predictions, and testing them (Chapter 7). This loop begins with a question, passes through methods and tools, and ends in consequences. Second, the level of the community. Knowledge is not completed within the individual; it becomes knowledge only through mutual criticism and verification mediated by shared external representations (Chapter 8, science as collective intelligence). Third, the level of society. Knowledge production is embedded in society through the structure of its sponsors, and its consequences act upon society (Chapter 9, and Chapters 10 onward).
From these three levels, the individual's loop supplies method, question, and consequence; the level of the community supplies sharing and verification; the level of society supplies position and effect. The act of knowledge production, that is, decomposes into the following five constituent elements:
- Methods and tools. With what, and how, does one inquire?
- The question. Why, and what, does one ask?
- The community of knowledge. How does one share one's products and submit them to verification?
- Society. Where does one's own knowledge production stand within society?
- Consequences. Who answers, and how, for the validity of the products and for the results they bring about in society?
The five elements are practices of distinct modes — using, questioning, sharing and submitting to verification, being positioned, and answering for — and are not reducible to one another. A normative system must give an answer for every element that the conditions of the age render problematic (the requirement of completeness), and it places one principle on each element (the requirement of minimality).
4. The Conditions of the Age and the Five Principles
The decomposition of the act transcends any particular age, but the answer each element should receive depends on the conditions of the age. From the descriptive analysis running through Chapter 9, we extract four conditions that set knowledge production in the AI age apart from what preceded it.
- C1: The automation of how. The scarcity of method and execution disappears, and the obsolescence of tools accelerates (Chapter 9; the situation in which technical trends shift week by week is described in Chapter 12).
- C2: The growing scarcity of motivation. When the production of answers becomes abundant, what becomes scarce are questions (what and why) and the motivation to pose them (Chapter 9; the inversion of scarcity in Chapter 10).
- C3: The transformation of the structure of verification. This has two faces. C3a: as AI-generated artifacts proliferate, the reliability of knowledge can no longer be guaranteed by the individual virtue of its producer and comes to depend on collective verification mediated by shared external representations (Chapter 8's collective predictive coding, CPC). C3b: a system that has generated something cannot certify its own correctness; verification requires agents independent of the system that did the generating (Chapter 8).
- C4: Direct social intervention. Knowledge production acts directly on the operation of social systems (Chapters 10 through 12).
These four conditions render all five elements problematic. Setting the elements against these conditions yields the five principles, one by one.
- Setting the question against C2 yields Intrinsic motivation. Under the condition that motivation becomes a scarce resource, the norm governing this element becomes: do not subordinate the grounds of your questions to any single external metric; do not let the intrinsic source run dry. Questions arising from public need or from commission are not excluded. What is excluded is the state in which the supply of questions is held entirely by commissions and evaluation metrics. This is the core.
- From methods and tools together with C1 comes Agility. Under the condition that methods become obsolete at high speed, the norm becomes: do not cling to any particular method; respond promptly to updates. That agility commands responsiveness rather than conservatism is because the very tools of verification are themselves being updated. Those who fail to keep pace with updates do not thereby avoid error; they drop out of the collective process by which error is detected. Caution in adoption is carried separately by Professionalism, discussed below.
- From the community together with C3a comes Openness. Under the condition that verification can be constituted only collectively, the norm becomes: share data, code, and protocols in verifiable form. Sharing is not unbounded, however. The requirement is that the necessary information reach multiple verifiers independent of the generating system; full disclosure, limited sharing, and third-party audit are distinguished as means to that end. Disclosure of a kind that encourages the linkage of data across contexts is what non-aggregation (Supplementary Essay 1) forbids, and it cannot be justified in the name of Openness. The protection of those affected and the constraints of safety likewise sit inside this principle.
- From consequences together with C3b comes Professionalism (expertise and responsibility). Under the condition that verification must be independent of the generating system, the norm becomes: verify for yourself on the foundation of deep expertise, and accept the responsibility of explanation. Human beings do not monopolize the work of verification itself. Provided independence is preserved, another system may carry it out (Chapter 8). What remains on the human side is the judgment of what to do with the product, and the acceptance of the consequences (Chapter 11's HOL and the standing of those affected). What this principle demands is both the capacity to organize independent verification and the standing to bear the consequences.
- From society together with C4 comes Social awareness. Under the condition that knowledge production intervenes directly in society, the norm becomes: carry a map of your own position.
The A-I-S-O-P ordering is a mnemonic order, not a logical one. Logically, Intrinsic motivation stands at the core, and the other four principles surround it. This is a formal restatement of the main text's remark that intrinsic motivation supports the other principles as their precondition.
Remove any one of the five, and the remaining four cannot substitute for it. Remove Agility, and methods that go unupdated chain the standards of questioning and verification to old tools. Remove Intrinsic motivation, and the supply of questions becomes subordinate to external evaluation metrics, with what gets asked decided by the interests of whoever commissions the work. Remove Openness, and collective verification fails to operate; errors remain unshared, lodged within each individual. Remove Professionalism and let Openness double for consequences, and shared errors may well be verifiable, but no one answers for them, and responsibility diffuses. Remove Social awareness, and the effects of knowledge production upon society fall outside the field of view; errors of position become invisible precisely to those who commit them. In every case, at least one component of G is damaged. This, however, is the deficit that follows when each principle is taken exactly as defined in this essay; it is not a proof that the gap could not be filled even by stretching the remaining four to their utmost. Minimality is a requirement, not a proven property.
The principles also differ in the strength of their binding force. Openness and Professionalism draw their force mainly from G1, and by G1's near-universality they bind a wide readership. Intrinsic motivation and Social awareness draw theirs mainly from G2, and bind those readers who accept the account of the human being — relational value and vulnerability — that underlies Proposition Six. Agility straddles both.
The glosses Chapter 9 gave the five principles — Agility as “swiftly taking up new tools” (method), Intrinsic motivation as “the power to pose questions rooted in one's own intellectual curiosity” (question), Openness as “the sharing of data, code, and protocols” (community), Social awareness as “where one stands in society, and whom one may affect and how” (society), Professionalism as “the responsibility to verify and explain for oneself” (consequence) — point to the five constituent elements one by one. The decomposition given here makes explicit an arrangement the main text had performed implicitly.
5. Correspondence with CUDOS and PLACE
If the decomposition of the act transcends the age, then the older normative systems should be legible on the same five elements. And indeed CUDOS1 concentrates on the community (communism, universalism, organized skepticism), while disinterestedness can be read as a discipline of the question. It places no principle on method, consequence, or society. PLACE2 spans all five elements (proprietary = community, local = society, authoritarian = method, commissioned = question, expert = consequence). Not every item of the old systems fits within exactly one element; the assignments here follow each item's principal focus. When Chapter 9 says that social awareness is the axis missing from both CUDOS and PLACE, it means not a vacant element but an absent answer: PLACE's “local” occupied the element of society with the answer of subordination, not with a map of one's position. What changed with the succession of normative systems was not the structure of the act but the conditions of the age. What Chapter 9 narrated historically as a succession of sponsors is precisely this succession of conditions. From this vantage point, the destinations of the old principles can be set out.
| Old principle | Destination | Relation |
|---|---|---|
| Communism (C) | Openness | Inheritance and extension (from papers to data, code, and protocols) |
| Universalism (U) | Openness | Absorption (evaluation by evidence rather than by person is implemented as verification through shared artifacts) |
| Disinterestedness (D) | Intrinsic motivation | Inversion (see below) |
| Organized Skepticism (OS) | Openness + Professionalism | Distribution of function (see below) |
| Proprietary (P) | Openness | Opposition (though not total negation; staged disclosure on safety grounds is handled by Chapter 12's controlled multipolarity — what the glossary calls open-weight governance) |
| Local (L) | Social awareness | Sublimation (from subordination to an agenda, to a map of one's position) |
| Authoritarian (A) | Agility | Dissolution (the authority of methods and organizations cannot withstand high-speed updating) |
| Commissioned (C) | Intrinsic motivation | Antithesis (made explicit in the main text) |
| Expert (E) | Professionalism | Inheritance, with responsibility added |
Two of these transitions require argument.
The inversion from disinterestedness to Intrinsic motivation. Merton's disinterestedness performed two functions: (a) the control of bias, and (b) the discipline of motivation. In the AI age, (a) is institutionally transferred to collective verification through Openness and to mutual monitoring among diverse agents. This is a transition of the same shape as Chapter 5's relocation of control, from the control of a single AI to mutual monitoring within an ecosystem of AIs — a transfer from individual virtue to structural property (Merton himself treated disinterestedness not as individual psychology but as an institutional pattern). As for (b), in an age when the production of answers has become abundant, the problem itself changes. What requires control is not an excess of motivation but its exhaustion. The focus of the norm therefore inverts: from keeping private interest out of judgment to sustaining intrinsic motivation.
The destination of skepticism. The function of organized skepticism is distributed between Openness (sharing in verifiable form is what makes collective skepticism possible) and Professionalism (judging the validity of AI-generated hypotheses). That skepticism is not preserved as an independent principle follows from the book's consistent position that the efficacy of verification is carried not by individual virtue but by the structure of verification (Chapter 5; the non-domination of Chapter 11).
6. Candidates for a Sixth Principle
Most candidates for a sixth principle can be mapped onto one of the five constituent elements, or are carried by another layer of the book. One case, however, remains open.
- Integrity (research integrity). Implied at the intersection of Openness (sharing in verifiable form) and Professionalism (accountability). It is not made independent because the efficacy of integrity is carried by the structure of verification, not by individual virtue.
- Humility (epistemic humility). The acceptance of bounded rationality (Chapter 5) is part of the substance of Social awareness (carrying a map of one's own position) and of Professionalism (the responsibility to verify); it requires no independent principle.
- Sustainability and concern for future generations. Absorbed as the extension of Social awareness along the axis of time. Elevating it to an independent principle carries the risk of sliding, under value indeterminacy (Chapter 8), into longtermist expected-value maximization. The book answers that problem with the design of slack in Chapter 11, and we refrain from stacking a second norm with the same function.
- Care and relationality. The addressee of this norm lies outside the act of knowledge production (human relations in general) and is carried by Chapter 11's relational value. It flows into AISOP through Social awareness and Intrinsic motivation.
- Retaining disinterestedness. As argued in the previous section, it has already been inverted.
- Education and transmission (open). The work of forming the next generation of practitioners has been institutionalized inseparably from knowledge production, from apprenticeship to the Humboldtian university. Chapter 12, too, treats education as the formation of practitioners. One can absorb it as the temporal aspect of the community of knowledge — that is, as the reproduction of verifiers. But that absorption amounts only to a redescription of function, and it fails the criterion stated next: teaching, learning, and handing down can be described as a mode distinct from sharing and submitting to verification. This case is left open. If a sixth constituent element is established, this is its likeliest location.
To establish a sixth principle, one must exhibit a sixth constituent element not reducible to these five. The criterion of judgment is this: if the candidate is a sixth mode standing alongside the five modes of using, questioning, sharing and submitting to verification, being positioned, and answering for, then it is an element. If it merely specifies how an existing mode is to be performed, then it is the content of a principle, not a new element. Should a sixth element be exhibited, AISOP ought to be revised.
7. Limits and Falsification Conditions
This derivation has four limits.
First, the derivation is hypothetical, and its binding force divides into two tiers. G1 (error correction), by way of bounded rationality, binds hypothetically nearly any agent with long-term aims. By contrast, G2 (the survival of the questioning practice), together with Intrinsic motivation and Social awareness, which draw their force mainly from it, binds those who accept the account of the human being — relational value and vulnerability — that underlies Proposition Six. Whoever rejects Proposition Six is not bound by AISOP as a whole; but that is the same status the book gave to Proposition Six itself.
Second, the decomposition into five is itself one choice of carving. That there is no unique answer to the question of what counts as a constituent element of the act is precisely the point at which Chapter 7's discussion of ontological complexity (the carving itself — what one counts as an element — constitutes part of the problem) rebounds upon the book itself; a different carving could yield a different set of principles. What the derivation shows is not necessity but non-arbitrariness — traceability from the premises. Nor is it settled that five exhaust the matter: education and transmission remains open as a candidate for a sixth constituent element (Section 6).
Third, the four conditions are descriptive predictions, and they may fail. If motivation does not become scarce, the arrangement that places Intrinsic motivation at the core collapses. If the efficacy of verification reverts to individual virtue, the division of labor between Openness and Professionalism will require recomposition. If a condition fails, so does the derivation of the corresponding principle.
Fourth, the final test of a norm lies not in its derivation but in the results of its operation. CUDOS was not derived; it was described after the fact from a practice that was already working. The same is true of PLACE. Yet wide adoption does not make a norm desirable: the fact that PLACE became dominant did not prevent Chapter 9 from treating it critically. AISOP is a proposal that precedes description, and what will be tested is whether, in the communities that adopt it, the detection and correction of error actually operate, the diversity of questions is preserved, and responsibility for consequences can be identified. If these are impaired, it is the norm that must be revised.
Notes
See Robert K. Merton, “The Normative Structure of Science” (first published 1942), collected in The Sociology of Science: Theoretical and Empirical Investigations (University of Chicago Press, 1973). Merton treats the four norms not as individual psychology but as an institutional pattern of conduct sustained by publicity, mutual monitoring, and sanction.
See John Ziman, Real Science: What It Is, and What It Means (Cambridge University Press, 2000).