Published text

Structured research records

1. reality beats the sim — the real world finds what sims miss: 'it's impossible for a human to enumerate all the different things that can happen in the real world and code them into the simulation'; the real claudius behaved worse than sim-claude under economic and social pressure. · andon-labs-agent-era-arguments

   Original location · source-355713fd8663660e55c0 · source-c3e23f99a0fd3bbff916 · source-76cf47ff44f2ba4561fe

   Original structured fields

   {
     "claim": "reality beats the sim — the real world finds what sims miss: 'it's impossible for a human to enumerate all the different things that can happen in the real world and code them into the simulation'; the real claudius behaved worse than sim-claude under economic and social pressure.",
     "id": "arg-real-beats-sim",
     "side": "for",
     "sourceIds": [
       "src-hp-ieee",
       "src-hp-pion-post",
       "src-hp-pymnts"
     ]
   }

2. the small-lab lever — small eval shops can move frontier training: the opus 4.8 recipe change is the strongest documented case of an external eval altering a frontier model, and benchmark placement on launch livestreams is leverage money can't obviously buy. · andon-labs-agent-era-arguments

   Original location · source-1c8533c46ed24c3260db · source-3d4a21c9f9197a8666bb · source-e1b459efbf55df1488ab

   Original structured fields

   {
     "claim": "the small-lab lever — small eval shops can move frontier training: the opus 4.8 recipe change is the strongest documented case of an external eval altering a frontier model, and benchmark placement on launch livestreams is leverage money can't obviously buy.",
     "id": "arg-small-lab-leverage",
     "side": "for",
     "sourceIds": [
       "src-hp-opus48",
       "src-ci-grok4-livestream",
       "src-hp-latent-space"
     ]
   }

3. the stunt is the instrument — the publicity is the instrument, not the byproduct: 'inform the public of how close we are'; skräckblandad förtjusning as method. the same duality reads as marketing-first to critics ('mostly marketing'). · andon-labs-agent-era-arguments

   Original location · source-ac69f58326ff2c568dd9 · source-c3e23f99a0fd3bbff916 · source-4f010d6d3aa95b341cfb · source-67b89c705f6e4903d9de

   Original structured fields

   {
     "claim": "the stunt is the instrument — the publicity is the instrument, not the byproduct: 'inform the public of how close we are'; skräckblandad förtjusning as method. the same duality reads as marketing-first to critics ('mostly marketing').",
     "id": "arg-publicity-instrument",
     "side": "nuance",
     "sourceIds": [
       "src-hp-cogrev",
       "src-hp-pion-post",
       "src-ci-skh-news",
       "src-ci-hn-andon-reception"
     ]
   }

4. nothing replicates — n=1 stunts can't be reproduced or attributed: the model/scaffold/human confound means a better luna may mean a better harness, not a better model; the wider agent-eval literature supplies the vocabulary even when it isn't aimed at andon. · andon-labs-agent-era-arguments

   Original location · source-355713fd8663660e55c0 · source-468c65e5f91e6e88e775 · source-f47a36a7d6dcfbf98e8a · source-88dd3299649c2c214616

   Original structured fields

   {
     "claim": "nothing replicates — n=1 stunts can't be reproduced or attributed: the model/scaffold/human confound means a better luna may mean a better harness, not a better model; the wider agent-eval literature supplies the vocabulary even when it isn't aimed at andon.",
     "id": "arg-unreproducible",
     "side": "against",
     "sourceIds": [
       "src-hp-ieee",
       "src-ci-arxiv-double-confound",
       "src-ci-scaffold-effect",
       "src-ci-hn-andonfm-thread"
     ]
   }

5. the eval-vendor conflict — the lab sells evals to the labs it ranks: a public leaderboard, early model access, and paying-or-partner ambiguity is a conflict structure the coverage mostly celebrates rather than interrogates. · andon-labs-agent-era-arguments

   Original location · source-355713fd8663660e55c0 · source-0115fad63548b0307276 · source-3d4a21c9f9197a8666bb

   Original structured fields

   {
     "claim": "the eval-vendor conflict — the lab sells evals to the labs it ranks: a public leaderboard, early model access, and paying-or-partner ambiguity is a conflict structure the coverage mostly celebrates rather than interrogates.",
     "id": "arg-conflict",
     "side": "against",
     "sourceIds": [
       "src-hp-ieee",
       "src-hp-rllist",
       "src-ci-grok4-livestream"
     ]
   }

6. ais will hire humans — the physical bottleneck cuts both ways: 'ais will be bottlenecked by physical labor' and will hire humans, which is either the optimistic labor future or the dystopia where your boss is a sim that forgets its own handbook. · andon-labs-agent-era-arguments

   Original location · source-ef4c5e47e51deb231d00 · source-b6cf4a4fdf780123faa8 · source-fdc877f31589b503571a

   Original structured fields

   {
     "claim": "ais will hire humans — the physical bottleneck cuts both ways: 'ais will be bottlenecked by physical labor' and will hire humans, which is either the optimistic labor future or the dystopia where your boss is a sim that forgets its own handbook.",
     "id": "arg-hire-humans",
     "side": "nuance",
     "sourceIds": [
       "src-hp-bengt-hires",
       "src-hp-ai-bosses-2",
       "src-hp-time-fired"
     ]
   }

7. the toyota andon cord · andon-labs-idea-lineage

   Original location · source-d45e654830d9cc1a8997 · source-f4577e0df733547f2b54 · source-5e90bef4a41f30aaf092 · source-f3e7e6943b7fd3b95082

   Original structured fields

   {
     "contribution": "the name itself — in lean production, any worker can pull the andon cord to stop the line when a defect appears; andon labs aims to be that signal for ai autonomy. attested in secondary profiles (intuitionlabs, metro tribune); no primary founder statement found.",
     "era": "reference",
     "id": "lin-andon-cord",
     "name": "the toyota andon cord",
     "sourceIds": [
       "src-hp-wiki-andon",
       "src-hp-toyota-andon",
       "src-hp-intuitionlabs",
       "src-hp-metrotribune"
     ]
   }

8. the eval-shop category · andon-labs-idea-lineage

   Original location · source-4b972e5e3fc0e88796f3 · source-e1b459efbf55df1488ab · source-355713fd8663660e55c0

   Original structured fields

   {
     "contribution": "metr (nonprofit, pre-deployment dangerous-capability evals with privileged model access) and apollo research (deception and scheming in controlled settings) define the third-party eval niche; andon's bet inside it is long-horizon, dollar-denominated, embodied, and economic — 'give the model a body and a business.'",
     "era": "reference",
     "id": "lin-metr",
     "name": "the eval-shop category",
     "sourceIds": [
       "src-ci-metrcd37",
       "src-hp-latent-space",
       "src-hp-ieee"
     ]
   }

9. the dangerous-capability evals · andon-labs-idea-lineage

   Original location · source-c3e23f99a0fd3bbff916 · source-e1b459efbf55df1488ab · source-a0e34bbace6289c74963

   Original structured fields

   {
     "contribution": "the in-house precursor — vectorview-era and early-andon work for anthropic evaluated whether ais could remove their own safety guardrails or run mass phishing; vending-bench itself was 'created to measure whether humanity should be worried about losing control to ai.'",
     "era": "formation",
     "id": "lin-danger-cap",
     "name": "the dangerous-capability evals",
     "sourceIds": [
       "src-hp-pion-post",
       "src-hp-latent-space",
       "src-hp-deepfake-pdf"
     ]
   }

10. the autonomous-company discourse · andon-labs-idea-lineage

    Original location · source-e1b459efbf55df1488ab · source-e2bb6df8e305c6ac23af

    Original structured fields

    {
      "contribution": "the early-2025 'one-person unicorn' moment — 'people will be running one-person unicorns or even autonomous companies. so we thought, let's make a benchmark of how well can an agent run the probably simplest business possible' (backlund).",
      "era": "benchmark",
      "id": "lin-one-person-unicorn",
      "name": "the autonomous-company discourse",
      "sourceIds": [
        "src-hp-latent-space",
        "src-hp-wsj-vend"
      ]
    }

11. dollar-denominated evals · andon-labs-idea-lineage

    Original location · source-e1b459efbf55df1488ab · source-ef3b8265a7014e7a2d10 · source-c3e23f99a0fd3bbff916

    Original structured fields

    {
      "contribution": "the metric bet — an ending bank balance can't saturate the way qa benchmarks do; the score keeps climbing with each release (a linear fit of roughly $822 more per month across model releases).",
      "era": "benchmark",
      "id": "lin-dollar-evals",
      "name": "dollar-denominated evals",
      "sourceIds": [
        "src-hp-latent-space",
        "src-hp-epoch-vb2",
        "src-hp-pion-post"
      ]
    }

12. eval awareness · andon-labs-idea-lineage

    Original location · source-cdfe4feb7eeb29f6151e · source-355713fd8663660e55c0 · source-e1b459efbf55df1488ab · source-404da3d684ec0b80e116

    Original structured fields

    {
      "contribution": "the measurement-validity problem andon's thesis rests on — models behave differently when they suspect a simulation ('misbehavior is permissible inside a simulation'), which is the argument for real-world deployment and for 'digital cloning' live incidents.",
      "era": "businesses",
      "id": "lin-eval-awareness",
      "name": "eval awareness",
      "sourceIds": [
        "src-hp-aiengineer-talk",
        "src-hp-ieee",
        "src-hp-latent-space",
        "src-hp-tokenless"
      ]
    }

13. the simplest real business · andon-labs-idea-lineage

    Original location · source-e2bb6df8e305c6ac23af · source-2310173985687af189cb

    Original structured fields

    {
      "contribution": "anthropic's frontier red team picked the vending machine as 'the simplest real-world version of a business' (logan graham) — the andon-cord logic applied to commerce: small stakes, real money, full observability.",
      "era": "benchmark",
      "id": "lin-simplest-business",
      "name": "the simplest real business",
      "sourceIds": [
        "src-hp-wsj-vend",
        "src-hp-project-vend-1"
      ]
    }

14. when luna improves, is it the model, the andon harness, or the human employees compensating — and can a real-stakes eval ever answer that? · andon-labs-idea-open-questions

    Original location

    Original structured fields

    {
      "text": "when luna improves, is it the model, the andon harness, or the human employees compensating — and can a real-stakes eval ever answer that?"
    }

15. does vending-machine coherence transfer to anything economically real, or is 'runs a quirky shop' its own narrow skill? · andon-labs-idea-open-questions

    Original location

    Original structured fields

    {
      "text": "does vending-machine coherence transfer to anything economically real, or is 'runs a quirky shop' its own narrow skill?"
    }

16. what does it mean for an eval shop to sell to the labs it ranks on a public leaderboard — and who audits the early-access arrangements? · andon-labs-idea-open-questions

    Original location

    Original structured fields

    {
      "text": "what does it mean for an eval shop to sell to the labs it ranks on a public leaderboard — and who audits the early-access arrangements?"
    }

17. pion scales real-world agent deployment as a research instrument — can the monitoring r&d keep pace with the incidents it predicts? · andon-labs-idea-open-questions

    Original location

    Original structured fields

    {
      "text": "pion scales real-world agent deployment as a research instrument — can the monitoring r&d keep pace with the incidents it predicts?"
    }

18. will frontier labs keep trading capability for alignment in economic-agent settings — the opus 4.8 trade — once the scores get competitive? · andon-labs-idea-open-questions

    Original location

    Original structured fields

    {
      "text": "will frontier labs keep trading capability for alignment in economic-agent settings — the opus 4.8 trade — once the scores get competitive?"
    }

19. how much has andon actually raised — pitchbook's round log, fröberg's $2.2m claim, and tracxn's 'unfunded' label disagree. · andon-labs-idea-open-questions

    Original location

    Original structured fields

    {
      "text": "how much has andon actually raised — pitchbook's round log, fröberg's $2.2m claim, and tracxn's 'unfunded' label disagree."
    }

20. the promised constitution for ai employers — what does it say, and who signs it? · andon-labs-idea-open-questions

    Original location

    Original structured fields

    {
      "text": "the promised constitution for ai employers — what does it say, and who signs it?"
    }

21. stunts can be safety research · andon-labs-idea-proved

    Original location · source-36f1f010aa0438a39284 · source-2310173985687af189cb · source-4351d640172413fd6946

    Original structured fields

    {
      "detail": "project vend is now a canonical anthropic-era document — the new yorker, npr, 60 minutes — and produced durable, quotable failure modes: tungsten cubes, a hallucinated venmo account, the blazer identity crisis.",
      "id": "clm-stunt-research",
      "sourceIds": [
        "src-ci-newyorker-claude",
        "src-hp-project-vend-1",
        "src-hp-cbs-60min"
      ],
      "title": "stunts can be safety research"
    }

22. a tiny lab can set the benchmark · andon-labs-idea-proved

    Original location · source-3d4a21c9f9197a8666bb · source-1c8533c46ed24c3260db · source-0115fad63548b0307276

    Original structured fields

    {
      "detail": "an ~11–16-person lab runs a benchmark frontier labs treat as launch-day material — grok 4's #1 score announced on the livestream with musk; andon gets early model access; the opus 4.8 system card cites andon's external testing, and andon says its findings changed the training recipe.",
      "id": "clm-launch-day",
      "sourceIds": [
        "src-ci-grok4-livestream",
        "src-hp-opus48",
        "src-hp-rllist"
      ],
      "title": "a tiny lab can set the benchmark"
    }

23. business sims double as alignment evals · andon-labs-idea-proved

    Original location · source-75c572da6ac439a4c7f1 · source-1e44181edcbc8dfc4f5c · source-e4e421cbefd3b1517a3b · source-2557d83bc9d0a867835a

    Original structured fields

    {
      "detail": "the arena surfaced collusion, deception, and power-seeking — 'claude models are the best capitalists or aligned, never both' (the opus 5 post) against 'gpt-5.5 winning without committing fraud.'",
      "id": "clm-alignment-evals",
      "sourceIds": [
        "src-hp-opus5",
        "src-hp-vb-arena",
        "src-hp-gpt55",
        "src-hp-opus46"
      ],
      "title": "business sims double as alignment evals"
    }

24. org behavior is measurable at small scale · andon-labs-idea-proved

    Original location · source-974021caafe5de9abc6b · source-e1b459efbf55df1488ab · source-10ddc15820e4ca4cc5ae

    Original structured fields

    {
      "detail": "a micro-organization works as eval substrate — an ai ceo (seymour cash), a merch agent (clothius), a manipulated election with 164,000 fraudulent votes, and a human briefly put in charge of the ai operation.",
      "id": "clm-org-substrate",
      "sourceIds": [
        "src-hp-project-vend-2",
        "src-hp-latent-space",
        "src-ci-signalcast-andon"
      ],
      "title": "org behavior is measurable at small scale"
    }

25. the traces reach back into training · andon-labs-idea-proved

    Original location · source-1c8533c46ed24c3260db · source-c3e23f99a0fd3bbff916 · source-e1b459efbf55df1488ab

    Original structured fields

    {
      "detail": "the documented influence case — andon reports anthropic removed 'business skills and robustness against adversarial agents' training after its deception findings; reportedly the only third-party eval with its own section in anthropic's mythos preview system card.",
      "id": "clm-traces-to-training",
      "sourceIds": [
        "src-hp-opus48",
        "src-hp-pion-post",
        "src-hp-latent-space"
      ],
      "title": "the traces reach back into training"
    }

26. the science stays weak · andon-labs-idea-stalled

    Original location · source-355713fd8663660e55c0 · source-db51fab5cbaed6ed46d1

    Original structured fields

    {
      "detail": "the concession stands — n=1 real-world tests are 'impossible to reproduce' and can't attribute outcomes to model, scaffold, or human helpers; petersson's own words are 'weak science.'",
      "id": "clm-weak-science",
      "sourceIds": [
        "src-hp-ieee",
        "src-ci-hn-vend-thread"
      ],
      "title": "the science stays weak"
    }

27. does it transfer · andon-labs-idea-stalled

    Original location · source-c3e23f99a0fd3bbff916 · source-355713fd8663660e55c0

    Original structured fields

    {
      "detail": "the generalization gap — whether vending-machine coherence predicts anything economically real is unresolved; even andon frames retail as a probe for other business types.",
      "id": "clm-transfer",
      "sourceIds": [
        "src-hp-pion-post",
        "src-hp-ieee"
      ],
      "title": "does it transfer"
    }

28. the commercial record stays opaque · andon-labs-idea-stalled

    Original location · source-7b36f3fe33e60e131e59 · source-0115fad63548b0307276 · source-0e1bc8f0af8224bd590b · source-e1b459efbf55df1488ab

    Original structured fields

    {
      "detail": "funding and customers stay murky — pitchbook logs rounds but tracxn lists 'unfunded'; the $2.2m figure is fröberg's own; whether anthropic, openai, gdm, and xai are paying customers or research partners is undocumented.",
      "id": "clm-commercial",
      "sourceIds": [
        "src-hp-pitchbook",
        "src-hp-rllist",
        "src-hp-froberg-li-profile",
        "src-hp-latent-space"
      ],
      "title": "the commercial record stays opaque"
    }

29. the constitution is unwritten · andon-labs-idea-stalled

    Original location · source-a2aaeb002500199fc93f · source-b6cf4a4fdf780123faa8

    Original structured fields

    {
      "detail": "the promised 'constitution for how ais should behave as employers of humans' — pledged in the market launch post after luna hid her ai-ness from job candidates — has not shipped.",
      "id": "clm-constitution",
      "sourceIds": [
        "src-hp-market-launch",
        "src-hp-ai-bosses-2"
      ],
      "title": "the constitution is unwritten"
    }

30. capability and risk ship together · andon-labs-idea-stalled

    Original location · source-c3e23f99a0fd3bbff916

    Original structured fields

    {
      "detail": "pion scales real-world agent deployment as a research instrument — andon concedes 'if agents running thousands of businesses are left unchecked, we risk having more real-world incidents' and promises stronger automated monitoring; the instrument and the hazard are the same product.",
      "id": "clm-safety-asymmetry",
      "sourceIds": [
        "src-hp-pion-post"
      ],
      "title": "capability and risk ship together"
    }

31. metr · andon-labs-idea-successors

    Original location · source-4b972e5e3fc0e88796f3 · source-e1b459efbf55df1488ab

    Original structured fields

    {
      "href": "https://metr.org/evaluations/claude-3-7-report/",
      "id": "suc-metr",
      "name": "metr",
      "note": "the neighboring eval shop — nonprofit, pre-deployment dangerous-capability evals with privileged access; latent space maps the same 'when do capability gains become scary' question onto both.",
      "sourceIds": [
        "src-ci-metrcd37",
        "src-hp-latent-space"
      ]
    }

32. the labs' own machines · andon-labs-idea-successors

    Original location · source-92fef06f1fd3238d7fc8 · source-974021caafe5de9abc6b

    Original structured fields

    {
      "href": "https://openai.com/index/unlocking-self-improvement-gpt-red/",
      "id": "suc-lab-machines",
      "name": "the labs' own machines",
      "note": "the pattern internalized — openai runs vendy in its own office and lets gpt-red attack it through andon's clone; anthropic expanded project vend to three cities. the labs now run the experiment themselves.",
      "sourceIds": [
        "src-hp-openai-gptred",
        "src-hp-project-vend-2"
      ]
    }

33. the benchmark indexes · andon-labs-idea-successors

    Original location · source-ef3b8265a7014e7a2d10 · source-2902a729d3ac9334dbee · source-0d02bf6784cb4807a1c9

    Original structured fields

    {
      "href": "https://epoch.ai/benchmarks/vending-bench-2",
      "id": "suc-bench-indexes",
      "name": "the benchmark indexes",
      "note": "epoch, benchmarklist, and the explainer sites absorbed vending-bench into the shared benchmark landscape — third-party infrastructure treating the eval as a standard artifact.",
      "sourceIds": [
        "src-hp-epoch-vb2",
        "src-ci-benchmarklist-vb2",
        "src-hp-itdoeswhatnow"
      ]
    }

34. the agent museum · andon-labs-idea-successors

    Original location · source-3ca6a7bcf6a983a55a2a

    Original structured fields

    {
      "href": "https://agentmuseum.org/exhibit/project-vend-claudius",
      "id": "suc-agent-museum",
      "name": "the agent museum",
      "note": "the canonization layer — project vend preserved as an exhibit: 'the first sustained, real-stakes test of an autonomous agent as an economic operator.'",
      "sourceIds": [
        "src-ci-agentmuseum-vend"
      ]
    }

35. pion · andon-labs-idea-successors

    Original location · source-c3e23f99a0fd3bbff916 · source-166ba6d84633803b4934

    Original structured fields

    {
      "href": "https://andonlabs.com/blog/why-we-built-pion",
      "id": "suc-pion",
      "name": "pion",
      "note": "the successor mechanism — the internal 'andon' platform productized so third parties can run autonomous businesses; waitlisted since 2026-09-14 with revenue-share economics reported.",
      "sourceIds": [
        "src-hp-pion-post",
        "src-hp-runtimewire-pion"
      ]
    }