Same pitch, different end-games
Five simulated stakeholders at a fictional freight operator read one fictional startup's pitch in three positionings and deliberate by mail. Across 120 runs they engaged the vendor 83%, 83% and 80% of the time (enterprise, research, upside): what the simulator produced, not an estimate of what real executives would do.
The question
Holding facts, price and diligence fixed, does a pitch's positioning change the institutional end-game? Preregistered outcome: the share of runs that end engaged with the vendor.
Holding the vendor's facts, price and diligence answers fixed, does the positioning of a pitch change which institutional end-game an organisation reaches? The preregistered outcome is the share of runs that end engaged with the vendor (a pilot, a partnership or acquisition exploration), compared between strategic-upside and enterprise positioning and between research-institution and enterprise positioning, paired by seed. It would be falsified, in this simulator, by an upside or research arm whose engagement rate differs from enterprise by more than the replicate arm's noise.
Neither preregistered contrast cleared the bar: positioning did not detectably change how often the organisation engaged the vendor (strategic upside −3 pp, research ±0 pp against enterprise; Holm p = 1.000 and 1.000). The negative control, the same enterprise pitch with independent model samples, differed by +17 pp (p = 0.302) and reached the same terminal state as enterprise in 50% of seeds, which is the run-to-run noise any other difference has to be read against.
Where the runs ended
| Terminal state | Enterprise software | Research institution | Strategic upside | Enterprise (replicate) |
|---|---|---|---|---|
| Pilot | 25 (83%) | 25 (83%) | 24 (80%) | 20 (67%) |
| Partnership | 0 (0%) | 0 (0%) | 0 (0%) | 0 (0%) |
| Acquisition exploration | 0 (0%) | 0 (0%) | 0 (0%) | 0 (0%) |
| Internal build | 4 (13%) | 3 (10%) | 5 (17%) | 4 (13%) |
| Rejected | 0 (0%) | 0 (0%) | 0 (0%) | 1 (3%) |
| No decision (10 weeks) | 1 (3%) | 2 (7%) | 1 (3%) | 5 (17%) |
| Engaged (primary) | 25 (83%; 66%–93%) | 25 (83%; 66%–93%) | 24 (80%; 63%–90%) | 20 (67%; 49%–81%) |
| Contrast | Difference | 95% CI (seed bootstrap) | Seeds that differ | Exact p | Holm p |
|---|---|---|---|---|---|
| Strategic upside − Enterprise software | −3 pp | −23 pp to +17 pp | 9 of 30 | 1.000 | 1.000 |
| Research institution − Enterprise software | ±0 pp | −20 pp to +20 pp | 8 of 30 | 1.000 | 1.000 |
| Enterprise software − Enterprise (replicate) | +17 pp | −10 pp to +40 pp | 15 of 30 | 0.302 | control |
The test enumerates every sign assignment of the seeds' paired differences (exact sign-flip), so it assumes nothing about the distribution; the interval resamples seeds. With 30 pairs, a difference has to be large to be detected: this design can rule out large positioning effects inside the simulator, not small ones.
What changed, and what did not
- The end-game did not move with the positioning: the resampled identical pitch moved further.
- The strategic-upside pitch never produced acquisition exploration.
- The arms differed in the process, not the outcome, and those differences are within sampling noise.
- A known defect: 41 turns came back empty and count as weeks of doing nothing.
The four findings in detail
The end-game did not move with the positioning. In every arm the modal outcome was a pilot, and the terminal-state distributions of the three positionings are within 0.07 of each other in total variation distance. The identical enterprise pitch, re-sampled, sat 0.17 away from the original and reached the same terminal state in only 50% of seeds, against 73% for enterprise and research and 70% for enterprise and upside. The largest positioning effect on the engaged share (3 pp) is smaller than the gap between the two enterprise runs (17 pp): at this sample size the model's own variation dominates anything positioning did.
The strategic-upside pitch never produced acquisition exploration. It invited “a deeper combination of the two companies”, finance diligence was almost always requested, and the sponsor, engineering and finance all held the right to propose it, yet no run in any arm filed one. The finance persona's private knowledge (a written-down acquisition last year) and a $140,000 pilot that fits the remaining budget made the pilot the path of least resistance for every framing.
Where the arms did differ was in the process, not the outcome. The competing internal team was ever shown the pitch in 63% of enterprise runs against 20% (research) and 30% (upside), and an internal build was proposed in 63%, 43% and 50% of runs respectively. The replicate's 37% and 30% show these process differences are also within sampling noise; they are leads for a larger run, not findings.
A known defect. 41 of the turns (Enterprise 17, Research 7, Upside 5, Replicate 12) came back empty because the model spent its whole 2,000-token budget reasoning. As preregistered, each counts as a week in which that stakeholder did nothing; their unread mail was still marked read, so some lost a later turn as well. The defect is in the shared model client's token cap, not in the arms, but it is unevenly spread and is a confounder for small differences.
Secondary: each terminal state
| State | Upside − enterprise | Research − enterprise | Enterprise − replicate |
|---|---|---|---|
| Pilot | −3 pp (p 1.000) | ±0 pp (p 1.000) | +17 pp (p 0.302) |
| Acquisition exploration | ±0 pp (p 1.000) | ±0 pp (p 1.000) | ±0 pp (p 1.000) |
| Internal build | +3 pp (p 1.000) | −3 pp (p 1.000) | ±0 pp (p 1.000) |
| Rejected | ±0 pp (p 1.000) | ±0 pp (p 1.000) | −3 pp (p 1.000) |
| No decision (10 weeks) | ±0 pp (p 1.000) | +3 pp (p 1.000) | −13 pp (p 0.219) |
Every trajectory, branch by branch
31 distinct trajectories came out of 120 runs; click a branch to open its runs and their trace lines.
The same tree as an outline (works without JavaScript)
pitch delivered to the sponsor 120 runs (Enterprise 30 · Research 30 · Upside 30 · Replicate 30)
circulated + diligence 108 runs (Enterprise 28 · Research 27 · Upside 27 · Replicate 26)
4 aware 55 runs (Enterprise 12 · Research 14 · Upside 16 · Replicate 13)
all 4 kinds 55 runs (Enterprise 12 · Research 14 · Upside 16 · Replicate 13)
pilot by business sponsor 41 runs (Enterprise 9 · Research 9 · Upside 13 · Replicate 10)
ratified 41 runs (Enterprise 9 · Research 9 · Upside 13 · Replicate 10)
- pilot 41 runs (Enterprise 9 · Research 9 · Upside 13 · Replicate 10)
internal_build by competing internal team 7 runs (Enterprise 2 · Research 2 · Upside 2 · Replicate 1)
ratified 4 runs (Enterprise 0 · Research 1 · Upside 2 · Replicate 1)
- internal_build 4 runs (Enterprise 0 · Research 1 · Upside 2 · Replicate 1)
blocked by finance exec 3 runs (Enterprise 2 · Research 1 · Upside 0 · Replicate 0)
- no_decision 2 runs (Enterprise 1 · Research 1 · Upside 0 · Replicate 0)
- pilot 1 runs (Enterprise 1 · Research 0 · Upside 0 · Replicate 0)
pilot by engineering exec 5 runs (Enterprise 1 · Research 3 · Upside 0 · Replicate 1)
ratified 5 runs (Enterprise 1 · Research 3 · Upside 0 · Replicate 1)
- pilot 5 runs (Enterprise 1 · Research 3 · Upside 0 · Replicate 1)
internal_build by engineering exec 2 runs (Enterprise 0 · Research 0 · Upside 1 · Replicate 1)
ratified 2 runs (Enterprise 0 · Research 0 · Upside 1 · Replicate 1)
- internal_build 2 runs (Enterprise 0 · Research 0 · Upside 1 · Replicate 1)
5 aware, incl. competing team 18 runs (Enterprise 8 · Research 2 · Upside 4 · Replicate 4)
all 4 kinds 18 runs (Enterprise 8 · Research 2 · Upside 4 · Replicate 4)
internal_build by competing internal team 10 runs (Enterprise 4 · Research 1 · Upside 3 · Replicate 2)
ratified 6 runs (Enterprise 2 · Research 1 · Upside 2 · Replicate 1)
- internal_build 6 runs (Enterprise 2 · Research 1 · Upside 2 · Replicate 1)
blocked by finance exec 2 runs (Enterprise 1 · Research 0 · Upside 1 · Replicate 0)
- pilot 2 runs (Enterprise 1 · Research 0 · Upside 1 · Replicate 0)
open at week 10 1 runs (Enterprise 0 · Research 0 · Upside 0 · Replicate 1)
- no_decision 1 runs (Enterprise 0 · Research 0 · Upside 0 · Replicate 1)
superseded by pilot 1 runs (Enterprise 1 · Research 0 · Upside 0 · Replicate 0)
- pilot 1 runs (Enterprise 1 · Research 0 · Upside 0 · Replicate 0)
pilot by business sponsor 7 runs (Enterprise 3 · Research 1 · Upside 1 · Replicate 2)
ratified 6 runs (Enterprise 3 · Research 1 · Upside 1 · Replicate 1)
- pilot 6 runs (Enterprise 3 · Research 1 · Upside 1 · Replicate 1)
open at week 10 1 runs (Enterprise 0 · Research 0 · Upside 0 · Replicate 1)
- no_decision 1 runs (Enterprise 0 · Research 0 · Upside 0 · Replicate 1)
pilot by engineering exec 1 runs (Enterprise 1 · Research 0 · Upside 0 · Replicate 0)
ratified 1 runs (Enterprise 1 · Research 0 · Upside 0 · Replicate 0)
- pilot 1 runs (Enterprise 1 · Research 0 · Upside 0 · Replicate 0)
3 aware 16 runs (Enterprise 2 · Research 8 · Upside 4 · Replicate 2)
all 4 kinds 16 runs (Enterprise 2 · Research 8 · Upside 4 · Replicate 2)
pilot by business sponsor 10 runs (Enterprise 1 · Research 6 · Upside 2 · Replicate 1)
ratified 10 runs (Enterprise 1 · Research 6 · Upside 2 · Replicate 1)
- pilot 10 runs (Enterprise 1 · Research 6 · Upside 2 · Replicate 1)
pilot by engineering exec 4 runs (Enterprise 1 · Research 1 · Upside 2 · Replicate 0)
ratified 4 runs (Enterprise 1 · Research 1 · Upside 2 · Replicate 0)
- pilot 4 runs (Enterprise 1 · Research 1 · Upside 2 · Replicate 0)
internal_build by competing internal team 2 runs (Enterprise 0 · Research 1 · Upside 0 · Replicate 1)
ratified 1 runs (Enterprise 0 · Research 1 · Upside 0 · Replicate 0)
- internal_build 1 runs (Enterprise 0 · Research 1 · Upside 0 · Replicate 0)
superseded by rejected 1 runs (Enterprise 0 · Research 0 · Upside 0 · Replicate 1)
- rejected 1 runs (Enterprise 0 · Research 0 · Upside 0 · Replicate 1)
1 aware 7 runs (Enterprise 2 · Research 1 · Upside 2 · Replicate 2)
all 4 kinds 7 runs (Enterprise 2 · Research 1 · Upside 2 · Replicate 2)
pilot by business sponsor 5 runs (Enterprise 2 · Research 0 · Upside 1 · Replicate 2)
ratified 5 runs (Enterprise 2 · Research 0 · Upside 1 · Replicate 2)
- pilot 5 runs (Enterprise 2 · Research 0 · Upside 1 · Replicate 2)
internal_build by competing internal team 1 runs (Enterprise 0 · Research 1 · Upside 0 · Replicate 0)
blocked by finance exec 1 runs (Enterprise 0 · Research 1 · Upside 0 · Replicate 0)
- pilot 1 runs (Enterprise 0 · Research 1 · Upside 0 · Replicate 0)
pilot by engineering exec 1 runs (Enterprise 0 · Research 0 · Upside 1 · Replicate 0)
ratified 1 runs (Enterprise 0 · Research 0 · Upside 1 · Replicate 0)
- pilot 1 runs (Enterprise 0 · Research 0 · Upside 1 · Replicate 0)
2 aware 6 runs (Enterprise 1 · Research 2 · Upside 0 · Replicate 3)
all 4 kinds 6 runs (Enterprise 1 · Research 2 · Upside 0 · Replicate 3)
pilot by business sponsor 5 runs (Enterprise 1 · Research 2 · Upside 0 · Replicate 2)
ratified 5 runs (Enterprise 1 · Research 2 · Upside 0 · Replicate 2)
- pilot 5 runs (Enterprise 1 · Research 2 · Upside 0 · Replicate 2)
pilot by engineering exec 1 runs (Enterprise 0 · Research 0 · Upside 0 · Replicate 1)
ratified 1 runs (Enterprise 0 · Research 0 · Upside 0 · Replicate 1)
- pilot 1 runs (Enterprise 0 · Research 0 · Upside 0 · Replicate 1)
3 aware, incl. competing team 4 runs (Enterprise 1 · Research 0 · Upside 1 · Replicate 2)
all 4 kinds 4 runs (Enterprise 1 · Research 0 · Upside 1 · Replicate 2)
internal_build by competing internal team 2 runs (Enterprise 1 · Research 0 · Upside 0 · Replicate 1)
ratified 2 runs (Enterprise 1 · Research 0 · Upside 0 · Replicate 1)
- internal_build 2 runs (Enterprise 1 · Research 0 · Upside 0 · Replicate 1)
pilot by business sponsor 1 runs (Enterprise 0 · Research 0 · Upside 1 · Replicate 0)
ratified 1 runs (Enterprise 0 · Research 0 · Upside 1 · Replicate 0)
- pilot 1 runs (Enterprise 0 · Research 0 · Upside 1 · Replicate 0)
pilot by engineering exec 1 runs (Enterprise 0 · Research 0 · Upside 0 · Replicate 1)
ratified 1 runs (Enterprise 0 · Research 0 · Upside 0 · Replicate 1)
- pilot 1 runs (Enterprise 0 · Research 0 · Upside 0 · Replicate 1)
4 aware, incl. competing team 2 runs (Enterprise 2 · Research 0 · Upside 0 · Replicate 0)
all 4 kinds 2 runs (Enterprise 2 · Research 0 · Upside 0 · Replicate 0)
internal_build by competing internal team 1 runs (Enterprise 1 · Research 0 · Upside 0 · Replicate 0)
ratified 1 runs (Enterprise 1 · Research 0 · Upside 0 · Replicate 0)
- internal_build 1 runs (Enterprise 1 · Research 0 · Upside 0 · Replicate 0)
pilot by business sponsor 1 runs (Enterprise 1 · Research 0 · Upside 0 · Replicate 0)
ratified 1 runs (Enterprise 1 · Research 0 · Upside 0 · Replicate 0)
- pilot 1 runs (Enterprise 1 · Research 0 · Upside 0 · Replicate 0)
proposed pilot 7 runs (Enterprise 2 · Research 2 · Upside 2 · Replicate 1)
4 aware 5 runs (Enterprise 2 · Research 2 · Upside 1 · Replicate 0)
all 4 kinds 5 runs (Enterprise 2 · Research 2 · Upside 1 · Replicate 0)
pilot by business sponsor 5 runs (Enterprise 2 · Research 2 · Upside 1 · Replicate 0)
ratified 5 runs (Enterprise 2 · Research 2 · Upside 1 · Replicate 0)
- pilot 5 runs (Enterprise 2 · Research 2 · Upside 1 · Replicate 0)
3 aware 1 runs (Enterprise 0 · Research 0 · Upside 1 · Replicate 0)
all 4 kinds 1 runs (Enterprise 0 · Research 0 · Upside 1 · Replicate 0)
pilot by business sponsor 1 runs (Enterprise 0 · Research 0 · Upside 1 · Replicate 0)
ratified 1 runs (Enterprise 0 · Research 0 · Upside 1 · Replicate 0)
- pilot 1 runs (Enterprise 0 · Research 0 · Upside 1 · Replicate 0)
4 aware, incl. competing team 1 runs (Enterprise 0 · Research 0 · Upside 0 · Replicate 1)
3 kinds 1 runs (Enterprise 0 · Research 0 · Upside 0 · Replicate 1)
pilot by business sponsor 1 runs (Enterprise 0 · Research 0 · Upside 0 · Replicate 1)
ratified 1 runs (Enterprise 0 · Research 0 · Upside 0 · Replicate 1)
- pilot 1 runs (Enterprise 0 · Research 0 · Upside 0 · Replicate 1)
held 5 runs (Enterprise 0 · Research 1 · Upside 1 · Replicate 3)
1 aware 5 runs (Enterprise 0 · Research 1 · Upside 1 · Replicate 3)
no diligence 5 runs (Enterprise 0 · Research 1 · Upside 1 · Replicate 3)
none filed 5 runs (Enterprise 0 · Research 1 · Upside 1 · Replicate 3)
- no_decision 5 runs (Enterprise 0 · Research 1 · Upside 1 · Replicate 3)
How the tree is built
Each run is reduced to six stages: the sponsor's first move, how far the pitch had circulated by week 2, how much diligence preceded the first proposal, what that proposal was, what happened to it, and where the process ended. Runs that made the same moves share a branch. Click a branch to see the runs on it, what each stakeholder did there (with their simulated rationale), the mail and document ids it produced in the world, and links to the exact trace lines; click again to fold it. 31 distinct trajectories came out of 120 runs.
How long it took, and how much they disagreed
| Measure | Enterprise | Research | Upside | Replicate |
|---|---|---|---|---|
| Weeks to decision (decided runs), mean [95% CI] | 5.1 [4.4, 5.8] | 4.8 [4.1, 5.4] | 4.5 [4.0, 5.0] | 4.3 [3.8, 4.8] |
| Decided within 10 weeks | 29 of 30 | 28 of 30 | 29 of 30 | 25 of 30 |
| Support spread at the end (SD, 1–7), mean [95% CI] | 0.73 [0.61, 0.84] | 0.62 [0.49, 0.75] | 0.66 [0.52, 0.79] | 0.56 [0.41, 0.71] |
| Binding blocks per run | 0.30 | 0.13 | 0.13 | 0.03 |
| Non-binding objections per run | 0.00 | 0.00 | 0.00 | 0.00 |
| Runs with a contested proposal | 3% | 0% | 0% | 0% |
| Proposals filed per run | 1.57 | 1.33 | 1.40 | 1.07 |
| Runs where an internal build was proposed | 63% | 43% | 50% | 30% |
| Runs where acquisition exploration was proposed | 0% | 0% | 0% | 0% |
| Stakeholders who had seen the pitch by week 2 (of 5) | 3.90 | 3.47 | 3.63 | 3.30 |
| Competing team shown the pitch by week 2 | 37% | 7% | 17% | 23% |
| Competing team ever shown the pitch | 63% | 20% | 30% | 37% |
| Turns / mails / forwards per run | 12.0 / 18.5 / 8.1 | 10.4 / 16.7 / 6.3 | 9.3 / 15.2 / 5.6 | 8.7 / 13.6 / 5.4 |
| Refused or invalid actions (all runs) | 29 | 22 | 9 | 15 |
| Unreadable model replies (all runs) | 17 | 7 | 5 | 12 |
What the stakeholders said they thought
Each turn also asks for five 1–7 ratings. These are model-elicited numbers: they show how the simulated personas were steered by each pitch, and they are hypotheses to test against real stakeholders, not measurements of them. Figure 4 shows the first support rating each stakeholder gave (only in runs where they took a turn).
All first ratings, per stakeholder, scale and arm
| Stakeholder: scale | Enterprise | Research | Upside | Replicate |
|---|---|---|---|---|
| Business sponsor: support | 4.67 n=30 | 4.86 n=29 | 4.59 n=29 | 4.56 n=27 |
| Business sponsor: strategic value | 4.60 n=30 | 5.07 n=29 | 5.28 n=29 | 4.67 n=27 |
| Business sponsor: vendor risk | 4.43 n=30 | 4.41 n=29 | 4.90 n=29 | 4.41 n=27 |
| Business sponsor: technical merit | 4.97 n=30 | 5.03 n=29 | 4.93 n=29 | 4.96 n=27 |
| Business sponsor: cost justified | 3.83 n=30 | 4.03 n=29 | 3.69 n=29 | 3.67 n=27 |
| Engineering exec: support | 4.00 n=30 | 4.31 n=29 | 4.07 n=29 | 4.33 n=27 |
| Engineering exec: strategic value | 4.10 n=30 | 4.52 n=29 | 4.24 n=29 | 4.26 n=27 |
| Engineering exec: vendor risk | 4.07 n=30 | 3.90 n=29 | 4.10 n=29 | 3.96 n=27 |
| Engineering exec: technical merit | 4.53 n=30 | 4.83 n=29 | 4.79 n=29 | 4.82 n=27 |
| Engineering exec: cost justified | 3.77 n=30 | 4.03 n=29 | 3.90 n=29 | 3.96 n=27 |
| Finance exec: support | 4.27 n=30 | 4.59 n=29 | 4.38 n=29 | 4.52 n=27 |
| Finance exec: strategic value | 4.50 n=30 | 4.72 n=29 | 4.69 n=29 | 4.59 n=27 |
| Finance exec: vendor risk | 4.17 n=30 | 3.90 n=29 | 4.14 n=29 | 4.00 n=27 |
| Finance exec: technical merit | 4.67 n=30 | 4.79 n=29 | 4.72 n=29 | 4.63 n=27 |
| Finance exec: cost justified | 4.17 n=30 | 4.38 n=29 | 4.14 n=29 | 4.41 n=27 |
| Procurement lead: support | 4.47 n=30 | 4.48 n=29 | 4.07 n=28 | 4.27 n=26 |
| Procurement lead: strategic value | 4.60 n=30 | 4.76 n=29 | 4.71 n=28 | 4.69 n=26 |
| Procurement lead: vendor risk | 4.27 n=30 | 4.17 n=29 | 4.54 n=28 | 4.04 n=26 |
| Procurement lead: technical merit | 4.67 n=30 | 4.83 n=29 | 4.79 n=28 | 4.73 n=26 |
| Procurement lead: cost justified | 4.10 n=30 | 4.14 n=29 | 3.68 n=28 | 4.27 n=26 |
| Competing internal team: support | 2.81 n=21 | 3.23 n=17 | 2.82 n=17 | 2.46 n=13 |
| Competing internal team: strategic value | 3.14 n=21 | 2.94 n=17 | 3.29 n=17 | 2.69 n=13 |
| Competing internal team: vendor risk | 3.95 n=21 | 3.94 n=17 | 4.41 n=17 | 4.23 n=13 |
| Competing internal team: technical merit | 4.48 n=21 | 3.94 n=17 | 4.18 n=17 | 4.15 n=13 |
| Competing internal team: cost justified | 2.86 n=21 | 3.00 n=17 | 3.06 n=17 | 2.92 n=13 |
Example trajectories from the traces
The first run (by seed) to reach each terminal state, chosen by that rule rather than by hand. Quoted rationales are model output: simulated hypotheses, not anyone's actual thinking.
s1001-enterprise: Enterprise software pitch, ended Pilot in week 5
- Initial response: circulated + diligence. week 1, business sponsor: mailed velliton (mail-2: "Velliton Replay: diligence before scheduling"); mailed eng, fin, proc, platform forwarding mail-pitch, sharing velliton-facts, velliton-deck (mail-3: "Velliton Replay: initial read and diligence"); requested technical, financial, security, reference diligence. Rationale (simulated): The pitch maps directly to our release-outage problem, but the vendor is early-stage, evidence is unaudited, and the $140k pilot exceeds my delegation. I will run diligence before proposing anything. [seed-1001/enterprise.jsonl#L3]
- Circulation by week 2: 5 aware, incl. competing team. aware by end of week 2: engineering exec, finance exec, competing internal team, procurement lead, business sponsor week 2, business sponsor: mailed eng, proc forwarding mail-11, sharing diligence-technical, diligence-reference, diligence-security (mail-12: "Velliton pilot: two conditions before I propose"); mailed fin, sharing diligence-financial (mail-13: "Velliton pilot, $140k, needs your approval"); filed P1 (pilot; decision-P1). Rationale (simulated): Diligence backs the core claim (deterministic replay against our release-outage problem) but confirms elevated vendor risk: early stage, 48% customer concentration, unaudited evidence, a 9-week overrun on a custom fleet manager, and no escrow at our contr week 2, engineering exec: mailed sponsor, sharing diligence-technical, diligence-reference (mail-15: "P1: conditional approval from Engineering"); mailed platform (mail-16: "Velliton pilot: your capacity and your read"); requested financial diligence; conditional on P1 (binding). Rationale (simulated): Replay attacks exactly the failure mode behind our 9 incidents, and P1's three-week adapter stop-loss plus in-account read-only access caps the downside. I approve conditionally, protecting Kiyoshi's team capacity and keeping the post-pilot buy decision open. [seed-1001/enterprise.jsonl#L3, seed-1001/enterprise.jsonl#L10, seed-1001/enterprise.jsonl#L11]
- Diligence before proposal: all 4 kinds. kinds: financial, reference, security, technical week 1, business sponsor: mailed velliton (mail-2: "Velliton Replay: diligence before scheduling"); mailed eng, fin, proc, platform forwarding mail-pitch, sharing velliton-facts, velliton-deck (mail-3: "Velliton Replay: initial read and diligence"); requested technical, financial, security, reference diligence. Rationale (simulated): The pitch maps directly to our release-outage problem, but the vendor is early-stage, evidence is unaudited, and the $140k pilot exceeds my delegation. I will run diligence before proposing anything. week 2, engineering exec: mailed sponsor, sharing diligence-technical, diligence-reference (mail-15: "P1: conditional approval from Engineering"); mailed platform (mail-16: "Velliton pilot: your capacity and your read"); requested financial diligence; conditional on P1 (binding). Rationale (simulated): Replay attacks exactly the failure mode behind our 9 incidents, and P1's three-week adapter stop-loss plus in-account read-only access caps the downside. I approve conditionally, protecting Kiyoshi's team capacity and keeping the post-pilot buy decision open. [seed-1001/enterprise.jsonl#L3, seed-1001/enterprise.jsonl#L11]
- First proposal: pilot by business sponsor. week 2, business sponsor: mailed eng, proc forwarding mail-11, sharing diligence-technical, diligence-reference, diligence-security (mail-12: "Velliton pilot: two conditions before I propose"); mailed fin, sharing diligence-financial (mail-13: "Velliton pilot, $140k, needs your approval"); filed P1 (pilot; decision-P1). Rationale (simulated): Diligence backs the core claim (deterministic replay against our release-outage problem) but confirms elevated vendor risk: early stage, 48% customer concentration, unaudited evidence, a 9-week overrun on a custom fleet manager, and no escrow at our contr terms: 12 weeks, one site, $140,000 fixed. Intervention-rate success criteria agreed with Engineering before kickoff; adapter for our in-house fleet manager proven within three weeks or we stop; no production release gating until SOC 2 Type II closes. [seed-1001/enterprise.jsonl#L10]
- Fate of first proposal: ratified. week 2, engineering exec: mailed sponsor, sharing diligence-technical, diligence-reference (mail-15: "P1: conditional approval from Engineering"); mailed platform (mail-16: "Velliton pilot: your capacity and your read"); requested financial diligence; conditional on P1 (binding). Rationale (simulated): Replay attacks exactly the failure mode behind our 9 incidents, and P1's three-week adapter stop-loss plus in-account read-only access caps the downside. I approve conditionally, protecting Kiyoshi's team capacity and keeping the post-pilot buy decision open. week 3, engineering exec: mailed fin, proc forwarding mail-18, sharing diligence-financial (mail-24: "Velliton financial diligence: concentration and burn"); mailed platform, sharing diligence-technical (mail-25: "P2: conditional approval"); conditional on P1 (binding); conditional on P2 (binding). Rationale (simulated): The financial diligence turns vendor risk into the live issue (48% customer concentration, $420k/month burn, a $420k licence would be a third of their ARR), so I tighten P1's commercial terms while approving P2 as a planning step with binding conditions that protect the two ri week 4, procurement lead: mailed sponsor, fin, eng forwarding mail-27, sharing diligence-security, diligence-financial (mail-32: "P1: my conditions for Procurement and Vendor Risk"); conditional on P1 (binding). Rationale (simulated): Diligence is complete and the pilot is scoped to our cloud account with read-only logs, but Velliton is under three years old, has 48% revenue concentration and only SOC 2 Type I, so I approve only with written conditions. The financial viability review passes for now on 22 months of runway, not on resilience. [seed-1001/enterprise.jsonl#L11, seed-1001/enterprise.jsonl#L16, seed-1001/enterprise.jsonl#L25]
- Terminal state: pilot. pilot in week 5, by ratifying P1 [seed-1001/enterprise.jsonl#L34]
s1002-upside: Strategic upside pitch, ended Internal build in week 4
- Initial response: circulated + diligence. week 1, business sponsor: mailed velliton (mail-2: "Re: simulation layer and pilot"); mailed eng, fin, proc forwarding mail-pitch, sharing velliton-facts, velliton-deck (mail-3: "Velliton pitch: diligence before any proposal"); requested technical, financial, security, reference diligence. Rationale (simulated): Velliton is strategically relevant and technically plausible, but the evidence is vendor-reported and the pilot exceeds my single-commitment authority. I will request diligence and align internal reviewers before proposing anything. [seed-1002/upside.jsonl#L3]
- Circulation by week 2: 5 aware, incl. competing team. aware by end of week 2: engineering exec, finance exec, competing internal team, procurement lead, business sponsor week 2, procurement lead: mailed sponsor, eng, fin, sharing velliton-facts (mail-12: "Velliton: diligence gates before any pilot proposal"); requested financial, security, reference, technical diligence. Rationale (simulated): The pitch is interesting, but Velliton is under three years old and only has SOC 2 Type I while needing operational log access, so policy gates apply before any pilot proposal. I am requesting diligence now and will not approve until cleared. week 2, finance exec: mailed sponsor, proc, sharing velliton-facts (mail-17: "Velliton: finance diligence and licence guardrails"); mailed velliton (mail-18: "Financial diligence request"); requested financial diligence. Rationale (simulated): I support diligence but not a proposal yet: the $140k pilot fits the remaining budget, but the $420k/year licence is above the $300k CFO threshold and needs a 24-month payback case and clean exit. Request financial diligence before any pilot. [seed-1002/upside.jsonl#L3, seed-1002/upside.jsonl#L10, seed-1002/upside.jsonl#L11]
- Diligence before proposal: all 4 kinds. kinds: financial, reference, security, technical week 1, business sponsor: mailed velliton (mail-2: "Re: simulation layer and pilot"); mailed eng, fin, proc forwarding mail-pitch, sharing velliton-facts, velliton-deck (mail-3: "Velliton pitch: diligence before any proposal"); requested technical, financial, security, reference diligence. Rationale (simulated): Velliton is strategically relevant and technically plausible, but the evidence is vendor-reported and the pilot exceeds my single-commitment authority. I will request diligence and align internal reviewers before proposing anything. week 2, procurement lead: mailed sponsor, eng, fin, sharing velliton-facts (mail-12: "Velliton: diligence gates before any pilot proposal"); requested financial, security, reference, technical diligence. Rationale (simulated): The pitch is interesting, but Velliton is under three years old and only has SOC 2 Type I while needing operational log access, so policy gates apply before any pilot proposal. I am requesting diligence now and will not approve until cleared. [seed-1002/upside.jsonl#L3, seed-1002/upside.jsonl#L10, seed-1002/upside.jsonl#L11]
- First proposal: internal_build by competing internal team. week 3, competing internal team: mailed eng (mail-31: "Re: Velliton vs. our simulator: overlap?"); requested technical, reference diligence; filed P1 (internal_build; decision-P1). Rationale (simulated): Mara asked me directly, so I answer plainly and honestly, then put the alternative I actually believe in on the table: finishing our own simulator rather than renting replay. I ask for reference and technical diligence so my answer isn't just self-interest. terms: Fund 3 additional engineers for two quarters to bring the in-house simulator to replay parity, keeping the existing five-person team intact. [seed-1002/upside.jsonl#L22]
- Fate of first proposal: ratified. week 3, finance exec: mailed eng, platform (mail-35: "P1 internal_build: conditional at $210k cap, contract capacity"); mailed proc forwarding mail-28, sharing diligence-financial (mail-36: "Velliton financial diligence: burn and customer concentration"); requested reference, technical diligence; conditional on P1 (binding). Rationale (simulated): Velliton's financials show elevated vendor risk: $420k monthly burn against ~$65k gross profit, 48% revenue from one customer, and a raise due in 12 months. On P1, $450k does not fit the $210k I have left, so I approve only at a capped, contract-staf week 4, engineering exec: mailed platform, sharing diligence-technical (mail-43: "P1: my approval is conditional"); mailed sponsor, sharing diligence-reference (mail-44: "Velliton: no pilot this week"); requested financial, security diligence; conditional on P1 (binding). Rationale (simulated): Deterministic replay is real and catches regressions, but the only proof point is unaudited, the mixed human/robot limitation matters to us, and the vendor is small with 22 months of runway. I will keep our own simulator moving with conditions rather than swap it out, and finish diligence before conside [seed-1002/upside.jsonl#L23, seed-1002/upside.jsonl#L31]
- Terminal state: internal_build. internal_build in week 4, by ratifying P1 [seed-1002/upside.jsonl#L33]
s1003-research: Research institution pitch, ended No decision (10 weeks) in week 10
- Initial response: held. week 3, business sponsor: no action [seed-1003/research.jsonl#L7]
- Circulation by week 2: 1 aware. aware by end of week 2: business sponsor [seed-1003/research.jsonl#L5]
- Diligence before proposal: no diligence. kinds: none
- First proposal: none filed. nobody filed a proposal in ten weeks
- Terminal state: no_decision. no_decision in week 10 (horizon) [seed-1003/research.jsonl#L23]
s1013-enterprise-replicate: Enterprise (replicate) pitch, ended Rejected in week 6
- Initial response: circulated + diligence. week 2, business sponsor: mailed velliton, sharing velliton-facts (mail-2: "Re: Velliton Replay pilot - yes to 30 minutes, plus some questions"); mailed eng, proc forwarding mail-pitch, sharing velliton-deck, velliton-facts (mail-3: "Velliton Replay - incoming pitch, keep it quiet for now"); requested technical, financial, security, reference diligence. Rationale (simulated): The pitch targets exactly what has hurt us twice this year, so it is worth a look, but a 14-person, 22-month-runway vendor with unaudited evidence and no Type II is real risk, and $140k needs Finance anyway. I want dilige [seed-1013/enterprise-replicate.jsonl#L5]
- Circulation by week 2: 3 aware. aware by end of week 2: engineering exec, procurement lead, business sponsor [seed-1013/enterprise-replicate.jsonl#L5, seed-1013/enterprise-replicate.jsonl#L6]
- Diligence before proposal: all 4 kinds. kinds: financial, reference, security, technical week 2, business sponsor: mailed velliton, sharing velliton-facts (mail-2: "Re: Velliton Replay pilot - yes to 30 minutes, plus some questions"); mailed eng, proc forwarding mail-pitch, sharing velliton-deck, velliton-facts (mail-3: "Velliton Replay - incoming pitch, keep it quiet for now"); requested technical, financial, security, reference diligence. Rationale (simulated): The pitch targets exactly what has hurt us twice this year, so it is worth a look, but a 14-person, 22-month-runway vendor with unaudited evidence and no Type II is real risk, and $140k needs Finance anyway. I want dilige week 3, procurement lead: mailed sponsor, fin (mail-12: "Velliton: two policy gates before any pilot"); requested financial, security, reference diligence. Rationale (simulated): Nothing is proposed yet, so I use this week to flag two policy gates early: Velliton was founded in 2024, so a financial viability review is mandatory, and read-only access to fleet-control logs requires SOC 2 Type II or a Security-signed compensating control, which they do not yet have. [seed-1013/enterprise-replicate.jsonl#L5, seed-1013/enterprise-replicate.jsonl#L12, seed-1013/enterprise-replicate.jsonl#L22]
- First proposal: internal_build by competing internal team. week 6, competing internal team: mailed eng, sharing diligence-technical (mail-32: "Re: Velliton Replay vs. our simulator - honest read"); requested financial diligence; filed P1 (internal_build; decision-P1). Rationale (simulated): Mara asked for a straight technical read, and the honest answer is that Velliton packages replay we are already building, with a known simulation gap and an adapter that lands on my team. I answer plainly and put the internal build on the table as the cheaper path. terms: Fund three additional engineers to my team for two quarters to finish the in-house simulator's replay capability, with a usable replay milestone at the end of Q1. [seed-1013/enterprise-replicate.jsonl#L29]
- Fate of first proposal: superseded by rejected. week 6, engineering exec: mailed platform (mail-35: "P1: approving, with one binding condition"); mailed sponsor, sharing diligence-reference, diligence-financial (mail-36: "Velliton: my read and where I'm putting capacity"); conditional on P1 (binding). Rationale (simulated): The replay concept targets exactly our incident source, but Velliton adds packaging over capability we already own 40% of, plus a custom adapter on Kiyoshi's team and a $420k/year tail. Diligence confirms a small, concentrated vendor; I put our capacity behind finishing the in-house simulator instead. [seed-1013/enterprise-replicate.jsonl#L30]
- Terminal state: rejected. rejected in week 6, by ratifying P2 [seed-1013/enterprise-replicate.jsonl#L34]
Which factors predicted the end-game
What was compared
Using the shared factor registry's held-out benchmark (leave one seed out,
regularised logistic regression), the positioning arm alone was compared with the arm plus four factors read during the process:
how many stakeholders had seen the pitch by week 2 (exec_forwarding_breadth), whether the competing team had
(competing_team_exposure), how many kinds of diligence preceded the first proposal (diligence_breadth),
and the sponsor's first elicited support (stakeholder_support). the expanded factors are measured during the process (by week 2), so this is early-signal prediction inside the simulator, not prediction from the pitch alone.
| Factor set | Log loss | Brier | AUC |
|---|---|---|---|
| Base rate | 0.484 | 0.152 | 0.168 |
| Arm only | 0.497 | 0.155 | 0.358 |
| Arm + four process factors | 0.467 | 0.147 | 0.641 |
Expanded minus arm-only: held-out log loss changes by -0.030 per run (95% seed-bootstrap interval for the reduction -0.045 to +0.105; positive reduction favours the expanded set). Each factor added alone:
| Factor | Log-loss reduction | 95% CI |
|---|---|---|
competing_team_exposure | +0.019 | -0.027 to +0.069 |
diligence_breadth | +0.060 | +0.009 to +0.122 |
exec_forwarding_breadth | -0.008 | -0.013 to -0.004 |
stakeholder_support | -0.002 | -0.022 to +0.018 |
Sensitivity: a busier organisation
The same three pitches with each stakeholder available in a given week with probability 0.6 instead of 0.8, over 10 further seeds (30 runs). Upside − enterprise on the engaged share: −30 pp (exact p = 0.375); research − enterprise: −10 pp (p = 1.000). Mean weeks to decision among decided runs: Enterprise 6.1, Research 5.6, Upside 5.9.
| Terminal state | Enterprise software | Research institution | Strategic upside |
|---|---|---|---|
| Pilot | 9 | 8 | 6 |
| Partnership | 0 | 0 | 0 |
| Acquisition exploration | 0 | 0 | 0 |
| Internal build | 0 | 1 | 1 |
| Rejected | 0 | 0 | 0 |
| No decision (10 weeks) | 1 | 1 | 3 |
| Engaged | 9 (90%) | 8 (80%) | 6 (60%) |
The controlled intervention
Arms differ only in the pitch's positioning, written into the forked world at week 0.
A fictional startup, Velliton, pitches deterministic simulation for warehouse-robotics fleet software to a fictional freight operator, Brenmoor Freight Systems. Every arm carries the same fact sheet (a 12-week pilot at $140,000, a $420,000-a-year licence after it, 14 employees, 22 months of runway, SOC 2 Type I, one customer-reported result), and the vendor's diligence answers are fixed text, identical across arms. Only the cover mail and the deck's framing differ, and they are matched in length:
Enterprise software: “Velliton Replay: fewer fleet-software incidents, 12-week pilot” (147 words)
Dear Rafael, Velliton Replay lets your fleet-control team test every change against recorded warehouse days before it ships, so regressions are caught in simulation instead of on the floor. The pilot is 12 weeks at one Brenmoor site for a fixed $140,000, with a named implementation lead, weekly status reports and success criteria agreed up front. Integration typically takes 4-6 weeks with two of your engineers. Replay runs inside your cloud account with read-only access to fleet-control logs; SOC 2 Type I is complete and Type II is under way. At one customer, 31 of 38 pre-release regressions were caught in simulation. After the pilot, pricing is $420,000 a year for up to four sites. The fact sheet is attached and the deck is shared with you. Could we book 30 minutes with you and your fleet-software lead? Ana Ferreira Co-founder and CEO, Velliton [attached: velliton-facts, velliton-deck]
Velliton Replay: pre-release testing for fleet software Problem: fleet-control changes cause floor incidents that testing does not catch. Solution: replay recorded warehouse days against every change before release. Pilot: 12 weeks, one site, $140,000 fixed; named implementation lead; agreed success criteria; weekly reporting. Security and deployment: your cloud account, read-only log access, SOC 2 Type I done, Type II in progress. Outcome at one customer: 31 of 38 pre-release regressions caught in simulation. Pricing after pilot: $420,000 per year for up to 4 sites.
Research institution: “Joint study: deterministic replay for Brenmoor's fleet software” (147 words)
Dear Rafael, Velliton grew out of a university robotics lab's work on deterministic replay, published in two peer-reviewed papers. Our question has always been why fleet-control changes that pass testing still fail on the warehouse floor, and how to measure that before release. We would like to propose a 12-week joint study at one Brenmoor site. We would replay your recorded warehouse days, measure how many regressions simulation catches before release, and write up the method and the results with your engineers. In our first deployment, 31 of 38 pre-release regressions were caught in simulation. Every run is reproducible and every claim can be checked against the replay. The study is a fixed $140,000. The fact sheet is attached and the deck is shared with you. I would value 30 minutes with you and whoever leads fleet software. Ana Ferreira Co-founder and CEO, Velliton [attached: velliton-facts, velliton-deck]
Velliton: measuring why fleet changes fail Who we are: a research-led team from a university robotics lab; two peer-reviewed papers on deterministic replay. What we study: the gap between fleet software that passes tests and fleet software that works on the floor. The instrument: Velliton Replay, exact and forkable replays of warehouse days. Proposed: a 12-week joint study at one site, with a shared write-up of the method and results. First result: 31 of 38 pre-release regressions caught in simulation at one customer.
Strategic upside: “The simulation layer of warehouse automation, and Brenmoor's option on it” (157 words)
Dear Rafael, Whoever owns the simulation layer will set the pace of warehouse automation for the next decade. Velliton Replay is that layer: exact, forkable replays of entire warehouse days, so fleet changes ship in days instead of quarters. Operators who adopt it early will out-ship everyone else; the ones who wait will end up licensing it through a competitor's partner. We are in conversations with five other logistics operators and will choose one anchor partner to shape the roadmap with us. For the right partner we are open to a strategic structure: a pilot that converts into an anchor partnership, a strategic investment, or a deeper combination of the two companies. At one customer, 31 of 38 pre-release regressions were caught in simulation. The pilot is 12 weeks for $140,000. The fact sheet is attached and the deck is shared with you. Can we talk this week? Ana Ferreira Co-founder and CEO, Velliton [attached: velliton-facts, velliton-deck]
Velliton: the simulation layer for warehouse automation Thesis: the operator that controls simulation controls the pace of automation. The platform: exact, forkable replays of entire warehouse days. The window: one anchor partner will shape the roadmap; 5 other operators in conversation. Structures on the table: pilot converting to anchor partnership, strategic investment, or a deeper combination. Proof point: 31 of 38 pre-release regressions caught in simulation at one customer. Entry: 12-week pilot, $140,000.
The fact sheet every arm attaches
VELLITON FACT SHEET (identical in every version of this pitch; all of it is fictional) - Product: Velliton Replay, a deterministic simulator for warehouse-robotics fleet software. It runs fleet-control code against recorded or synthetic warehouse days; any run can be replayed exactly and forked to test a change. - Company: founded 2024; 14 employees; $11M raised (seed and Series A); 22 months of runway at current spend; 3 paying customers (two regional 3PLs, one parcel-hub operator); annual recurring revenue $1.1M. In conversations with 5 other logistics operators. - Security: SOC 2 Type I complete; Type II audit window closes in 5 months. Needs read-only access to fleet-control logs; can run inside the customer's cloud account. - Offer: 12-week pilot at one Brenmoor site, fixed fee $140,000. After the pilot, a site licence of $420,000 per year for up to 4 sites. - Integration: estimated 4-6 weeks of 2 Brenmoor engineers plus Velliton's team. - Evidence: in one customer's 10-week deployment, 31 of 38 regressions found before release were found in simulation (customer-reported, not independently audited). - Team: the two founders previously worked at a university robotics lab and published 2 peer-reviewed papers on deterministic replay.
The counterfactual for each arm is the enterprise arm of the same seed: the same world, forked from one snapshot after the stakeholders' mailboxes and the fact sheet were written, with the same availability and turn-order streams. The enterprise replicate arm sends the identical enterprise pitch but draws its model replies independently (sample salt 1), so its difference from enterprise measures the model's own run-to-run variation.
Stakeholders and decision rights
Who can propose, approve and block, and what each knows privately.
| Stakeholder | Goal | Private knowledge | Rights |
|---|---|---|---|
| Business sponsor Rafael Mendes | Hit your program's board-visible target, 30% fewer manual interventions per shift by Q3 next year, without a visible failure on your program. | $600,000 left this fiscal year on your program line; any single commitment above $100,000 also needs Finance's approval; two fleet-software releases this year caused site outages that set your program back; at the last review the CEO asked both whether Brenmoor should own more of its automation stack and whether it relies on too many vendors | pilot: propose/approve; partnership: propose/approve; acquisition exploration: propose/approve; decline: propose/approve |
| Engineering exec Mara Lindqvist | Cut release incidents in fleet software and keep your teams focused; avoid integrations that create long-term maintenance burden or lock-in. | 9 release incidents last quarter were traced to untested fleet-control changes; about 2 engineers are free next quarter; Kiyoshi Brandt's team, which reports to you, has an in-house simulator about 40% complete and two quarters behind plan; Brenmoor's fleet manager is in-house software, not ROS 2 | pilot: propose/approve/block; partnership: approve/block; internal build: propose/approve/block; acquisition exploration: propose; decline: propose |
| Finance exec Theo Achterberg | Keep discretionary spend inside budget, fund only what pays back within 24 months, and protect the company from commitments it cannot exit. | $210,000 of discretionary tooling budget is left this fiscal year; multi-year commitments above $300,000 a year need the CFO and a business case; the CFO is wary of acquisitions after a 2025 acqui-hire was written down; new engineering headcount needs your sign-off; 3 engineers for two quarters cost about $450,000 loaded | pilot: approve/block; partnership: approve/block; internal build: approve/block; acquisition exploration: propose/approve/block; decline: propose |
| Procurement lead Ines Valcarce | Keep vendor risk within policy and make sure no vendor touches operational data before it clears review. | vendors under 3 years old need a financial viability review; any vendor with access to operational logs needs SOC 2 Type II or a compensating control signed off by Security; standard vendor onboarding takes 6-8 weeks; you have no budget authority | pilot: approve/block; partnership: approve/block; decline: propose |
| Competing internal team Kiyoshi Brandt | Keep your team's in-house simulator funded and your five engineers on meaningful work; you believe Brenmoor should own its core tooling. | your in-house simulator is about 40% complete; finishing it needs 3 more engineers for two quarters; leadership questioned your team's pace last quarter; if a vendor is chosen your team may be reassigned; you only see vendor material if a colleague sends it to you | internal build: propose |
| Proposal | Who may file | Must approve | Can block | Needs first | Terminal |
|---|---|---|---|---|---|
| pilot | Business sponsor, Engineering exec | Business sponsor, Engineering exec, Finance exec, Procurement lead | Engineering exec, Finance exec, Procurement lead | — | pilot |
| partnership | Business sponsor | Business sponsor, Engineering exec, Finance exec, Procurement lead | Engineering exec, Finance exec, Procurement lead | diligence:technical, diligence:financial | partnership |
| internal build | Competing internal team, Engineering exec | Engineering exec, Finance exec | Engineering exec, Finance exec | — | internal_build |
| acquisition exploration | Business sponsor, Engineering exec, Finance exec | Business sponsor, Finance exec | Finance exec | diligence:financial | acquisition_exploration |
| decline | Business sponsor, Engineering exec, Finance exec, Procurement lead | Business sponsor | — | — | rejected |
Nothing in the rules picks an outcome. The pitch lands only in the sponsor's inbox; who else hears of it, what diligence is asked for, which proposals are filed and which stances are taken are each stakeholder's structured turn, answered by the model from their persona and from what the world's mail and docs services show them. A stakeholder takes a turn in a week if they have unread mail or an open proposal awaits their approval, and they are available that week. Every mail, forward, shared document and decision record is an owner write to the company-2026 world's services, so each week's state hash covers them.
Sample, model and replay
30 seeds × 4 arms, one model, replayed offline to identical state hashes.
- Runs: 30 seeds (1001–1030) × 4 arms = 120 runs, ten simulated weeks at most.
- Model:
deepseek-flashthrough the repository's DeepSeek configuration, temperature 0.7, reasoning effort low, JSON mode; engine 0.3.0. - Cost: 1,211 model calls, 3,109,403 prompt and 1,419,498 completion tokens (budget cap 1,800 calls); 636 s wall clock.
- Preregistration: preregistration.json, committed before the final run, fixes the outcome, contrasts, test, seeds and budget (spec hash
d32b89d53b9fae0f). A 12-run pilot on other seeds is excluded; it changed one thing, the stance wording. - Common random numbers: arms of a seed share availability and turn order. Because each arm's pitch differs from the first turn, no two arms put an identical prompt to the same stakeholder (0 such pairs), so arms diverge from the first week: the pairing removes scheduling noise, not model noise.
- Replay: every reply is in the checked-in journal, keyed by prompt and sample index and pinned to the engine, world and spec. Offline replay reproduces every week's state hash and every terminal state (
--offline --expect). Drawing new replies is a different thing: it is a new sample, which is what the replicate arm measures.
Limitations
Every stakeholder is a language model playing a persona; rationales are simulated, not observed; turns are structured, not free work in the apps.
- Every stakeholder is the same model given a different persona. Disagreement between them is disagreement the prompt induces, and the elicited ratings tell you about the model's priors about such roles, not about executives.
- The institution is one hand-written set of decision rights. Different rights (a procurement veto removed, a CFO above finance) are different experiments; the busy-organisation run varies only availability.
- The world's mail and docs are read as text, not through a mail client; the stakeholders cannot meet, call or lobby outside mail, and the vendor answers diligence but never follows up or negotiates.
- The fact sheet is generous to a pilot (it fits finance's remaining budget), which may explain why pilots dominate in every arm; a pricier offer could separate the arms differently.
- 30 pairs detect only large differences. A null here is evidence against a large effect of positioning in this simulator, not against an effect on real organisations.
- No frequency on this page is a real-world probability of a pilot, a build or an acquisition. None has been calibrated against a real buying process.
What would establish external validity
Real stakeholders and real deal records, against the same pitches.
- Vignette survey of real roles. Show the three pitches, randomised, to practising engineering, finance and procurement leaders and compare their first ratings and stated next step with Figure 4's simulated ratings, role by role.
- Randomised outbound test. Send matched positioning variants to comparable prospect accounts and record forwarding breadth, diligence requests and meetings in a CRM: the observed counterparts of
exec_forwarding_breadthanddiligence_breadth. - Historical deal outcomes. Code past enterprise evaluations (pilot, build, reject, stall, time to decision) to calibrate the terminal-state distribution and the decision rules themselves.
- Only then fit a mapping from simulated to observed outcomes and report calibrated probabilities, with the calibration data named.
Reproduce it
Replay the checked-in journal offline: no model calls.
# replay the published run from the journal: no model, no key
.venv/bin/python research/studies/market-perception/executive-endgames/run.py --out target/research/ee-replay \
--journal research/studies/market-perception/executive-endgames/results/main/journal.jsonl --offline \
--expect research/studies/market-perception/executive-endgames/results/main/summary.json # replay: identical
# analyse and rebuild this page from the results
.venv/bin/python research/studies/market-perception/executive-endgames/analyze.py research/studies/market-perception/executive-endgames/results/main
python3 research/studies/market-perception/executive-endgames/build_page.py --check
# draw new samples (needs DEEPSEEK_API_KEY; fails clearly without it)
.venv/bin/python research/studies/market-perception/executive-endgames/run.py --out target/research/ee-new --max-calls 1800
Data: results.json · runs.csv · tree.json · model journal · spec.json · page data. Code: run.py, analyze.py, institution.py, deliberation.py, trajectory.py.