Skip to content
DAY ZERO

Method

Rules that freeze before they run

Every rule set and every cohort in this project was hashed and committed before the outcome it governs existed. That ordering is what makes a failure a result instead of an embarrassment — and this page is the story of one real failure and what it changed.

A caution before any numbers: the tallies below measure whether frozen sourcing rules would have surfaced known Array portfolio companies from public evidence available before their announcement dates. Every case is a company with a known outcome, so hindsight bias is structural. None of these figures is investment performance, precision, recall, or a win rate — and the three runs below are not comparable to each other as performance, because each answers a different question.

The record

Three runs, three different questions

Phase 2 · v1 · FROZEN, THEN RUN

The system failed

0 PASS2 PARTIAL4 MISS4 UNKNOWN

Ten historical cases; the rules had never seen any of them. Zero passes. The diagnosis: v1 defined independence as different hostnames, so a builder whose entire verifiable life is on GitHub — organisation creation, sustained construction, collaborators, releases — collapsed into “one source” and could never converge.

Phase 3 · v2 · POST-HOC EXPLORATORY

The rules changed

2 PASS1 PARTIAL3 MISS4 UNKNOWN

v2 was frozen before this rerun — but the cohort is the same ten cases that motivated the redesign, so this result is labelled exploratory and can prove nothing on its own. It is shown because hiding it would misstate how the repair was developed.

Phase 4A · v2 · UNSEEN COHORT

The real test

2 PASS0 PARTIAL1 MISS6 UNKNOWN

9 cases v2’s design had never touched, selected deterministically and committed before any evidence was retrieved. Out-of-sample with respect to rule development — not to venture outcomes, since every case is still a known portfolio company.

The repair

What v2 actually changed

The failure was not that v1 was too strict. It was that v1 measured independence on the wrong axis — and the repair had to get stricter, not looser, to be defensible.

v1 · independence = hostnames

Two pieces of evidence were independent if they came from different hostnames. GitHub org creation in 2024, a shipped system in 2025, and outside collaborators in 2026 all counted as one channel — while a repo plus a tweet about the repo counted as two.

v1 ad0b7ae00630f7948e7c4444440af7c20fed61169370e46e076cd8f575a3566c

v2 · independence = modalities + events

Independence became evidence modalities (construction, formation, identity, collaboration) counted over distinct dated events — while tightening four other axes at once: a minimum modality count, a minimum event count, mandatory construction evidence, and mandatory temporal spread.

v2 435dfb8a568d8f07124125b08566cc9ced48f4d17ef76064978905968287f434

The check that keeps the repair honest: negative controls. All 6 v1-rejected control repositories — curated lists, hype repos, thin wrappers — were re-run under v2. 0 were incorrectly promoted. Loosening that let junk through would have been a worse failure than the one being fixed.

Phase 4A

The unseen test

Nine eligible portfolio companies v2 had never seen, ordered by a deterministic rule — SHA-256 of the v2 rule hash joined with the case ID, sorted ascending — frozen in a commit, and only then researched. Binding the ordering to the v2 hash means it could not be chosen after the fact without visibly changing v2 itself. Case-by-case:

CaseCompanyCutoffVerdictWhy
U01Perspective AI2025-01-29● PASSall convergence checks satisfied
U02CandorIQ2025-07-21○ UNKNOWNno verifiable pre-cutoff evidence: no GitHub organisation could be linked to the company under v2 identity rules (name-only matches may not merge)
U03Wabi2025-11-04○ UNKNOWNno verifiable pre-cutoff evidence: no GitHub organisation could be linked to the company under v2 identity rules (name-only matches may not merge)
U04MokSa.ai2024-04-21✕ MISSfailed: distinct_modalities, distinct_events, includes_construction, temporal_spread
U05Capsule2021-01-13○ UNKNOWNno verifiable pre-cutoff evidence: no GitHub organisation could be linked to the company under v2 identity rules (name-only matches may not merge)
U06ORO2022-11-02○ UNKNOWNno verifiable pre-cutoff evidence: no GitHub organisation could be linked to the company under v2 identity rules (name-only matches may not merge)
U07Tumble2022-10-04○ UNKNOWNno verifiable pre-cutoff evidence: no GitHub organisation could be linked to the company under v2 identity rules (name-only matches may not merge)
U08Blumira2021-08-17● PASSall convergence checks satisfied
U09Zingly.ai2025-07-20○ UNKNOWNno verifiable pre-cutoff evidence: no GitHub organisation could be linked to the company under v2 identity rules (name-only matches may not merge)

Nine known-outcome cases cannot produce an accuracy, precision, recall or win-rate statistic, and none is reported. The six UNKNOWNs are themselves a finding: under v2’s conservative identity rules, no GitHub organisation could be verifiably linked to those companies pre-announcement — name-only matches may not merge. That is the structural blindness of a GitHub-led system stated as data: companies that form without a public construction trail are invisible to it, and pretending otherwise would require exactly the fuzzy matching the rules forbid.

What broke next

The unseen test found a new weakness

Perspective AI · U01 · PASS — and that is the problem

Perspective AI passed the v2 convergence gate on a marketing/content repository — SCSS, no licence, no meaningful engineering surface. Every convergence check was legitimately satisfied: the evidence was independent, multi-modal, temporally spread, and included construction events. The pass is valid under the frozen rules, and the rules are wrong.

The diagnosis: v2 repaired evidence independence, but it does not require the evidence to be technically deep at the convergence stage. Depth signals exist elsewhere in the system — they are simply not wired into this gate.

The rule was not changed after seeing the result. A technical-depth eligibility requirement at the convergence gate is the leading candidate for a future v3 — and no v3 has been designed, frozen or validated, so none is claimed. The weakness stands in the record exactly as the v1 failure does.

Proof

The commit history is the argument

Freeze-before-measure only means something if the ordering is verifiable. Every step below is a real commit in the repository, in this order.

  1. 🔒 lockedPhase 1 — ontology and evidence philosophy

    Signal ontology, evidence states, privacy rules, the no-score rule. No collection yet.

    c4f958c3b90ff803de343ff61981c45b87be70c7

  2. 🔒 lockedv1 rules frozen — before any validation ran

    Eligibility, independence and convergence rules hashed into a frozen manifest.

    498dabe71a13eea12450ae24a2b8a2094632b338

  3. ✕ failurev1 holdout run against the frozen rules

    0 PASS / 2 PARTIAL / 4 MISS / 4 UNKNOWN. The failure is committed, not retried.

    425c46b031b94f14a6f682110709afd06f2021a7

  4. 🔒 lockedHeadroom reproduction protocol pre-registered

    Claims, thresholds, baselines and verdict rules hashed before any measurement.

    636f4992ddd72a9b8dcaef9107765befc3e0b750

  5. 🔒 lockedv2 rules designed from the v1 diagnosis and frozen

    Independence redefined as modalities + distinct events; stricter on four axes.

    f35afa1a5f09bb9014306038b8a2efd4de3f52cd

  6. ● resultReproduction + deep diligence executed

    Headroom: PARTIALLY REPRODUCED. Sandlock: advance to founder conversation.

    f78a41c0dd2d69e75906ab058ee24612ad17ea60

  7. 🔒 lockedUnseen validation cohort frozen — before evidence retrieval

    Nine cases selected deterministically from portfolio history v2 had never touched.

    662392ab2e9e2eeec6549e08b2819d65aa03d4d8

  8. ● resultUnseen validation + identity audit run

    2 PASS / 0 PARTIAL / 1 MISS / 6 UNKNOWN — v2's repair holds on cases it never saw.

    302e5f76aef90952f26aec95c8aa4d431db0ef1f

  9. ✕ failureWhat broke next — Perspective AI

    One of those two passes came from a marketing repository. The gate checks independence, not technical depth. Left unpatched, and published as a limitation.

    302e5f76aef90952f26aec95c8aa4d431db0ef1f

Read the sequence as pairs: a locked node is a rule set or a cohort, hashed and committed; the node under it is what happened when that frozen thing was run. The hash always comes first — that ordering is the entire epistemic claim, and it is checkable with git log rather than taken on trust.

The three hashes that matter

v1 rules (frozen before v1 holdout)ad0b7ae00630f7948e7c4444440af7c20fed61169370e46e076cd8f575a3566cv2 rules (frozen before v2 rerun and Phase 4)435dfb8a568d8f07124125b08566cc9ced48f4d17ef76064978905968287f434unseen cohort freeze commit662392ab2e9e2eeec6549e08b2819d65aa03d4d8

Why these commit IDs are not the original ones

Before first publication, the repository’s history was privacy-scrubbed to remove raw third-party profile caches that had been committed during collection. Git object IDs therefore changed, and the hashes shown here are the post-scrub equivalents.

What did not change: the commit ordering, every freeze-before-result boundary, and every rule, threshold and result the ordering governs. No finding was altered — the rewrite removed data, not conclusions, and it happened before any public repository existed. The full account, including what was found and what was deliberately left byte-identical, is in research/prepublication_privacy_audit.md.

Accounting

What was and wasn't measured

Human analyst active time

NOT_MEASURED

No human ran the review timer during Phase 4, so no number is reported. Claude Code and tool wall-clock time is machine time, and it is never relabelled as human analyst time. The timer instrument exists and is tested; it produces a figure only when a human actually starts and stops it.

What a defensible v2 means

v2 survives Phase 4A as a design: it converged in two of the three cases where identity could be resolved, promoted zero negative controls, and failed in a way that is documented, diagnosed and left unpatched — including one of its own two passes being the failure. It does not survive as a finished system — the Perspective AI gap and the six identity-unresolvable cases bound what it can currently claim. Both statements belong in the record; see also the identity audit for the assumption Phase 4 revised.