Runtime feature releases: remaining delivery plan¶
This plan orders the rest of the runtime feature-release programme. It turns the backlog that the 20 October PR audit left under "follow-up acceptance" ("heading six") into deliverables. Each deliverable gets dependencies, numbered acceptance criteria and a way to verify each criterion. It does not change any decision in the accepted plan. Requirement and test IDs (RT-xx, OD-xx) refer to that plan, mainly §2 (MVP), §5 (increments) and §15 (acceptance tests).
Boundary. This applies to every slice, including the Bramble-only local slices. Nothing here publishes, promotes, deploys or activates anything in production. Live binding, import, enrolment and runtime-policy activation in any shared environment are outside this plan's execution; it only prepares and rehearses them in isolated environments and writes their prerequisites down. Staging execution needs a separate owner decision (§8). Deployments only move forward (ADR-023): a rollback is a planned replacement release.
1. Where the programme stands¶
| Wave | PRs | Delivers | State on 23 October 2026 |
|---|---|---|---|
| 1 Foundation | #4344–#4347 | Shared catalog, evaluator, writer; legacy downgrade guard; explicit lineage import | Merged |
| 2 Global control API | #4348–#4360 | Authority observer; exact mutations and receipts; consumer contracts; stop primitive; selected API and PM consumers; browser recovery; split capabilities; stop API; preview/commit/restore; outcome lookup; host acknowledgements; chart metadata; deployment-constraint transactions | Merged |
| 3 Evidence and constraints | #4361–#4375 | Scoped compatibility reports; constraint evidence; collector publication; desired provenance; migration and statistics evidence; source revalidation; retained reports; explicit readiness; read-only capture; publisher; collector inventory; verified constraint completion; coordinator adapters; break-glass | Merged |
| 4 Browser and admission | #4376–#4378 | Lossless revisions; production policy mode Legacy|Runtime (default Legacy); browser lineage pin |
Merged |
| 5 Operator UI | #4379–#4385 | Feature definitions; Prepared release intents; admin preview/commit UI; offline content compiler; fleet UI; stop UI; restore UI | Merged (kept as one stack, decision D1) |
| 6 Remaining acceptance | This plan, §3–§7 | Compatibility checker; multi-process convergence and recovery; operational prerequisites; registry and developer experience (6R, §5A); release lifecycle; availability; targeting; help and What's New, including in-app walkthroughs (6G.4); reader migration | In progress: 6A and 6C merged (#4405–#4409, #4413); 6B merged apart from 6B.4 PR 1 (#4419), which is open; 6R merged apart from the Q2 registration (#4438) (§5A); 6D merged (PRs A–G; G, #4461, at 21c73ed99) (§6.1); 6E.1–6E.3 merged (#4450, #4457, #4460) (§7.1); 6F PR A merged (#4463) and PR B in review (#4472) (§7.2); 6G PR A merged (#4466, 7f99ec0d); 6G PR B in #4474 (§7.3); the rest planned |
Two corrections from the audit must survive every reconciliation. They are checked by content after each merge, not by patch ID:
- #4371
0663336c:scripts/publish-runtime-workload.pybuilds its opener withbuild_opener(ProxyHandler({}), NoRedirects), and classifies HTTP 408, 425 and 429 as Unknown (exit 3), together with their tests. - #4374
f7fed343:RuntimeDeploymentCoordinatorTests.cscovers configuration, scope, refusal, server actor and receipt mapping for both providers, plus the HTTP deadline, revision and body-limit tests and the injectable coordinator seam.
2. Decisions¶
- D1. Keep wave 5 as one stack. The release and content slices (#4379, #4380, #4382) are needed by 6D and 6G. Each is default-off. Splitting them into new bases would discard reviewed history without reducing total work.
- D2. Reconcile by merging main. Never rebase or replay whole stacks. A simulated predecessor merge
(21 October) is clean everywhere except one hunk in #4380
(
EndpointAuthorizationExceptions.cs, plus how-to prose). - D3. MVP first. 6A, 6B and 6C complete the accepted MVP evidence (§2 of the accepted plan: change, stop and restore one eligible global control with safe adoption, recovery and compatibility checks). 6D–6H follow as separate usable slices. Each slice ships default-off with an explicit flag decision.
- D4. Synthetic tests never stand in for operational readiness. Every evidence record names its layer: unit, integration, hermetic multi-process or rehearsal.
- Later owner decisions are numbered on the programme page and in the programme ledger, which lives in the maintainer's handover folder outside this repository; each is summarised here. They were answered 2026-10-23 unless marked open. These are programme decisions: "programme D16" is not the CD plan's D16 in runtime-feature-releases.md.
- D11: the API hosts the release jobs as its own release-management module.
- D12: the database clock sets
ConfirmedAtUtc($$NOW), and the confirming host is recorded asconfirmedBy(6D question Q2, §6.1). - D13: the registry's communication defaults decide announcements, with a per-release quiet override that needs a reason.
- D14: a notice cancelled by a stop is not resent on restore.
- Programme D15: the 6B.4 safe-value fallback. Answered A on 2026-10-23: build it, default off (6B.4 PR 4, #4449).
- Programme D16: the browser fallback poll interval. Answered A on 2026-10-23: keep the 5-minute poll and state that the 60 s claim covers tabs with a live hub connection.
- D17: the 6R wave (§5A), with YAML for every human-written source.
- D18: one Features and releases admin area, built in steps.
- D19: stay built. OpenFeature only without loss of functionality (assessed in §5A: not adopted as the caller interface). Targeting is built in-house, with no paid or closed-source third-party tools.
- D20: in-app walkthroughs as 6G.4, with our own M3 CDK component.
- D24 (Chris, 2026-10-23): cross-environment administration. Now: each environment's own admin page, linked
to the others. Later (planned option): one admin web app that calls each environment's existing API with the
person's own sign-in for that environment (a CORS allow-list for that one origin); each environment's preview,
confirm and audit still apply. Ruled out: a central service holding write credentials. Proposed, owner-gated:
a per-environment flag-state metrics export (the existing
/metricsport 9464 plus GMP) with a drift dashboard. - D25, D26, D27 (Chris, 2026-10-23: all A): project-permission features are omitted when no project is open (as built in #4457); the anonymous availability view is dropped; seen and dismissed rows are kept on account deletion until the OD-6 subject-deletion work (as built in #4460). Details in §7.1.
- D36 to D43 (open, awaiting Chris): the eight 6G questions (help and What's New, walkthroughs, admin screens), with options and recommendations, in §7.3.
- Principle: every change must reduce complexity, not add to it.
- D28 to D35 (open, awaiting Chris): the eight 6F questions, with recommendations, in §7.2.
3. 6A — trustworthy compatibility checker and safe transition¶
Goal. A candidate build, a policy change or a mode switch is admitted only by a checker that reads a fresh, complete, environment-bound report. Runtime-to-Legacy is refused while stored policy and legacy values disagree.
Depends on: waves 3 and 4 merged.
| Slice | Content |
|---|---|
| 6A.1 | Offline checker CLI (scripts/runtime-compatibility-check), read-only. It takes a report and a candidate catalog/build manifest and gives exit codes 0 (pass), 1 (refuse), 2 (incomplete/unstable/unsupported) and 3 (unknown transport). Uses report schema v1 and the N-1 window. |
| 6A.2 | Runtime-to-Legacy parity preflight. It computes, per enrolled key, the runtime decision against the legacy value. Any difference or active stop refuses (RT-15). Pods never refuse to start over this. |
| 6A.3 | Replacement-build gate. A candidate whose catalog cannot honour the accepted policy, or whose migrations are not applied, is refused (RT-50, RT-67 logic). |
| 6A.4 | CI lane wiring for synthetic reports only. Trusted collection is never run on candidate code (RT-65). |
Acceptance:
- The checker passes a complete synthetic staging-like report and then refuses that same report for a
production-like environment (foreign
environmentId/bindingEpoch) (RT-62). - Expired, N-2-schema,
Unstable,IncompleteandUnknownOrUnsupported-field reports all fail with exit 2. N-1 passes (RT-61, RT-63). - The parity preflight refuses with the exact differing keys when a stop is active or a value differs, and passes once they agree (RT-15).
- A candidate with an older catalog digest, or missing a required migration, is refused (RT-50).
- Validator fixtures prove that no service credential reaches a job that runs candidate code (RT-65).
Verification: checker unit tests with fixture reports; validate-workflows.sh fixtures; API tests for the
parity computation.
4. 6B — multi-process convergence, stop and restore, recovery and freshness¶
Goal. Prove the propagation criteria with separate processes, not just in-process instances. Wave 2 already contributes an in-process 2 API + 2 PM harness (#4358).
Depends on: waves 2–5; 6A for the gating checks.
| Slice | Content |
|---|---|
| 6B.1 | Hermetic multi-process stack mode: e2e/run-local.sh --replicas 2 starts two API and two PM processes with distinct InstanceUids and fixed expected inventory, behind a round-robin proxy, with Shared mode on a locally bound synthetic environment. Bramble only. |
| 6B.2 | Convergence and timing harness. It drives off → on → off → on through the admin API, samples each process's decision and acknowledgement directly (per-process ports, so one healthy replica cannot hide another), and records p50, p95 and maximum latency against the bounds in §4.1. |
| 6B.3 | Disruption scenarios: silent replica (Degraded at 60 s), older-catalog new replica (Action required, no gated work), Mongo pause and resume, process restart, scale-up and scale-down, mixed image versions, late or out-of-order acknowledgements, stop with the outbox, docs and notification paths unavailable. |
| 6B.4 | Browser: Playwright with two tabs, bootstrap, reconnect, focus and online recovery. A stop is visible within the bound. Reload-safe recovery of an unresolved mutation by its original mutation ID (persisted only in session storage, without values). |
| 6B.5 | Retired-host clock-skew proof. Retirement uses server-side observation time and monotonic expiry. A host with a skewed clock cannot keep a fleet Active or make it stale early. Delivered by #4407: MongoDB-stamped serverSeenAtUtc, a 35-second quiet period against MongoDB's time, a 15-minute legacy-row bound, and Collector-mode production admission. |
4.1 Bounds under test¶
The trial objectives come from accepted-plan §10:
- A decision uses authority checked within 30 s (contract
maximumAgeSeconds). - The fleet is confirmed Active within 60 s (
ConfirmationWindow). - The refresh period is 8–12 s (shipped reader:
Random.Shared.Next(8, 13)seconds, matching the accepted 10 s ± 2 s), the read deadline is 5 s and acknowledgement freshness is 30 s.
The harness measures against these and records any violation. It never assumes instantaneous switching.
Acceptance:
- Two API, two PM and a browser converge to Active for each flip. Per-process decisions match the committed revision, and the measured maximum is within 60 s (RT-11).
- A silent replica is Unknown and the fleet is Degraded by 60 s. An older-catalog replica is Action required and receives no gated work (RT-11, RT-67).
- An emergency stop reaches every process within the stale limit with the outbox, docs and notification paths down (RT-12, RT-40).
- With Mongo paused, decisions fail closed to the contract's safe value. After resume they converge without operator action (RT-03). A revision regression or equal-revision digest change after resume leaves a stopped flag stopped (RT-04). An epoch change without an acknowledged rebind quarantines the reader (RT-05).
- A new or restarted process makes no gated decision before its first successful read (startup refresh before serving). A restarted process supersedes its predecessor's acknowledgement in the same pod identity, and a late write from the old process cannot replace it.
- Scale-up adds a replica that converges before receiving gated work. Scale-down retires a replica without leaving the fleet Active on a missing host's last acknowledgement. Expected inventory that disagrees with observed replicas is Action required.
- A mixed-version rollout (old and new images at once) keeps every replica on the accepted policy or refuses the replica that cannot honour it (RT-67). Delayed and out-of-order acknowledgements never move a host backwards or report a superseded identity as Current.
- A browser reload during an unresolved commit resolves it by the original mutation ID. It never issues a new mutation (RT-02 browser half).
- Clock-skew fixtures (±5 min) cannot hold a fleet Active or retire a fresh host.
- The evidence matrix (layer, revision, processes, assertions, timings, failures, gaps) is stored with the run.
Verification: Playwright plus a harness script on Bramble's local E2E stack; captured fleet output and timing CSV.
4.2 6B.4 detail¶
Verified against main at 4ee7c0d00. Browsers are never part of Active and make no admission decision, so the
browser bound is the 60 s confirmation window, measured from the commit response to the visible state in a tab with a
live hub connection. Expected worst case with G1 is about 27 s (pod refresh 12 s, read 5 s, hint, fetch about 9.5 s).
Record p50, p95 and max, and report against 30 s as information only.
Gaps that need product code
| Gap | Problem | Fix | Flag decision |
|---|---|---|---|
| G1 | In Shared mode nothing sends a revision hint, and there is no backplane, so a tab sees a flip only on focus, online, reconnect, reload or the 5-minute poll (matrix P3.11) | New IRuntimePolicyRevisionListener in SyRF.Mongo.Common/RuntimeFeatureFlags/, called by RuntimePolicyAuthorityRefreshService only when the observed identity changes (never at start-up; 2 s bound; failures swallowed). API RuntimePolicyBrowserHint.cs broadcasts RuntimeFeatureFlagsRevision(revision) on the pod's own connections. Shared commit and stop also broadcast, best-effort |
Not flagged. It runs only in Mode=Shared, which no shared environment uses. Hints carry a revision only and write nothing, so on → off → on is safe |
| G2 | A reload loses the unresolved mutation ID (RuntimePolicyChangeService, RuntimeEmergencyStopService keep it in memory only) |
New runtime-policy-unresolved-store.ts: one sessionStorage key syrf.runtimePolicy.unresolved.v1 holding { v, kind, key, mutationId, expectedIdentity, proposedIdentity, savedAtUtc } and no values, reason or digest. Written before the commit, cleared on saved, known refusal, cancel and Discard, kept on unknown. A stored record puts the service in a new recovering phase: read the authority first, then getMutationOutcome(mutationId). Committed and matching: saved. Unknown with a moved authority: "Not committed; superseded". Unknown with the authority unchanged: stay unresolved with Check again and Discard. Never sends preview, policy or emergency-stop after a reload |
Not flagged. Administrator-only, Shared mode, read-only. With no record or blocked storage the page is today's page; unknown versions are dropped |
| G3 | A failed first fetch keeps deployed values, which is permissive under a stop (RT-14). The server still refuses, so this is visibility only | Generator emits RUNTIME_SAFE_VALUES (keys with a consumerContract and productionEligible: true); a failed bootstrap applies them until the first accepted snapshot |
Flagged: deployment-static browserRuntimeSafeFallback (catalog-only, default false). A runtime flag cannot decide this because it is read before any snapshot exists |
Acceptance criteria
| # | Criterion | Verification |
|---|---|---|
| 1 | Only a change of the observed identity notifies the listener (once per change, never at start-up, never for an equal identity). A throwing or hanging listener never delays refresh or acknowledgement | New cases in RuntimePolicyAuthorityRegistrationTests (SyRF.Mongo.Common.Tests) |
| 2 | The listener broadcasts in Shared mode and is not registered in Legacy or Shadow. Shared commit and stop broadcast after success; a hub failure leaves the 200 receipt intact | RuntimeFeatureFlagsControllerTests plus a registration test (SyRF.API.Endpoint.Tests) |
| 3 | The record is written before the commit is sent and holds exactly the allowlisted fields (a test fails on any other). Cleared on saved, refusal, cancel, Discard; kept on unknown. Throwing storage keeps today's behaviour | pnpm exec ng test --no-watch --include="src/app/admin/runtime-feature-flags/**/*.spec.ts", new store spec and cases in both service specs |
| 4 | Recovery covers: Committed match, Committed mismatch, Unknown superseded, Unknown unchanged, lineage refusal, and authority read before outcome. The HTTP mock records zero previewPolicy, commitPolicy and emergencyStop calls, with a positive control in an ordinary commit |
Same specs plus runtime-feature-flags.component.spec.ts (banner; stop stays available) |
| 5 | run-local.sh --runtime-fleet --spec runtime-fleet-browser-<name> applies the config overlay, skips the harness driver and runs only the new Playwright project @runtime-fleet. Every other project ignores those files; every other --spec is still refused |
scripts/test-runtime-fleet-harness.sh, actionlint, shellcheck |
| 6 | Tab A is the admin page on api-1, tab B the membership page on api-2 (own login, apiOrigin override). Four flips committed on the other pod show in tab B within 60 s with no focus, online or reload event. A tab opened after each flip renders the committed revision. Timings go to browser-timings.csv |
runtime-fleet-browser-convergence.spec.ts on Bramble |
| 7 | An emergency stop from tab A hides "Disable membership" in tab B within 60 s and the admin page shows stopped. Restore brings back the prior override within 60 s | Same spec |
| 8 | Negative control: a routeWebSocket passthrough that drops RuntimeFeatureFlagsRevision frames leaves tab B stale for 75 s with focus suppressed. Criterion 6 is the positive control |
Same spec |
| 9 | With hint frames dropped: a focus event converges within 15 s; setOffline(false) after a flip converges within 15 s; a restored hub socket converges within 60 s |
runtime-fleet-browser-recovery.spec.ts |
| 10 | Reload during an unresolved commit: a route forwards PUT …/policy then aborts the response; after reload the page resolves to Saved by the original mutation ID (read from storage beforehand), with no preview, policy or emergency-stop request afterwards and exactly one new audit receipt. Variants: abort before forwarding gives "Not committed; superseded" after a flip from tab B; the same for an emergency stop; tab B's storage is empty |
Same spec |
| 11 | Each run writes the timing CSV, evidence.json (pass, per-criterion results, git SHA, stack) and the Playwright report. The rehearsal checklist gains browser rows beside R7.6, R8.3/R8.5 and R9.1 |
A Bramble run in the handover evidence directory with SHA256SUMS |
| 12 | Only if G3 is built: with browserRuntimeSafeFallback=true and a stop active, an aborted bootstrap hides the control until the first good fetch; with it off, today's behaviour. Tested on → off → on |
runtime-feature-flags.service.spec.ts, pnpm run validate:generated, one E2E case |
PR split (E2E on Bramble only, never the shared dev host)
- PR 1, web reload recovery (G2), #4419. Store, both services, banner, criteria 3 and 4. MVP; reviewable alone.
- PR 2, Shared hint (G1), #4420. Listener seam, API implementation, broadcasts, criteria 1 and 2. Parallel with PR 1. Moves the browser row in §10.3 of the accepted plan (runtime-feature-releases.md) and matrix P3.11 to "per pod, Shared".
- PR 3, E2E lane and specs. After PRs 1 and 2: the
@runtime-fleetproject,e2e/helpers/runtime-policy.ts(preview, commit, stop, outcome), both specs, checklist rows, criteria 5 to 11. If PR 2 slips, the recovery spec can land first. - PR 4, safe-default fallback (G3), #4449 (open). Approved (D15). Default off; criterion 12 and the flag decision in
manage-feature-flags.md.
Open questions (answered A by Chris on 2026-10-23, programme D15 and D16). G3: build the safe-value fallback now, default off (recommended, because it makes a stop visible when the bootstrap fails without changing default behaviour). The 5-minute poll: keep it, and state that the 60 s claim covers tabs with a live hub connection (recommended; a 60 s poll would multiply anonymous snapshot reads).
Not in scope: a browser age limit on stale snapshots (follow-up), a SignalR backplane, change streams, a round-robin proxy, and any staging or production run.
5. 6C — operational prerequisites and rehearsal planning¶
Goal. Everything an operator needs to bind, import, enrol and run the collector, written down and rehearsed in isolation, with no live execution.
Depends on: 6A, 6B.
| Slice | Content |
|---|---|
| 6C.1 | Runbooks: environment binding, lineage preview/apply with writer freeze, consumer enrolment, collector identity and scope, coordinator identity, expected-inventory values, recovery and rebind, stop and restore, Legacy return. Each runbook lists preflight checks and their expected outputs. Written in Operate runtime feature flags (#4408). |
| 6C.2 | An isolated rehearsal in the local stack that follows the runbooks end to end, with the transcript recorded. Step map and recorded outcomes: isolated rehearsal checklist (run 2026-10-22 on 45091ad7: 47 PASS, 0 FAIL, 13 not exercised; #4413). |
| 6C.3 | A staging rehearsal plan (RT-69 checklist, OD-26), including entry and exit criteria for Chris's later gate. Not executed. Written in the staging rehearsal plan. |
| 6C.4 | A production prerequisites list: catch-up promotion, ADR-023 Job, OD-21 automation identity, OD-23 collector (camarades-infrastructure), chart values, and an approval record template. Not executed. Written in the production prerequisites. |
| 6C.5 | Legacy reader for initialized authority (owner decision D7=A). A host in Legacy mode against an initialized authority starts, serves deployment-controlled values, never applies runtime overrides or stops, never writes the authority, and does not register as a writer. Returning to Legacy therefore stays a configuration change after an import. Flag decision: not separately flagged, because the existing mode setting is the switch, and a Legacy host with initialized authority previously refused to start. |
| 6C.6 | First-time initialisation without a legacy document (owner decision D8=A). A reviewed initialize lineage mode creates the authority from deployed values (no overrides, no stops), with the same preview/apply digest discipline, writer freeze and refusals as import. Production uses it instead of import. Flag decision: not flagged, because it is an operator command that runs only when explicitly selected before host start. |
Acceptance:
- Every runbook step is exercised in the isolated rehearsal, and the transcript shows the expected output.
- The staging and production lists name the owner, approver, rollback-as-forward-release, and evidence to capture for each step.
- No runbook step requires a production credential during rehearsal.
- (6C.5) Legacy-mode API and PM hosts start against initialized authority and serve deployed values, with no write to the authority. A Shared → Legacy → Shared round trip leaves the authority byte-identical. Legacy behaviour against a legacy-shaped document is unchanged. An isolated
legacy-returnscenario proves the round trip, and runbook 10's STOP box is replaced with executable steps. - (6C.6)
initializepreview is deterministic, and apply requires the reviewed digest. It refuses an existing document, an unverified binding, registered writers and migrations that aren't Ready. An exact replay is idempotent. With an isolated production-like configuration, the initialised authority is accepted by Shared readers, the evaluator and fleet admission. An isolated rehearsal reaches Active, and the checker'spolicy-changepasses.
5A. 6R — registry and developer experience¶
Source: owner decisions of 2026-10-23, verified against main at f92b496fe. 6R runs before 6D Apply and replaces the
old "6D.0 registry communication fields" slice.
Principle. Every change must reduce complexity, not add to it. Each PR description carries a Complexity effect block with four before and after numbers: flags, concepts a caller must handle, generators, and files touched to add one flag. The inventory page (6R.2) computes them. Every human-written source is YAML.
Key constraint. No new registry field enters RuntimeFeatureFlagDefinition. Its fields feed
RuntimePolicyAuthority.ComputeCatalogDigest, and the digest is part of every stored RuntimePolicyIdentity, so any
change would make every stored identity foreign. A digest-pin test (value pinned at f92b496fe) enforces this. New fields
go on FeatureDefinition only. That changes the Prepared release snapshot, which is acceptable: no shared environment has
RuntimeFeatureReleases:Enabled, and 6D Apply refuses a changed snapshot ("prepare a new release").
6R.1 Registry v2 and one validator¶
- Registry v2.
src/charts/syrf-common/feature-definitions.yamlbecomesschemaVersion: 2(path kept). Each feature gainstype(Release | Operational | KillSwitch | Experiment | Permission),owner,lifecycle(Developing | Piloting | Released | Settled | Withdrawn; "Ready to settle" is computed, never stored), an optionalsettleDaysoverride (default per type: Release 30),flags, and an optionalcommunicationblock (userGuide,walkthroughId,whatsNewannounce/quiet,notifyreleaseAudience/administrators/none,alertingOwner). Defaults come fromtype: Release needs a user guide, announces and notifies the release audience; every other type is exempt, quiet and notifies no one. The user-guide requirement lives incommunication.userGuide; schema 2 refuses a top-levelguidance. - Groups and pins move into the registry.
flagGroupsreplacesRuntimeFeatureFlagGroups.cs(order kept, two retained unadvertised IDs resolve throughFindonly).deploymentManagedreplacesProjectStatisticsRuntimeActivation.PinnedKeys. Group IDs are versioned contracts: the validator refuses a change to thekeysof an existing group ID. - Other YAML sources (new, under
docs/content/):release-content.yaml(the manifest's shape; the JSON manifest is deleted; the compiler accepts both),whats-new.yaml(generatessrc/services/web/src/assets/data/whats-new.jsonbyte for byte), andwalkthroughs.yaml(format only; first file arrives with 6G.4). - Generator.
src/charts/syrf-common/scripts/generate-feature-definitions.tsstays the single registry generator and also writesRuntimeFeatureFlagGroups.generated.cs,RuntimeFeatureFlagPins.generated.csandwhats-new.json.FeatureDefinitiongainsTypeandCommunication, resolved at generation time so 6D reads final values.RuntimeFeatureFlagCatalog.generated.csis untouched. - One validator. New
scripts/feature-registry.tsplusscripts/feature-registry.test.ts, run byscripts/validate-generated.tsand aspnpm run validate:registry [--gitops <dir>]. Checks: (1) flag keys exist and no key has two owners; (2) required guidance has a non-Draft content entry once a feature leavesDeveloping; (3) What's New and user-guide links resolve (by path or Jekyllpermalink:), and walkthrough anchors exist asdata-tour-id; (4) group members are catalogued and group IDs never change; (5)env-mapping.yamland the generated feed pass their schemas (the 14 currentsecretKeySourceerrors are fixed by makinghelmSource.sourceoptional); (6) every registry source path is covered by thepr-tests.ymlchange regex and the.husky/pre-commitpattern; (7) no cluster-gitops check here: environment values are cluster-gitops' concern (Chris, 2026-10-23). SyRF publishes the valid key list instead (R1c), and cluster-gitops checks its own values against it. Nine dead keys exist there today. - Flag decision. Not flagged: build-time sources, generation and validation. Byte-identical outputs and unchanged
goldens prove runtime behaviour is unchanged, and the new fields have no consumer until 6D PR A, which sits behind
RuntimeFeatureReleases:Enabled.
6R.2 Developer experience¶
pnpm generate(scripts/generate-all.ts) runs the generators in dependency order (env blocks, web flags, runtime catalog, feature definitions, Dockerfiles, and nswag only when the spec hash changed), thenvalidate:registry. A test fails if a generator is added toGENERATOR_COMMANDSwithout an order slot.pnpm feature:new <id> --type <T> --owner "<team>" [--permission <P>] [--flag <key>] [--hosts api,project-management](--permissionrequired for a Release, 6E.1) writes exactly oneenv-mapping.yamlentry and one v2 registry entry (Release: default settle period, Required user guide with a topic the validator reports as "guidance missing" until content exists; others: Exempt with a reason), then regenerates. With--hosts api,project-managementit also writes the documented Project Management half (backendFeatureFlags) and adeploymentManagedpin, because Project Management never reads runtime overrides (#4292). It refuses an existing ID or key (and a removed key), writes nothing on refusal, and prints the remaining steps (consumer code, catalog digest pin, content, What's New, walkthrough, cluster-gitops). The slow end-to-end tests of these commands run aspnpm run test:tooling-e2e(CI's Validate Generated Code jobs, and pre-commit only when the tooling changes), sovalidate:generatedstays fast.pnpm feature:settle <id>does the mechanical part of settling aReleasedfeature: it removes the feature'senv-mapping.yamlflag entries, setslifecycle: Settled, adds the keys to the removed flags (removedFlags, generated asRuntimeFeatureFlagKeys.RemovedFlags; stored overrides for them are ignored and the keys are never reused), drops any pin, regenerates, and prints every code reference from the consumer manifest and a tracked-file search, so the developer deletes the "off" branches in the same clean-up PR ("remove flag<key>for<feature>"). It refuses a feature that is not Released, and writes nothing on refusal. Ordering safety: before merging, confirm every environment serves "on" (the cluster-gitops value and any runtime override), so making "on" unconditional changes nothing.- Generated key constants.
RuntimeFeatureFlagKeys.generated.csreplaces the 12 hand-written constants (same names and values) with one constant for each of the 54 catalogued keys, plusRemovedFlags. - Inventory page. Generated
docs/features/feature-inventory.md, in the MkDocs nav and deterministic. It shows flags, features by type, unclaimed flags, evaluation seams, and generators and files per flag. It holds no dates and no runtime state. - Settling is fact-based (Chris, 2026-10-23), with no manual dates. Git holds the intent: the type, plus an optional
settleDaysoverride. Each environment's own release history holds the fact: the release milestone where the feature became available to everyone (6D), or, before 6D, the flag's audit history. The environment's admin page lists "Ready to settle" (a Release type, available to everyone and unstopped for longer than its settle period), with a "Create clean-up issue" link that opens a prefilled GitHub issue in the viewer's browser, so the environment holds no GitHub credential. Nothing time-based ever fails a build, and runtime facts never feed compile-time checks. This removes thereviewByfield, the expiry test and the scheduled expiry job. - Flag decision. Not flagged: developer tooling and a generated document; only key constants change, with identical names and values.
| Adding one flagged feature | Before | After |
|---|---|---|
| Commands | 5–6, from 3 directories | 2: pnpm feature:new …, then pnpm run validate:generated |
| Hand-edited registry files | 1–3, plus up to 6 situational | 0 (the scaffold writes 2) |
| Hand-edited test pins | 3 (catalog fixture, flag count, catalog digest) | 1 (catalog digest; the fixture now checks only existing flags and the count assertion is gone, since the digest pins the set) |
| Generated files touched | 13–16 | 15–18 (adds key constants and the inventory page) |
| Links that can break unchecked | What's New, guidance, groups, cluster-gitops keys | none in SyRF (cluster-gitops checks its own values against the published key list) |
The generated-file count rises by two. That is accepted under the principle, because what a developer edits and reviews
falls: hand-edited sources go to 0, commands to 2, and the two new outputs replace hand-maintained key strings and an
inventory nobody keeps today. Generated files are machine-checked by validate:generated. The generator count stays at
5 in R2a and becomes 4 after the backlog merge of the two web flag generators (R2a left it: merging a 1,374-line and a
683-line generator is not low-risk).
R2a outcome (complexity effect). Flags 55 and caller seams 5 unchanged; generators 5 unchanged; commands to add a flag 5–6 → 2; hand-edited key constants 12 → 0; hand-edited test pins per flag 3 → 1; settling a feature becomes one command plus the printed deletions.
6R.3 Module split and one evaluation interface¶
- Reader package stays
SyRF.Mongo.Common, so startup-bound consumers and PM wiring are untouched. First, extract the four reader-to-writer reaches into reader-side files (RuntimePolicyReceiptValidation.cs,RuntimeFeatureFlagEffectiveValues.cs, and aRuntimePolicyFleet.Freshnessconstant). Host acknowledgements stay, because every Shared host writes its own heartbeat. - API-only module
src/services/api/SyRF.API.RuntimeAdministration/(referenced only bySyRF.API.Endpoint; added toapi.slnf,syrf.sln,docs/architecture/dependency-map.yaml, regenerated Dockerfiles). It receives the policy mutation store partials,RuntimeFeatureFlagStore, lineage import, environment report and workload observation stores,RuntimeReleaseIntent.cs, and every 6D store, reconciler and dispatcher (6D-Q1). - Architecture test (R3a, in
SyRF.ProjectManagement.Endpoint.Tests, modelled onDeletionArchitectureScan): fails if PM production code names any writer type. The writer list is derived, with a positive control that findsRuntimePolicyMutationStore. After the move, an assembly-closure test and a csproj scan keep PM free of the module. IFeatureDecisions(R3c, reader package):Decide(key)andDecideAll(keys)(one immutable capture). Synchronous, never throws,Valuealways abool(the catalogsafeValueapplies when a contract key is unavailable, stale or stopped),Reasonmapped 1:1 from the eight evaluator states,PolicyVersionastring?. It replacesIRuntimeFeatureFlagProvider.IsEnabled,SharedRuntimeFeatureFlagReader.EvaluateandIProjectStatisticsFlagSourcefor the 13existing-runtimeandshared-readerconsumers. Caller seams go from 5 to 2 (FeatureFlagsstays as deployed configuration). A parity test covers every catalog key and evaluator state.- Flag decision. R3a is a test only. R3b moves code without behaviour change. R3c is on hot paths but not flagged: the parity test proves identical values, and a flag would keep both paths alive (more complexity). A micro-benchmark shows no added allocation.
OpenFeature assessment. Not adopted as the caller interface (both SDKs are open source, so cost is not the reason). It
would lose three things: the catalog-owned fail-closed safe value (OpenFeature returns the caller's default on error, so a
permissive default fails open); synchronous decisions (the .NET client is async only, so authorisation and admission
callers would become async); and multi-key single capture (it evaluates one flag per call, so keys could read different
revisions). IFeatureDecisions uses OpenFeature vocabulary (value, reason, metadata), so an OpenFeature provider adapter
can be added later if an external tool needs the standard. Callers would not migrate.
Acceptance criteria¶
| # | Criterion | Verification |
|---|---|---|
| 1 | The generator over the migrated YAML reproduces whats-new.json, FeatureDefinitions.generated.cs and feature-definitions.generated.ts byte for byte; compiler bundle digests are unchanged |
pnpm run validate:generated, plus before and after sha256sum and digest file names in the R1a description |
| 2 | Each validator check (1–6) has a failing fixture, a passing positive control and an exact message (a permalink-only What's New path passes; a changed group membership fails; the 14 schema errors are gone) | tsx --test scripts/feature-registry.test.ts, inside validate:generated |
| 3 | RuntimePolicyAuthority.CatalogDigest equals its value pinned at f92b496fe, and RuntimeFeatureFlagCatalog.generated.cs is unchanged by R1a and R1b |
New digest-pin test in SyRF.Mongo.Common.Tests, plus checksums |
| 4 | Groups and pins from YAML leave RuntimeFeatureFlagSnapshot.golden.json, RuntimeFeatureFlagGroupTests and the pin refusal texts unchanged; both retained group IDs resolve through Find and are absent from All |
SyRF.API.Endpoint.Tests |
| 5 | Release resolves to announce and releaseAudience; each other type to quiet and none; an override wins; generated C# and TypeScript carry resolved values |
Generator unit tests and the C# catalog test |
| 6 | Each chart release publishes the valid config-key list generated from env-mapping.yaml (deterministic, checksummed). A cluster-gitops check against it reports the nine dead keys |
Generator test in SyRF; the cluster-gitops PR's own check run (Chris merges) |
| 7 | pnpm generate from a clean tree is a no-op and runs in the documented order; a generator without an order slot fails |
Order slots: scripts/generate-all.test.ts (in validate:generated); no-op against a temp copy: scripts/tooling-e2e.test.ts (pnpm run test:tooling-e2e, a CI step after validate:generated) |
| 8 | feature:new writes exactly the two source entries, regenerates and passes validate:generated; a re-run refuses and writes nothing; each type gets its defaults. feature:settle refuses a feature that is not Released, writes nothing on refusal, and leaves validate:generated passing |
scripts/feature-commands.test.ts (planning, refusals, type defaults) and, against a temp copy, scripts/tooling-e2e.test.ts |
| 9 | The inventory page is deterministic, in the nav, and shows the four complexity numbers | Checksums, docs/scripts/check-mkdocs-nav.sh |
| 10 | "Ready to settle" lists a Release feature only when it has been available to everyone and unstopped for longer than its settle period, from that environment's own history; the issue link is a prefilled URL with no credentials | API unit and integration tests with an injected clock; a component spec (built with the admin area) |
| 11 | The R3a scan fails on a fixture PM file naming RuntimePolicyMutationStore, and its positive control finds every derived writer; after R3b the assembly closure test passes for PM |
SyRF.ProjectManagement.Endpoint.Tests |
| 12 | R3b changes only using lines and project references; API and PM images build |
CI Test Summary, Dockerfile checksums |
| 13 | R3c parity for every key and state; DecideAll uses one capture; no added allocation |
SyRF.Mongo.Common.Tests, SyRF.API.Endpoint.Tests, PM tests |
| 14 | Each PR description has the Complexity effect block, matching the inventory page diff | Review |
Complexity before and after¶
| Before | After | |
|---|---|---|
Flags (env-mapping flag keys: 54 catalogued plus the web-only activeReviewerTrackingEnabled) |
55 | 55 (settling stale flags is backlog) |
| Caller evaluation seams | 5 | 3 after R3c (IFeatureDecisions, FeatureFlags, and IRuntimeFeatureFlagProvider for five pinned callers); 2 once they migrate |
| Registry generators | 5 | 5 (4 after the web flag generator merge) |
| Hand-edited sources per flagged feature | 1–3, plus up to 6 situational | 0 (scaffold writes 2) |
| Hand-written key constants | 12 | 0 |
| Hand-written JSON sources | 2 | 0 (YAML) |
| Writer code shipped in PM | yes | no (R3b) |
PR split¶
| PR | Content | Depends on | Lane |
|---|---|---|---|
| R1a (MVP), #4428 | Validator checks 1–6, schema drift fix, whats-new.yaml and release-content.yaml, CI regex and pre-commit updates, how-to updates |
main | A |
| R1b (MVP), #4432 (merged) | Registry v2 fields, type defaults, flagGroups, deploymentManaged, generated groups and pins, digest-pin test |
R1a | A |
| R1c, #4436 (merged) | Publish the valid config-key list with each chart release; a separate cluster-gitops PR adds that repo's own scheduled and PR check, which Chris merges | R1a | C |
| R2a, #4437 (merged) | pnpm generate, feature:new, feature:settle, generated key constants and removed flags |
R1b | B |
| R2b, #4441 (merged) | Inventory page | R1b | B |
| R3a, #4429 | Architecture source scan | main | D |
| R3b, #4433 (merged) | Reader extraction and API-only module move | R3a | D |
| R3c, #4439 | IFeatureDecisions and migration of the consumer seams; nine remaining reads pinned in legacySeamAllowList for follow-up |
R3b | D |
| Admin "Ready to settle" (acceptance 10) | The list and the prefilled "Create clean-up issue" link, in the Features and releases admin area (wave 7, D18). It reads 6D's release milestones, with each flag's audit history as the interim source | 6D PR A (milestones); R2b | Wave 7 |
Lanes A (R1a, R1b) and D (R3a, R3b) run in parallel from day one. 6D PR A starts after R1b, and ideally after R3b, so its
new files are created in the API module (if A is ready first it lands in SyRF.Mongo.Common and R3b moves it). B and C
follow R1b and do not block 6D. The MVP boundary is R1a + R1b.
Decisions on the open questions (Chris, 2026-10-23)¶
- Q1, expiry: replaced. There are no review-by dates; settling is fact-based (6R.2), shown in each environment's admin page and never blocking a build.
- Q2, existing flags: the 54 existing catalogued flags (the 55 env-mapping flag keys minus the web-only
activeReviewerTrackingEnabled) are registered in one agent-drafted PR after R1b, reviewed by Chris: #4438 (open, conflicting with main). - Q3, cluster-gitops: it is environment maintenance, so it belongs to cluster-gitops. SyRF publishes the valid key list (R1c), and cluster-gitops checks its own values in its own CI and opens clean-up issues. Clean-up PRs that touch production values stay owner-approved.
Simplification backlog (each a later PR; Shadow and legacy routes are owner-gated)¶
| Item | Precondition | Proof nothing is lost |
|---|---|---|
Settle stale flags with pnpm feature:settle (candidates: contactUsForm, newProgressIndicators, newScreeningOverview, newStageOverview, showUpdateInclusionInfo, plus the 9 dead cluster-gitops keys) |
R2b; flag true in production and staging for a promotion cycle; the cluster-gitops production PR needs Chris |
On-path tests pass with the flag deleted; validator shows no reference; rendered env differs only by the removed variable |
| One opaque policy version | 6B.4 PR 1 merged | Round-trip test for every field including Int64 revisions beyond 2^53; all 409 conflict tests pass; the 7-field body stays until no N-1 web build is served |
Remove Shadow mode (Mode=Shadow and SharedReaderMode=Shadow) |
Owner confirmation | Nothing configures either, production cannot run Shadow, writes refuse it, and Shared mode plus the offline checker give the same evidence. Start-up then fails closed on the invalid mode. ApplicationAuthorityMode.Shadow is unrelated and stays |
| Remove legacy write routes | Production on Mode=Shared with ProductionPolicyMode=Runtime; Legacy return kept as a read-only mode |
Shared-mode suites pass unchanged; groups then lose their writer |
| Merge the two web flag generators | R2a | All 11 outputs byte-identical; generators 5 to 4 |
6. 6D — release Apply, events, outbox and confirmation¶
Depends on: wave 5 (#4380 Prepared) and 6A (fleet confirmation input).
| Slice | Content |
|---|---|
| 6D.1 | An Apply transaction joining policy, release intent, PolicyCommitted event and outbox rows (RT-20). Idempotent by intent ID. |
| 6D.2 | A confirmation reconciler. It confirms only when the fleet reaches Active for that identity, using ConfirmedAtUtc from the server clock (RT-21). |
| 6D.3 | An outbox dispatcher with durable retry, staging and preview refusal (RT-25), a guard against acting after a stop (RT-23), and finished entries kept permanently with no TTL (ADR-023 / #4319). |
| 6D.4 | Stop and restore keep the release ID, original milestone dates and announcement identity (RT-24). Historical import queues nothing (RT-27). |
| 6D.5 | Normal enable is refused without required guidance or with an incompatible host, while a stop still works (RT-26). |
Acceptance: RT-20 to RT-27 pass as specified, including RT-22: a crash at each boundary resumes to one event per milestone and one announcement.
6.1 Detail¶
Verified against main at 4cbfe924a (after #4382). The merges since, up to 4ee7c0d00 (#4383–#4385, #4414), change the admin UI and the legacy reader, not the release store, routes or migrations named here. Today Prepare (#4380) writes a runtimeFeatureRelease record (status: "Prepared")
and a runtimeFeatureReleaseEvent receipt in one transaction, and nothing consumes it. The release indexes 0012 to
0017 and 0021 already exist; the next free migration is 0022. Everything below runs only in isolation.
Cross-cutting design
- Event IDs.
_id = "{mutationId}:{type}". Apply events use the Apply mutation ID. Events the reconciler derives use"{policyMutationId}/{releaseId}"withsourceMutationIdstored. The unique{mutationId, type}index (0013) is the exactly-once key. - Outbox dedup key (unique by
0016):"{environmentId}:{bindingEpoch}:{releaseId}:{stage}:{announcementId}:{announcementRevision}:{kind}", withannouncementId = "{releaseId}:{stage}"and revision 1. It never changes on stop or restore (RT-24). - Record CAS. Every release-record write filters on
{_id, version: V}and setsV+1; any other modified count aborts. Leases avoid duplicate work; the CAS and unique indexes are the correctness guarantee. - Transactions. Snapshot read concern, primary, majority write, binding re-read inside the transaction.
- Fault hook.
RuntimeReleaseFaultPointplus aninternal Action<…>? FaultHookon the store and dispatcher, a no-op unless set bySyRF.Mongo.Common.Tests. Points:ApplyAfterPolicyWrite,ApplyAfterEventInsert,ApplyBeforeCommit,ReconcileBeforeCommit,DispatchAfterLease,DispatchAfterHandler.
Status (2026-10-23). Merged: PR A (6D.1 Apply with the 6D.5 admission stub, #4443),
PR B (reconciler, #4444), PR C (dispatcher, #4445),
PR D (stop and restore recording, #4453), PR E (guidance admission,
#4446) and PR F (historical import, #4447).
PR G (#4461, merged at 21c73ed99) added the RT-22 crash matrix, the
release-lifecycle harness scenario and the close-out docs. 6D is complete for isolated environments: RT-20 to RT-27
all pass on main. Nothing in 6D runs in a shared environment.
What remains before any production release (written prerequisites only; 6D executes none of them, and each is a separate reviewed change with owner approval):
- A releasable registry entry.
project-membership-disablehas nocompatibleWorkflowVersions, so Apply refuses it withWorkflowCompatibilityMissing. Its guidance is Required, so it also needs a public article for every topic (or a reviewed exemption). - The compiled server guidance bundle mounted read-only on the API (
RuntimeFeatureReleases:GuidanceBundlePath, production prerequisite P12a). No environment mounts one. - Migrations
0011–0022applied and bound in the target database through the pre-deploy Job (ADR-023), which is enabled in no environment today (§5). RuntimeFeatureFlags:Mode=SharedandRuntimeFeatureFlags:ProductionPolicyMode=Runtimein production, with production fleet admission, Collector inventory and the 6C rehearsal evidence. Production runsLegacytoday.RuntimeFeatureReleases:EnabledandRuntimeFeatureReleases:OutboxDispatch:Enabledset per environment in cluster-gitops. Both are false everywhere.- Real What's New and in-app handlers (6G). Until they exist, a production binding has no outbound handler, so its
notices stay
Pendingand What's New staysPendingeverywhere. - An operator requeue for
Failedoutbox rows, and the refusal of an Apply under an active stop (#4456).
6D.1 Apply (RT-20). New routes POST /api/feature-releases/{releaseId:guid}/apply/preview (side-effect-free; digest
domain syrf-release-apply-v1; marked [ImpersonationReadOnlySafe]) and .../apply (body RuntimeReleaseApplyRequest(
MutationId, ExpectedIdentity, PreviewDigest, Reason); response RuntimeAppliedRelease, Status = "Applying",
ConfirmedAtUtc = null). They need ManageFeatureReleases, Shared mode and RuntimeFeatureReleases:Enabled. Refused when
the record is not Prepared, the definition snapshot differs from the catalog, the identity, preview or binding is stale,
or admission fails (422 runtime_release_admission). An exact retry returns the original; a changed body or a second
different Apply is refused. One transaction: re-check binding, validate receipt chain, replay check, require Prepared,
admission, CommitPolicyInTransactionAsync (extracted from MutateAsync by a pure refactor), record CAS 1→2
(status: "Applying", applied block, unconfirmed milestone), PolicyCommitted event, outbox rows (Pending,
requiresConfirmation, guardPolicy) and an AnnouncementQueued event.
- Files (writer and release code lives in the API-only
src/services/api/SyRF.API.RuntimeAdministration/since 6R R3b):RuntimePolicyMutationStore.cs; newRuntimeReleaseApplyStore.cs(partial) andRuntimeFeatureReleases/RuntimeReleaseLifecycle.cs(request, preview, result, event type, outbox kind, state,RuntimeReleaseDedupKey);RuntimeReleaseIntentStore.cs(injectable definitions);RuntimeReleaseIntentAdministration.cs,RuntimePolicyAdministration.cs,RuntimeFeatureFlagsController.cs(422 mapping); authorization classification files; goldens; regeneratedswagger.jsonandapi-client.generated.ts. - Acceptance: nothing is written at each of the three Apply fault points; an exact retry gives one audit row, one event and one row per dedup key;
two concurrent Applies commit once; stale preview, identity, binding, definition and non-Prepared each refuse with a stable message; switch off refuses
Apply but
GETstill reads Applying records (on → off → on); permission is checked first and Apply is a change for impersonation;MutateAsyncsuites pass unedited. Tests: newRuntimeReleaseApplyTests(replica set),RuntimeReleaseIntentAdministrationIntegrationTests. - Flag: behind existing
RuntimeFeatureReleases:Enabled(default false) and Shared mode. Off refuses new Applies only; applied work keeps converging.
6D.2 Confirmation reconciler (RT-21). A leased worker reconciles releases that are Applying, Degraded, Active or Paused
and have an applied block. In one transaction it reads server time first ($$NOW, Q2), then confirms only if the status is
Applying/Degraded, the capability holds, the store epoch matches with Revision ≥ applied, and the fleet state is Active.
Confirmation sets confirmedAtUtc, status: Active, semantics ObservedActivation (or ObservedAfterRecovery after a gap over 3 periods) and inserts
ActivationConfirmed. An Applying release past 60 s with no Active fleet becomes Degraded (or ActionRequired) with no milestone; a later Active still
confirms once. The lease (reconcileLease, 30 s) is acquired against $$NOW. Host: RuntimeReleaseReconcilerService (10 s ± 2 s), Shared mode and writer only; fails closed
if indexes are missing. New RuntimeReleaseReconciliation.cs and RuntimeReleaseReconcilerService.cs; RuntimePreparedRelease gains a nullable Milestones projection (golden update).
Acceptance: confirms only when every expected host is Current (never with Behind, Unknown, Incompatible, missing inventory, reversed capability or foreign epoch); ConfirmedAtUtc is MongoDB time and
unchanged by a ±5 min host skew; Degraded then Active confirms once; two racing reconcilers give one event; a non-membership key never confirms; the service is absent in Legacy and non-writer hosts. Flag: not
separately flagged (convergence of accepted work, a no-op without Applying records, absent in Legacy).
6D.3 Outbox dispatcher (RT-23, RT-25). Lease one due item (Pending or expired Leased, via 0017), guard in a transaction, handle through
IRuntimeReleaseSideEffectHandler (idempotent by dedup key; a kind with no handler stays Pending), then complete by CAS on the lease token. Guards: binding or epoch differs from the key gives Cancelled/BindingChanged; outbound kinds with delivery
disabled give Cancelled/DeliveryDisabled; testSink reaches only RuntimeReleaseTestSinkHandler; an unconfirmed milestone defers with backoff (cap 1 h); a stopped capability cancels NotifyInApp (AnnouncementCancelled) and defers
MaterialiseWhatsNew. Handler errors back off and become Failed after 20 attempts. Done, Cancelled and Failed rows are never deleted: no TTL, no clean-up job. New migration 0022_runtimeFeatureReleaseOutbox_releaseId_state
on {releaseId, state} (added to PmMigrations.cs, migrations.lock.json and both migration test files; like 0021, it stays outside the fixed 0011–0020 RuntimeReleaseIndexes group). New RuntimeReleaseOutboxDispatcher.cs, handler interface, test-sink handler and hosted service. Acceptance:
a stop between lease and guard cancels once with an empty sink (fault hook DispatchAfterLease); staging reaches only the sink, preview:pr-N and local: end DeliveryDisabled, a restored copy ends BindingChanged; failing handlers back off and survive as
Failed; no TTL index; dispatch off, on, off, on delivers nothing twice. Flag: new RuntimeFeatureReleases:OutboxDispatch:Enabled (default false), the only component with outbound side effects. Off keeps rows Pending.
6D.4 Stop, restore and import (RT-24, RT-27). The stop and restore write path is unchanged (no release, outbox or docs dependency). The reconciler reads audit receipts after applied.lastObservedRevision (index 0010, batch ≤ 100) and appends EmergencyStopped, Paused
or Restored events (ID "{receiptMutationId}/{releaseId}:{type}", occurredAtUtc = receipt time). Status goes Paused then Applying then Active; milestones are never rewritten and a restore adds no second ActivationConfirmed. When the reconciler records a stop, it cancels unsent NotifyInApp rows in
that same reconciler transaction; the stop's own write path is unchanged. Historical import is a new reviewed command, RuntimeReleaseHistoryEntryPoint (SYRF_RUNTIME_RELEASE_HISTORY_MODE=preview|apply, input, output, actor, wired next to the lineage entry point). Preview prints exact documents and a digest; apply needs the digest, writes an Active
imported: true record with no applied block plus a HistoricalImported event, and writes no outbox rows. Acceptance: after confirm, stop, restore the release ID, milestone dates and announcement ID are byte-identical with one of each event; a stop and restore inside one period still yields both events; the stop succeeds with the
release indexes dropped, the dispatcher throwing or the reconciler stopped; import keeps Date/Month/Unknown precision, queues nothing, and refuses a stale digest, unknown feature or duplicate key. Flag: neither is flagged (part of the reconciler; an explicit operator command).
6D.5 Release admission (RT-26). RuntimeReleaseAdmission runs inside Apply and its preview. Guidance (as built in PR E, #4446): when required, RuntimeReleaseGuidanceBundle reads the compiled server bundle (server-<digest>.json from
compile-release-content.ts) from RuntimeFeatureReleases:GuidanceBundlePath, re-verifies the file-name digest from the parsed content, and requires every topic present for the feature, published PublicPreview or PublicReleased (the Global audience), with its articles covering the reviewed compatibleWorkflowVersions; a record's optional ContentBundleDigest must match. Reasons GuidanceBundleMissing, GuidanceBundleDigestMismatch, GuidanceTopicMissing, GuidancePublicationNotReleased and GuidanceWorkflowNotCovered (unset or unreadable path fails closed; GuidanceUnverifiable is removed); exempt features pass with the exemption recorded. Hosts: an Action required fleet refuses with
IncompatibleHost or FleetActionRequired in every environment. Emergency stop never runs admission. Acceptance: missing bundle, missing topic, Draft-only topic and unset path give 422 with the exact reason and no write while the stop succeeds; a foreign catalog digest gives IncompatibleHost; C# and TypeScript digests agree on a golden fixture and a tampered
byte fails; an exempt feature passes with the exemption in the event. Flag: not flagged (it only tightens Apply; a bypass would defeat OD-5).
RT-22 crash table. A table-driven RuntimeReleaseCrashRecoveryTests (replica set) resumes to one PolicyCommitted, one ActivationConfirmed, one outbox row per dedup key, one sink row and a final Active. Built in PR G (#4461): 11 cases (each Apply fault point, and the gap of boundary 4, have their own row), each with a positive control that the fault fired.
| # | Boundary | Fault | Resumption |
|---|---|---|---|
| 1 | After Prepare, before Apply | process lost | Apply with the same mutation ID applies once |
| 2 | Inside Apply | hook at each of the three Apply points | nothing written; retry applies once |
| 3 | Apply committed, response lost | drop the result | retry returns the original; no duplicates |
| 4 | Inside confirmation | ReconcileBeforeCommit |
next tick confirms once (ObservedAfterRecovery if gap > 3 periods) |
| 5 | After confirmation | kill the reconciler | next tick is a no-op (CAS) |
| 6 | Mid-outbox, after lease | DispatchAfterLease |
lease expires, item re-leased, delivered once |
| 7 | Mid-outbox, after handler | DispatchAfterHandler |
re-delivery hits the sink's dedup key; Done once |
| 8 | Stop between confirmation and dispatch | stop committed | AnnouncementCancelled once; sink empty |
A release-lifecycle scenario (e2e/harness/runtime-fleet/release-lifecycle.mjs, run through run.mjs; Bramble, 2 API + 2 PM, local binding) repeats this end to end, SIGKILLing the lease-holding API mid-run. Built in PR G as a named-only scenario. It needs a releasable registry entry, so on main it stops at the Apply preview with WorkflowCompatibilityMissing (prerequisite 1 above); the full run used a rehearsal-only registry entry in the scratch clone (evidence 6D-release-lifecycle/ in the handover directory).
PR split (each default-off and independently mergeable)
| PR | Content | Depends on |
|---|---|---|
| A (MVP) | 6D.1 Apply with a fail-closed admission stub (guidance-required features refuse with GuidanceUnverifiable; IncompatibleHost checked), MutateAsync refactor, injectable definitions, routes, outbox insertion, forward-rule test |
main |
| B | 6D.2 reconciler and hosted service | A |
| C | 6D.3 dispatcher, test sink, 0022, dispatch switch |
A |
| E | 6D.5 guidance verification (content index, golden digest) | A |
| F | 6D.4 historical import (RT-27) | A |
| D | 6D.4 stop/restore recording and cancellation (RT-24) | B, C |
| G | RT-22 crash matrix, release-lifecycle scenario, docs (lifecycle how-to, matrix R.2/R.3, mongodb-reference.md, this plan's §6 status) |
B, C, D |
B, C, E and F run in parallel after A. The MVP slice is A + B. Each PR updates RuntimeReleaseCollectionsForwardRuleTests for the files it enrols.
Decisions
| # | Question | Status | Decision or recommendation |
|---|---|---|---|
| Q1 | Where do the reconciler and dispatcher run? | Answered 2026-10-23 (Chris) | The API hosts the release jobs as its own release-management module (one writer host, one fleet configuration). The writer code split is 6R.3 |
| Q2 | Whose clock sets ConfirmedAtUtc? |
Answered 2026-10-23 by Chris | The database clock ($$NOW); the host is recorded as confirmedBy |
| Q3 | Can a release be applied after unrelated policy changes? | Answered 2026-10-23 (Chris) | Yes, with a fresh reviewed Apply preview against the current policy |
| Q4 | Which announcements does an Apply queue before 6G has handlers? | Answered 2026-10-23 (Chris) | The registry's resolved communication defaults decide, with a per-release quiet override that needs a reason. It becomes a request field in 6D PR A (Announcement: Quiet plus a required QuietReason), replacing the guidance-based proposal |
| Q5 | Is a notice cancelled by a stop re-sent on restore? | Answered 2026-10-23 (Chris) | A notice cancelled by a stop stays cancelled; What's New returns with its original date |
| Q6 | How does Apply prove required guidance before 6G? | Answered 2026-10-23 (Chris) | The API reads the compiled server bundle from a configured read-only path and fails closed |
The "6D.0 registry communication fields" slice is replaced by 6R (R1b). Implementation of PR A waits on R1b, and ideally on R3b (§5A).
Not in scope: any shared-environment execution or RuntimeFeatureReleases:* change, 6G delivery, the availability endpoint (6E), targeting (6F), the release UI, an operator requeue for Failed rows, change-stream wake-ups, and feature definitions for historical features.
7. 6E–6H — availability, targeting, help and reader migration¶
| ID | Slices | Acceptance | Depends on |
|---|---|---|---|
| 6E availability | Availability endpoint (accepted plan §7.5): permission check first, then release and policy evaluation; discovery preference store (OD-14); discovery surfaces read availability only | A feature without the caller's permission is never reported available. Discovery hides it. RT-41 (API half: per-user dismissal; the rendering half is 6G) | 6D PRs A and B (merged), R1b (merged); detail in §7.1 |
| 6F targeting | PRs A–I and W1–W4 in §7.2: schema 2 reader and evaluator (A), project rules (B), project-context gates and availability (C), simulator (D), personal and combined rules (E), gradual rollout (F), consent (G), per-rule switches (H), targeted releases (I); web W1–W4 | RT-30 to RT-37 (§7.2) | A: 6E.2 (merged); see §7.2. E also needs the OD-6 subject-deletion answer (D34) |
| 6G help and What's New | Authoritative publication proof checked before disclosure; protected revisions, assets and search; rendering and sanitization; editorial dates (Europe/London); preferences and dismissal; RT-68 behaviour; RT-69 staging guide planned | RT-28, RT-29, RT-41 (rendering half; 6E owns the API half), RT-68. Forged historical-public claims, foreign audiences, old revisions and assets refuse before any disclosure or milestone | #4382, 6D, 6E (6F for audiences) |
| 6G.4 walkthroughs | In-app walkthroughs: our own M3 step-tour on the Angular CDK overlay. Steps live in walkthroughs.yaml (6R.1), anchored to data-tour-id attributes. Shown once when a feature becomes available; dismissal is remembered server-side; quiet releases skip it |
Keyboard and screen-reader accessible: Esc closes, focus is trapped and returned, reduced motion is respected | 6G, 6E, R1b |
| 6H reader migration | Public renderer, redirects, accessibility, link and content parity, retirement of the old reader | Link-parity report, automated accessibility checks, redirect map tests | 6G |
7.1 6E detail¶
Verified against main at 15bab4ba9. The full specification (contracts, files, line-level evidence) is the maintainer's
handover note spec-6E.md; this section holds what a reviewer needs. 6E is four PRs, A to D, each default-off or build-time.
What "available" means. A feature is available to a person when it has been released to them in this environment and they can use it now: they have the permission it needs, its capability is on, it is not stopped, and (while it is still Developing or Piloting) a release of it has been confirmed in this environment. It is a discovery and communication fact, not a gate: a tester switching a flag on does not create a release, and screens and routes keep reading the flag snapshot. Availability reads the same decisions as everything else, so there is no second evaluation path.
Permission first. Each feature declares the one existing authorization policy its use requires. Before any flag
decision or release lookup, the server drops every feature the caller may not use, using the existing policy handlers (never
a new check). A dropped feature is left out entirely, so the response is identical to one from a registry that never had the
feature, and counts, field shapes and reasons reveal nothing. The accepted plan's permission reason is therefore never sent.
The four slices
- 6E.1 Registry
permission(PR A).feature-definitions.yamlgains an optionalpermission:SignedIn, or the name of an existingProject*orApplication*policy (aStage*policy needs a stage and is refused). A Release must declare it; other types may omit it, and a feature without it is never reported.project-membership-disablegetsProjectEditMembershipsPolicy. The validator,pnpm feature:new --permission, the generated C# and TypeScript, and a test that every declared policy resolves in the API all carry it. The runtime catalog and its digest do not change. - 6E.2 Availability endpoint (PR B, MVP).
GET /api/features/availability(no project) andGET /api/projects/{projectId}/features/availability(admitted byProjectViewPolicy, so an absent or deleted project answers like every other project route). One evaluator applies permission, oneDecideAll, then the confirmed-release records for this environment and binding, and answersavailable,unavailableorunknownwith a reason (notReleased,stopped,staleAuthority,policyUnavailable). The response carries no flag keys, store epoch or digest (RT-60). Behind the existingRuntimeFeatureReleases:Enabledswitch (404 when off); production stays Legacy, where nothing Developing is available. - 6E.3 Discovery preference store (PR C).
PUT /api/account/announcements/{announcementId}/staterecords that one person has seen or dismissed one announcement (OD-14), one document per person and announcement. It always answers 204, so it never says whether a release exists. Availability returnsseenanddismissedfrom it. Same switch as 6E.2. - 6E.4 Browser availability client (PR D). A service with one
availabilitysignal per signed-in person and project that refetches on the existing triggers, drops late responses for a previous context, and renders nothing. Merged just before 6G's first consumer so it never sits unused.
Status (2026-10-23). PR A (#4450) is merged. PR B
(#4457) is merged: it implements 6E.2 with D25 option A (no project, no
Project* feature) and meets 6E.2-1 to 6E.2-9 in SyRF.API.Endpoint.Tests (FeatureAvailabilityHttpTests,
FeatureAvailabilityEvaluatorTests, FeatureAvailabilityContractTests, the endpoint catalogue) and
RuntimeReleaseStateReaderTests (replica set). Imported (#4447) and paused (#4453) records are document fixtures in
those PRs' shapes until they merge; its review follow-ups are #4459.
PR C (#4460) implements 6E.3 and meets 6E.3-1 to 6E.3-5 in
FeatureAvailabilityHttpTests, SupportImpersonationContractTests, SupportImpersonationHubAndClassificationTests and
RuntimeFeatureDiscoveryStoreTests (replica set). It follows D27 option A (account deletion keeps the rows; the OD-6
item carries the erasure follow-up); Chris chose A on 2026-10-23. A PUT writes only for an announcement availability reports now
(the latest released, announced record of a feature), so a superseded stage's announcement writes nothing. D has not
started.
Acceptance criteria
| # | Criterion | Verification |
|---|---|---|
| 6E.1-1 | A Release without permission, a Stage* policy and a misspelt policy each fail with an exact message; a valid entry passes |
tsx --test scripts/feature-registry.test.ts; FeatureDefinitionPermissionTests |
| 6E.1-2 | Generated C# and TypeScript carry the value; RuntimeFeatureFlagCatalog.generated.cs and the catalog digest pin are unchanged |
pnpm run validate:generated; RuntimeCatalogDigestPinTests |
| 6E.1-3 | feature:new --type Release without --permission refuses and writes nothing |
scripts/feature-commands.test.ts |
| 6E.2-1 | A project administrator gets available on the project route; a reviewer in that project, a non-member, and the administrator on the no-project route get no entry |
SyRF.API.Endpoint.Tests HTTP test (administrator is the positive control) |
| 6E.2-2 | The body for an unpermitted feature is byte-identical to the body from a registry without it; an absent and a deleted project answer like another ProjectViewPolicy route |
Same suite, fixed TimeProvider |
| 6E.2-3 | A dropped feature is never passed to DecideAll or the release reader |
Evaluator unit test with recording fakes |
| 6E.2-4 | Every row of the truth table gives its value and reason, including a stopped key with the record still Active, and an imported record of Unknown precision | FeatureAvailabilityEvaluatorTests (table-driven) |
| 6E.2-5 | A confirmed record from a foreign binding or environment leaves the feature notReleased; Legacy mode reports a Developing feature notReleased |
Replica-set test of the release-state reader, plus an HTTP test |
| 6E.2-6 | One DecideAll per request; each policy authorised once; at most one release query per 10 s per process under 50 concurrent requests |
Unit test with counters; reader test with a fake clock |
| 6E.2-7 | The golden pins exactly the response fields; no flag key, store epoch, digest or targeting appears | Golden test |
| 6E.2-8 | Switch off: 404 and zero store reads. On, off, on gives the same answers | HTTP test |
| 6E.2-9 | The routes are in the endpoint catalogue and pass the impersonation classification; an inactive user is refused like every authenticated route | EndpointAuthorizationCatalogTests; SupportImpersonationHubAndClassificationTests |
| 6E.3-1 | RT-41 API half: user A dismisses; A sees dismissed: true, user B false; a later joiner sees the same release date and seen: false |
HTTP test with two e2e users |
| 6E.3-2 | An unknown announcement, a quiet release and a foreign binding each return 204 and write nothing; a real one writes one row | HTTP test plus a collection count |
| 6E.3-3 | Concurrent identical PUTs leave one row with the first timestamps; a higher revision replaces, a lower one is ignored | Replica-set store test |
| 6E.3-4 | Read-only impersonation refuses the PUT; full impersonation writes the target's row | SupportImpersonationHubAndClassificationTests plus an HTTP case |
| 6E.3-5 | Rows written while on survive off and on | HTTP test |
| 6E.4-1 | A late response for the previous project is discarded; 404 gives an empty set; an error keeps same-context data and a context change clears it | pnpm exec ng test --no-watch --include="src/app/core/services/feature-availability.service.spec.ts" |
| 6E.4-2 | No extra request when no trigger fires; one fetch per accepted revision hint | Same spec, fake timers |
PR split
| PR | Content | Depends on | Parallel with |
|---|---|---|---|
| A (MVP), #4450 (merged) | 6E.1, and this section | main | 6D PRs C, E, F |
| B (MVP), #4457 (merged) | 6E.2 endpoint, evaluator, release-state reader, golden, swagger | A | 6D PRs C, E, F |
| C, #4460 | 6E.3 store and PUT; joins seen and dismissed into 6E.2 |
B | D |
| D | 6E.4 browser client; merges just before 6G's first consumer | B (C for the PUT) | C |
The MVP is A + B: on an isolated stack with RuntimeFeatureReleases:Enabled=true, a confirmed release of
project-membership-disable shows as available to a project administrator in that project and is absent for a reviewer, a
non-member and the no-project route. 6D PR D (stop recording) is not needed, because stops come from decisions.
Owner questions (programme decisions; Chris answered all three A on 2026-10-23)
| # | Question | Recommendation |
|---|---|---|
| D25 (Q1) | With no project open (the home page), should features that need a project permission count as available to someone who has it in at least one of their projects? Options: A no (hidden without a project; a page inside the project can show them); B yes (the server checks the person's active memberships, bounded and cached); C show them to every signed-in user | A for 6E. It is correct, cheap and fails closed. B can be added later without changing the contract if 6G wants project features on the home page. C breaks the permission rule |
| D26 (Q2) | The accepted plan also asks for a signed-out availability view. Do we still need it? Options: A drop it (signed-out readers see 6G's public What's New entries and the static feed); B build it for features marked public | A. Nothing signed-out needs a per-feature answer, and dropping it removes a second, anonymous contract |
| D27 (Q3) | When someone deletes their account, should their seen and dismissed records go too? Options: A keep today's behaviour (the records hold only a user ID and times) and erase them in the subject-deletion work OD-6 asks for before 6F; B delete them in the account-deletion path now (needs a {userId} index migration) |
A. It matches every other per-user store and needs no migration. Add a written follow-up to the OD-6 item. Awaiting Chris; #4460 follows A meanwhile and adds the follow-up to OD-6 in the accepted plan |
Operational prerequisites (written only). Enabling availability in an environment needs RuntimeFeatureReleases:Enabled=true
with RuntimeFeatureFlags:Mode=Shared and a bound environment; staging needs an owner decision (§8) and production stays
Legacy and switched off. There is no migration and no pre-deploy Job change.
Not in scope. Any shared-environment execution; UI of any kind (6G, 6G.4); targeting, audiences and cohorts (6F); home-page any-project evaluation (D25 B); the anonymous variant (D26 B); protected content and workflow versions (6G); account-deletion erasure (D27 B).
Each 6E–6H slice gets its own detailed plan section (contracts, files and tests) before implementation, added to this document in the PR that starts it.
7.2 6F detail¶
Verified against main at d63db13c1. The full specification (schema, validation rules, contracts and line-level
evidence) is the maintainer's handover note spec-6F.md; this section holds what a reviewer needs. Owner review of the
admin design canvas (2026-10-23) is binding except where a decision below is open.
What 6F adds. Rules that turn a feature on or off for one project, one person or one person in one project, a stable
gradual rollout, and (pending D29) consent. They live in the existing policy document as schema 2, written only when a rule
or feature setting exists; with none, the document is schema 1, byte-identical to today, so every older binary still reads
it. One evaluator decides, in the displayed order: requirements, stops, exclusions, specific allows (pair, then person,
then project), gradual rollout, the saved setting, the baseline. Steps before the allows only keep a feature off; allows and
rollouts only turn it on. Grains: Global keys take no rules; Project keys read project subjects only (every member gets
one decision); Personal keys read pair, person and project subjects. Production stays Legacy, and every non-Global key is
production-ineligible.
PR split (each default-off or build-time; main stays promotable)
| PR | Content | Depends on | Parallel with |
|---|---|---|---|
| A (MVP), #4463 | 6F.1 schema 2 reader, evaluator core, target context, reasons, host schema advert, report and checker additions. No writer | main | 6E C/D, 6D G |
| B (MVP), #4472 | 6F.2 project allow and exclude preview/commit, generator and registry grain checks, pilot contract (D33), constraint schema 1|2, RuntimeFeatureTargeting:Enabled switch |
A | none |
| C (MVP) | 6F.3 two-phase route gate, context factory, operation context; 6F.4 project-aware availability | B | D |
| D (MVP) | 6F.5 "Preview and check" simulator API | B | C |
| E | 6F.6 personal and combined rules | C; D31, D34 | F, G, H |
| F | 6F.8 gradual rollout and cross-language cohort vectors | B (C for rolloutPending) |
E, G, H |
| G | 6F.7 consent and environment settings | C (E for personal consent); D29, D30 | E, F |
| H | 6F.9 per-rule switches | B; D28 | E, F, G |
| I | 6F.10 targeted release intents | B, 6D A | E–H |
| W1–W4 | Browser gate, Targeting page, rollout and simulator UI, Beta features pages (M3, owner preview per screen) | C and 6E.4; B; D and F; G and W1 | each other as the spec lists |
Status (2026-10-24). PR A (#4463) is merged. PR B
(#4472, in review) implements 6F.2: POST …/{key}/targeting/preview and
PUT …/{key}/targeting through RuntimePolicyMutationStore (action Targeting, the existing receipt, replay and preview
patterns), save-time refusals as 422 runtime_targeting_refused, schema-2 admission against host acknowledgements, a
deployment constraint of policy schema 1 or 2, generator and registry check 9 grain rules, the pilot contract on
screeningKeywordHighlighting with the keyword-highlighting feature (pending D33, built to recommendation A), and the
RuntimeFeatureTargeting:Enabled switch (chart default false; bound on API and PM). It meets 6F.2-1 to 6F.2-10 in
RuntimePolicyTargetingMutationTests (replica set), RuntimeTargetingAdministrationIntegrationTests, the endpoint
catalogue and impersonation classification tests, and scripts/feature-registry.test.ts. It also closes the #4463 review
follow-up #4469: a deployment pin that fixes a value now decides on the
targeted path exactly as the context-free values do (RuntimeTargetingDeploymentPinParityTests), and stored rules survive
later policy changes and stops. With no rule saved, the pilot decides byte-identically to main in Legacy and Shadow mode;
Shared mode is the reviewed compatibility event recorded in the current-code matrix.
MVP boundary: A + B + C + D. On an isolated stack (Bramble, two API and two PM replicas, Mode=Shared, the switch on,
the pilot at default Off): an administrator previews and saves an allow for project P; P's gated route works for its
members and availability shows enabled: true; project Q answers 404 like an unrouted path; an exclusion with default On
turns it off for P only; the simulator explains both; removing every rule returns the document to schema 1; switching off
keeps exclusions and drops allows. Shortest critical path: A → B → C, with D beside C.
PR A acceptance (6F.1)
| # | Criterion | Verification |
|---|---|---|
| 6F.1-1 | A schema-1 document parses, digests and serializes byte-identically to main (golden taken from main before any change, plus a frozen copy of main's code over 200 generated policies); a schema-2 document round-trips; each validation rule refuses with its exact message, valid documents as controls |
RuntimePolicyAuthoritySchemaTests |
| 6F.1-2 | The same rules in any array order give one canonical document and one digest; empty targeting is refused under schema 2 and absent under schema 1 |
Same suite |
| 6F.1-3 | Every precedence row decides with its reason on synthetic Project and Personal contracts, including exclusion over allow, a switched-off rule skipped, rollout Off contributing nothing, default On turned off only by an exclusion, PrerequisiteMissing, and TargetingIncompatible |
RuntimeTargetingEvaluatorTests |
| 6F.1-4 | For 72 host, mode, environment and reader-state worlds, every catalog key decides as in a frozen copy of main's results, through DecideAll(keys), DecideAll(keys, None), DecideAll(keys, target) and Decide; the targeted Global path allocates nothing extra (warmed, minimum of long runs, 1 B epsilon, with an allocating positive control) |
Same suite |
| 6F.1-5 | A frozen schema-1-only reader reports a schema-2 document Unavailable and reads the schema-1 document written after the rules are removed as Ready; an acknowledgement row without the field parses as [1]; the checker does not pass a schema-1 candidate against a schema-2 report, refuses targeted-key-grain-changed and passes the schema-2 candidate |
SyRF.Mongo.Common.Tests; python3 scripts/test-runtime-compatibility-check.py |
| 6F.1-6 | The anonymous browser snapshot keeps schemaVersion: 1 and is identical with and without targeting (apart from the policy digest, which names the document) |
RuntimeTargetingBrowserEnvelopeTests golden |
Flag decision for A: not flagged. It reads a shape nothing writes yet, Global keys decide exactly as on main, and allows
and rollouts are never applied before PR B binds the switch (exclusions and stops can only keep a feature off).
"Global only" is enforced in six places (inaccuracy 3 below). PR A changes none of the refusals; it only reads and
decides targeting. They move in later PRs: the catalog generator's grain: Global check (done in B, which also keeps production eligibility
Global-only: non-Global stays production-ineligible, now refused by the generator too); stored and requested stops
(RuntimePolicyAuthority.ValidateStops, RuntimePolicyMutationStore.ValidateRequest) when a non-Global key first needs a
stop; release preparation (RuntimeReleaseIntentStore) in I; availability's non-Global row (FeatureAvailabilityEvaluator)
in C.
Open owner questions (programme decisions, awaiting Chris)
| # | Question | Recommendation |
|---|---|---|
| D28 (Q1) | Should each exclusion, allow and rollout rule have its own on/off switch? A: yes, switching is a reviewed change; B: no, delete and re-add | A. Keeps a rule's settings while paused; a rollout's switch is its "Turn off". PR A parses and honours enabled but writes nothing |
| D29 (Q2) | Should a feature declare whether people must agree to it (none, offered, required)? A: yes, a project's choice made by its administrators; B: no consent |
A. Leaving becomes an exclusion; only the person or project chooses, never support staff acting as them |
| D30 (Q3) | Who owns which setting? A: code owns what a feature is, each environment owns rules, rollout and stops and may only make settings stricter, with a reason; B: environments may loosen | A, building only the consent half in 6F; quieter announcements stay with D13 and 6G |
| D31 (Q4) | The API's user ID is the Investigator ID. Merge "user" and "investigator" into one level? A: one "person" level (pair, then person, then project); B: two levels | A. Two levels with one ID always agree. PR A builds A, marked pending in code |
| D32 (Q5) | What should Pause do, with no trusted creation time? A: hold the amount, and say new subjects inside the share still join; B: save the members as a list; C: add a creation time first | A; "Turn off" is the tool for stopping |
| D33 (Q6) | Which feature pilots project targeting? A: keyword highlighting (screeningKeywordHighlighting); B: membership disabling; C: a test-only flag |
A |
| D34 (Q7, OD-6) | Is subject deletion settled enough for personal targeting (opaque IDs only, inactive people never match and are listed, choices erased with D27's follow-up)? A: yes; B: also rewrite rules automatically on deletion | A. Automatic rewriting bypasses review. Answering unblocks E |
| D35 (Q8, changes OD-13 P-4) | Build a per-project emergency stop now? A: no, use a project exclusion or the environment stop, and add one with the first flag whose safe value is not Off; B: yes | A |
Corrections to the plan documents (applied in PR A, from the spec): README §6 and accepted §11.1 now use one person level (inaccuracy 1, D31) and say stops apply to every grain from the environment, with project stops deferred (2, D35); accepted §11.3 names the six enforcement points (3); README §6 rollout gains the algorithm ID as the first hash field, a fixed salt version 1, and says Pause cannot freeze a cohort (4, D32); RT-31 is proven with the shared evaluator on the PM host (5); the 6F rows above depend on 6E.2, not 6D (6); accepted §16 OD-6 is recorded against PR E (8); README §9 puts exclusions with project and personal rules (9); accepted §6.2, §7.1 and §7.5 note the schema-2 document, the compatibility additions and the availability fields (B3, B4). Inaccuracy 7 (§1, §2 and §7.1 status of #4457 and 6E.3) is corrected by #4460.
Operational prerequisites (written only). Using targeting needs Mode=Shared, a bound environment, the switch set to
the same value for API and PM, and every replica on a binary that advertises policy schema 2. Staging needs an owner
decision (§8); production stays Legacy and off. A rollback below a schema-2 binary while rules exist is refused by the
checker; the remedy is a forward revision that removes every rule, then the rollback. No migration and no pre-deploy Job
change.
Not in scope. Any shared-environment execution or RuntimeFeatureTargeting:* value; making a non-Global key
production-eligible; expressions or attribute rules; re-shuffling a cohort; project-level stops (D35 A); account-deletion
erasure (D27 follow-up).
7.3 6G detail¶
Verified against main at 21c73ed99 (6D PRs A–G and 6E.1–6E.3 merged). The full specification (contracts, files and
line-level evidence) is the maintainer's handover note spec-6G.md; this section holds what a reviewer needs. 6G covers
help and What's New for runtime releases, in-app walkthroughs (6G.4) and the release admin screens, which moved here from
wave 7 (2026-10-23). The admin screens follow the agreed design canvas.
What 6G adds. Feature-linked What's New text that the server discloses only to the people a feature is available to; help links that open the guide at the right section and show whether the feature is available; a walkthrough shown once when a feature becomes available; and admin screens to release, watch and settle features. It follows five rules:
- Content links to features through page and section metadata. What's New text names its
featureIdindocs/content/whats-new.yaml; guide pages name their topics in front matter (E1). Every human-written source stays YAML or Markdown. - One disclosure gate: availability. Personalised What's New entries travel inside the 6E.2 availability response,
per feature, only when that feature is
availableto the caller. There is no second evaluator or personalised endpoint. - The outbox decides what was announced. An entry shows once its
MaterialiseWhatsNewrow isDone, and "New to you" once itsNotifyInApprow isDoneby a non-test handler. Both production handlers are idempotent no-ops; there is no projection store. - Publication evidence (#4382). A public class counts only for a source in
user-guide/; a restricted entry's audience is its own feature until 6F; a Retired entry cannot claim it was public (D40). - Rendering. The server returns Markdown and the browser renders it only through
MarkdownPipeinuntrustedmode; the validator also refuses images, raw HTML and non-https, non-guide links. Date-only headings use Europe/London.
PR split (each default-off or build-time; main stays promotable)
| PR | Content | Depends on | Parallel with |
|---|---|---|---|
| A (MVP), merged #4466 | 6G.1 runtime What's New source, server bundle schema 2, compiler publication-evidence rules, WhatsNewEntryMissing, isoDate |
main | W1, AD1 |
| B (MVP), #4474 | 6G.2 What's New and in-app notice handlers, the announced-record fix, outbox join, bundle cache, availability fields (rehearsal, announcement.delivered, whatsNew); also #4473 (entry revision and bundle digest bound into the Apply digest and PolicyCommitted, one prepared-bundle pin rule, bounded bundle cache) |
A | W1, AD1, AD2 |
| C (MVP) | 6E.4 browser availability client (6E PR D), merged immediately before D | B | — |
| D (MVP) | 6G.3 home-page What's New and the project "New in this project" card | C | E1 |
| E1 | 6G.5 guide front-matter topics; release-content.yaml retired; help-topics.generated.ts |
A | B, C, D |
| E2 | 6G.5 help-topic link with an availability chip (RT-69 in-app half) | C, E1 | F |
| F | 6G.6 public What's New projection, signed-out merge, RT-69 rehearsal rows | B, D | E2 |
| G | 6G.7 server bundle built into the API image (D38) | A | all |
| H | 6G.8 protected help reader (D39: only with the first private-pilot article) | E1, C | — |
| W1 | 6G.4 walkthroughs.yaml, generator, validator, data-tour-id anchors |
main | A, AD1 |
| W2 | 6G.4 tour component and launcher | C, W1 | E2, F |
| AD1 | 6G.10 admin shell /admin/features (side menu Overview, Features, Flags, Releases, Environment, Content; feature tabs Overview, Targeting, Settings, History) and the split capabilities |
main | A, B |
| AD2 | 6G.11 Releases screen and its read routes (list, events, announcement rows) | AD1 | B |
| AD3 | 6G.12 Ready to settle: an Overview attention item, a Features filter and status, and the lifecycle state on the feature page (not a menu item) | AD1 | AD2 |
| AD4 | 6G.13 Content preview: per feature and audience, the What's New entry, guide section and walkthrough that audience would get (read-only) | A, W1, AD1 | AD2, AD3 |
flowchart LR
A[A content] --> B[B disclosure + handlers]
B --> C[C 6E.4 client]
C --> D[D What's New screens]
A --> E1[E1 page metadata]
E1 --> E2[E2 help link chip]
C --> E2
B --> F[F public projection]
D --> F
A --> G[G bundle in image]
E1 --> H[H protected reader]
W1[W1 walkthrough source] --> W2[W2 tour component]
C --> W2
AD1[AD1 admin shell] --> AD2[AD2 Releases]
AD1 --> AD3[AD3 Ready to settle]
AD1 --> AD4[AD4 Content preview]
W1 --> AD4
A --> AD4
AD1 --> W2T[6F W2 Targeting page]
MVP boundary: A + B + C + D. On an isolated Bramble stack (Shared mode, RuntimeFeatureReleases:Enabled=true,
dispatch on, a production-delivery binding fixture for the notice, a rehearsal-only registry entry), an applied and
confirmed release of project-membership-disable shows a sanitised "New in this project" entry, with its Europe/London
date and "New to you", to a project administrator in that project. A reviewer, a non-member and the home page show
nothing; "Hide" removes it for that administrator only; a stop hides it and a restore brings it back with the same date
and no "New to you" for people who had not seen it (D14). Shortest critical path: A → B → C → D.
PR A acceptance (6G.1)
| # | Criterion | Verification |
|---|---|---|
| 6G.1-1 | whats-new.json differs from main only by isoDate; no runtime entry, featureId or runtime text appears in it or in the public bundle (byte scan for each runtime title) |
pnpm run validate:generated; compiler test |
| 6G.1-2 | A public-class entry outside user-guide/, a restricted entry with a foreign audienceId, a Retired entry with a public historical class, an unparseable static date, a runtime entry with an image, raw HTML or an http: link, and a runtime entry for a quiet feature each fail with an exact message; the membership entry passes (positive control) |
tsx --test for the compiler, generator and validator tests |
| 6G.1-3 | C# and TypeScript agree on a schema 2 golden digest and a tampered byte fails; schema 1 bundles still verify | RuntimeReleaseGuidanceBundleTests |
| 6G.1-4 | The Apply preview refuses WhatsNewEntryMissing with no write; the same release with Announcement: Quiet and a reason is admitted; an emergency stop never reads the bundle |
RuntimeReleaseGuidanceAdmissionTests; RuntimeReleaseIntentAdministrationIntegrationTests |
Flag decision for A: not flagged. It changes build-time sources and generation, and adds a stricter admission inside the
existing default-off RuntimeFeatureReleases:Enabled Apply. There is no migration and no start-up work. Release records
prepared before it merges need preparing again only if their definition changed; none exist outside isolated stacks.
Status (2026-10-23). PR A is #4466. B to AD4 have not started.
Owner questions (open, awaiting Chris)
| # | Question | Options | Recommendation |
|---|---|---|---|
| D36 | D25 A hides project-permission features when no project is open, so the first real release (disabling memberships, for project administrators) would never appear in the home-page What's New. Where should it appear? | A: a "New in this project" card on the project overview, fed by that project's availability; the home page shows features that need no project. B: reopen D25 and evaluate the person's projects on the home page (bounded, cached). C: home page only, accepting that project features never show there | A. It shows the news where the feature is used, keeps D25 and adds no server work |
| D37 | The accepted plan shows "Planned" for scheduled releases (RT-29), which tells readers about a feature before it is released. Keep it? | A: drop it: What's New shows only confirmed releases; schedules stay on the admin Releases screen. B: show "Planned" to people who would be permitted to use the feature | A. It removes a surface and keeps What's New from claiming an unconfirmed release |
| D38 | How does the API get the compiled help bundle? Today it would be a Kubernetes Secret, capped at 1 MiB, needing a cluster-gitops change for every content edit | A: build it into the API image (CI compiles it; the Secret mount goes). B: keep the Secret (1 MiB limit). C: a ConfigMap (same limit) | A. No size cap, no secret handling and no per-edit GitOps change; the cost is an API image build when guide content changes |
| D39 | Build the protected in-app help reader (articles, server search, assets) now, or when needed? No private-pilot article exists and the first release's guide is public | A: plan it (6G.8) and build it with the first private-pilot article; until then public topics open the public guide, with the availability chip in the app. B: build it now | A. Nothing would use it yet; RT-28's API half moves with it (C4) |
| D40 | A retired page may claim it "was public", and nothing can prove that today (#4382: a declaration is not proof) | A: refuse such claims everywhere in 6G; 6H adds a publication record if retired archives are needed. B: record now which public revisions each environment has served | A. No retired content exists; B adds a collection for no current use. PR A builds A |
| D41 | Walkthrough text ships in the web bundle, so anyone can read it in the JavaScript. Acceptable? | A: yes: it describes UI that ships in the same bundle; the validator refuses a walkthrough for a feature whose topics are all restricted. B: serve it from the server bundle through availability | A. It needs no API and exposes nothing the shipped UI code does not |
| D42 | What is the in-app notice for a release? | A: the "New to you" label on the What's New entry (and the walkthrough, when there is one); no extra banner. B: also a dismissible banner where the feature is used. C: a notification centre | A. One surface fewer; B or C can follow if people miss announcements |
| D43 | "Ready to settle": the plan names each flag's audit history as an interim source beside release records | A: release records only (historical import covers features released before 6D); features without a GeneralRelease record are not listed. B: also read flag audit history | A. Only Release-type features are listed and each has, or can import, a record; B adds a second source to keep in step |
Amendments to the accepted plan (recorded here rather than reopened)
- C1 (§12.3, §7.4): the What's New projection is derived at read time from confirmed records, the current bundle and
live decisions, and shown once its
MaterialiseWhatsNewrow isDone; there is no projection store. Personalised entries travel in the availability response (6E.2 gainsrehearsal,announcement.deliveredandwhatsNew). - C2 (§12.1, §12.2, #4382): a public class is evidenced by the source living in
user-guide/; a restrictedaudienceIdmust equal itsfeatureIduntil 6F; Retired historical-public claims are refused (D40). Built in PR A. - C3 (RT-28): unpermitted, foreign-audience and old-revision requests answer the same 404 as an unknown topic, not the 403 RT-28 names, so they reveal nothing (6E's no-leak rule).
- C4 (§7): 6G acceptance becomes RT-29 (without "Planned" if D37 A), RT-41's rendering half, RT-68 and RT-69's in-app half; RT-28's API half moves with 6G.8 (D39). The 6G row's "#4382" dependency is a merged PR, not open work.
- C5 (§5A 6R.1):
release-content.yamlis replaced by page front matter (E1);whats-new.yamlgainsruntime:entries (PR A). - C6 (§5A "Admin Ready to settle", wave 7 Releases screen): delivered as 6G.11 and 6G.12 in the 6G.10 shell; release records are the only source (D43).
- C7 (production prerequisite P12a): the bundle ships in the API image (D38), not a Secret.
Operational prerequisites (written only). Using 6G in an environment needs RuntimeFeatureReleases:Enabled=true and
RuntimeFeatureReleases:OutboxDispatch:Enabled=true with RuntimeFeatureFlags:Mode=Shared and a bound environment;
staging needs an owner decision (§8) and production stays Legacy and off. With D38 A no chart value is needed for the
bundle. No migration and no pre-deploy Job change.
Not in scope. Any staging or production execution and any RuntimeFeatureReleases:* or cluster-gitops change; the
public renderer migration, redirects and retirement of the Jekyll reader (6H); a wider help-guide overhaul (accepted plan
§17); targeted audiences and per-audience guidance (6F); re-notification revisions above 1 (§12.4); a notification centre
or email; a signed-in landing page; "Planned" entries (if D37 A); publication records for retired content (D40 B); the
protected reader before its trigger (D39 A); an operator requeue for Failed outbox rows and the refusal of Apply under a
stop (#4456); cross-environment administration (D24's later option).
8. Parallel lanes and owner gates¶
- The wave 2–5 chain (#4344–#4385) has merged. Parallel lanes now: the open 6B.4 PRs, 6D PRs C and F, and 6E, each in its own worktree. Delegated work goes to Sonnet or Haiku only with a written specification, and every result is reviewed (diff read and checks rerun) before it is pushed.
- Owner gates (stop and ask): any staging execution of binding, import, enrolment or activation; any production step; any destructive migration or compensation; any change to infrastructure identities or secrets.
- Open owner decisions gating planned work (tracked on the programme page and in the programme ledger): none for 6E: D25 to D27 were answered A on 2026-10-23 (§7.1). Programme D15 and D16 were answered A on 2026-10-23, and 6B.4 PR 4 (#4449) has merged.
- Approved architecture decisions (2026-10-23): a unified Features and releases admin area, built in steps (proposed design canvas, temporary review material pending Chris's review, not a maintained reference: https://claude.ai/artifact/EKxMc2Xq9ihkCM6uwTNkdC); walkthroughs as 6G.4 with our own M3 CDK component; no paid or closed-source third-party tools, so targeting (6F) is built in-house.
- Cross-environment administration (D24, Chris, 2026-10-23): now, each environment's own admin page, linked to the others; later, possibly one admin web app calling each environment's API with the person's own sign-in for it (each environment's preview, confirm and audit still apply). A central service holding write credentials is ruled out. A per-environment flag-state metrics export with a drift dashboard is proposed and owner-gated.
9. Cross-service propagation evidence¶
The criteria recorded in the 20 October handover apply to every wave:
- flag-to-consumer inventory;
- independently reconciled workload, replica and generation inventory;
- fresh exact-authority acknowledgements;
- behaviour assertions;
- disruption, restart, scaling and mixed-version tests;
- measured bounds.
Wave 2 covers API membership, PM statistics serving, pages, stage overview and SignalR, and the browser runtime snapshot, with an in-process multi-instance harness. 6B adds separate processes, a browser and measured timings. Independent workload observation comes from wave 3 (#4363, #4370–#4372). Deployment-specific readiness remains a separate later gate.