Working with Generated Files

This guide explains how to work with generated files in the SyRF monorepo, including the verification system that ensures generated files stay in sync with their sources.

Overview

The SyRF monorepo contains 30 checksummed generated files produced by 6 generators (the GENERATOR_COMMANDS list in scripts/generated-manifest-guard.ts). pnpm generate runs all of them in this order:

Generator	Source	Output Files
generate-env-blocks.ts	env-mapping.yaml	8 files (Helm template, config TS/JSON, config-keys.generated.json)
generate-feature-flags.ts	env-mapping.yaml	4 files (feature flag TS files)
generate-runtime-flag-catalog.ts	env-mapping.yaml	2 files (C# runtime flag catalog and key constants)
generate-feature-definitions.ts	feature-definitions.yaml, docs/content/whats-new.yaml	6 files (C# and TS feature definitions, C# flag groups and deployment-managed pins, the What's New feed whats-new.json, the inventory page)
generate-dockerfiles.py	dependency-map.yaml	9 Dockerfiles
NSwag	swagger.json	1 file (TypeScript API client); pnpm generate runs it only when swagger.json changed

Two reviewed YAML sources have no checksummed output: docs/content/release-content.yaml is read only by the offline content compiler (pnpm run compile:release-content), and env-mapping.schema.json is a schema. The feature-registry validator checks both (see below).

How Verification Works

The system uses a hash-manifest approach stored in .generated-checksums.json:

1. sourceHash - SHA-256 of source file(s), detects when sources change
2. outputHash - SHA-256 of each generated file, detects manual edits
3. generatorHash - SHA-256 of generator script (warning only)

When you commit, the precommit hook validates:

• Source files haven't changed without regenerating outputs
• Generated files haven't been manually edited
• The feature registry's cross-links hold (scripts/feature-registry.ts): flag keys in the runtime catalog, Required guidance has content and its user-guide pages exist, release-content featureId/topicId and What's New userGuidePath/featureId resolve, flagGroups keep their members (against HEAD locally and the merge ref's base in CI), Settled and Withdrawn features have no live flag, env-mapping.yaml and whats-new.json match their schemas, and every source it reads is covered by the CI and pre-commit trigger patterns
• The C# feature-flag consumer inventory is complete (scripts/feature-flag-consumers.ts)

Quick Reference

One command

# Every generator in dependency order (NSwag only when swagger.json changed), then the registry validator.
# On a clean tree it changes nothing.
pnpm generate

# Only some generators, still in order
pnpm generate generate-env-blocks generate-feature-flags

# Add a feature and its flag, or settle a released one (see Managing Feature Flags)
pnpm feature:new <feature-id> --type <type> --owner "<team>" [--permission <SignedIn|Project…Policy|Application…Policy>]
# --permission is required for --type Release
pnpm feature:settle <feature-id>

scripts/generate-all.test.ts (in validate:generated) fails if a generator is added to GENERATOR_COMMANDS without an order slot in scripts/generate-all.ts. The slow checks that run the real generators and feature commands in a temporary copy of the repository files (pnpm generate is a no-op on a clean tree; feature:new and feature:settle end to end) are scripts/tooling-e2e.test.ts: pnpm run test:tooling-e2e. CI runs them after validate:generated in both Validate Generated Code jobs; the pre-commit hook runs them only when scripts/feature-commands*, scripts/generate-all*, scripts/tooling-e2e.test.ts or scripts/temp-repo-copy.ts change. nswag:spec stays separate, because it needs a .NET Release build.

Validation Commands

# Validate all generated files
pnpm run validate:generated

# Registry cross-links and schemas only (fast; names the file and entry)
pnpm run validate:registry

# Run individual generator validators
cd src/charts/syrf-common && pnpm run validate:env-blocks
cd src/services/web && pnpm run validate:flags
python3 scripts/generate-dockerfiles.py --validate

Regeneration Commands

# Regenerate config files (8 files)
cd src/charts/syrf-common && pnpm run generate:env-blocks

# Regenerate feature flags (4 files)
cd src/services/web && pnpm run generate:flags

# Regenerate the C# runtime flag catalog and key constants (2 files)
cd src/charts/syrf-common && pnpm run generate:runtime-catalog

# Regenerate feature definitions, flag groups, pins and the What's New feed (5 files)
cd src/charts/syrf-common && pnpm run generate:feature-definitions

# Regenerate Dockerfiles (9 files)
python3 scripts/generate-dockerfiles.py

# Regenerate API client (1 file) - requires swagger.json
cd src/services/web && pnpm run nswag:generate

# Regenerate swagger.json from API assembly (requires API build first)
dotnet build src/services/api/SyRF.API.Endpoint -c Release
cd src/services/web && pnpm run nswag:spec

Developer Workflows

Valid configuration keys (config-keys.generated.json)

generate:env-blocks also writes src/charts/syrf-common/config-keys.generated.json: for each service (api, project-management, quartz, identity, web) the Helm valuePaths and the raw envNames (SYRF__...) that env-mapping.yaml defines, plus schemaVersion and a content digest. It is deterministic (sorted, no timestamps) and ships inside the packaged syrf-common chart, so it is also inside every service chart that depends on it (charts/syrf-common/config-keys.generated.json) and at that path in the git ref a service chart is pinned to. No workflow change is needed.

It is the published list of keys a chart version reads. Environment maintenance belongs to cluster-gitops (decision D22), so the check lives there and SyRF never reads that repository: cluster-gitops compares the keys in its own values files with services.<service>.valuePaths of the chart version it pins, and reports unknown keys (dead or misspelt) as clean-up work. Compare full dotted key paths. valuePaths lists the leaf paths a chart reads, and a parent path only where the chart reads the parent itself (for example emailTransport, a section condition). So treat a values key as known when it equals a listed path, or when it is a dotted-segment prefix of one (emailTransport matches emailTransport.smtp.host, but email does not): a parent object that contains listed leaves. Keys read only inside hand-written helper templates are not listed (see the file's coverage note), so report unknowns for review rather than failing on them blindly.

Modifying env-mapping.yaml

To add a feature flag, use pnpm feature:new (it edits the YAML and regenerates). For any other change to environment variables or flags:

# 1. Edit the source
vim src/charts/syrf-common/env-mapping.yaml

# 2. Regenerate (env blocks, web flags, runtime catalog and key constants, feature definitions all read it)
pnpm generate

# 3. Review and commit all changes
git add -A
git commit -m "feat: add new config option"

Modifying the What's New feed or release content

Both are reviewed YAML under docs/content/:

• whats-new.yaml → pnpm --filter syrf-common-chart generate:feature-definitions regenerates src/services/web/src/assets/data/whats-new.json (the file the web app fetches). See Update the What's New feed.
• release-content.yaml has no generated output; pnpm run validate:generated checks its references and pnpm run compile:release-content docs/content/release-content.yaml user-guide <server-dir> <public-dir> compiles it. See Managing Feature Flags.

Then run pnpm run validate:generated and commit the source with its regenerated outputs and checksums.

Modifying dependency-map.yaml

When changing service dependencies:

# 1. Edit the source
vim docs/architecture/dependency-map.yaml

# 2. Regenerate Dockerfiles
python3 scripts/generate-dockerfiles.py

# 3. Review and commit
git add -A
git commit -m "chore: update service dependencies"

Modifying API Endpoints or DTOs

When changing API controllers, DTOs, or endpoints, building the API automatically regenerates both swagger.json and the TypeScript client:

# 1. Make API changes
vim src/services/api/SyRF.API.Endpoint/Controllers/...

# 2. Build API in Release mode — swagger.json, TypeScript client, and checksums are regenerated automatically
dotnet build src/services/api/SyRF.API.Endpoint -c Release

# 3. Review and commit
git add -A
git commit -m "feat(api): add new endpoint"

Automatic generation details:

• The API .csproj has an AfterBuild target that runs NSwag spec generation followed by TypeScript client generation
• Only runs on Release builds (-c Release) — Debug builds skip generation for faster iteration
• TypeScript client generation requires pnpm install in src/services/web/ (warns and skips if missing)
• Disable with dotnet build -c Release /p:GenerateOpenApi=false if you want a Release build without regeneration

Manual generation (if you need to run steps individually):

# Generate swagger.json only (from compiled API assembly)
cd src/services/web && pnpm run nswag:spec

# Generate TypeScript client only (from existing swagger.json)
cd src/services/web && pnpm run nswag:generate

# Generate both spec + client + checksums in one command
cd src/services/web && pnpm run nswag:all

Note: The swagger.json file is committed to the repository as the source of truth for the API client generation. This allows the TypeScript client to be regenerated without running the API.

If Precommit Fails

If you see a precommit failure like:

❌ Generated file validation failed!

Error: Source changed for "generate-env-blocks"
  Sources: src/charts/syrf-common/env-mapping.yaml
  Fix: cd src/charts/syrf-common && pnpm run generate:env-blocks

Follow the fix command shown (or run pnpm generate), then stage and commit again:

# Run the suggested command
cd src/charts/syrf-common && pnpm run generate:env-blocks

# Stage the regenerated files
git add -A

# Commit again
git commit -m "your message"

Generated Files Reference

From env-mapping.yaml (generate-env-blocks)

File	Purpose
src/charts/syrf-common/templates/_env-blocks.tpl	Helm template for env vars
src/services/web/src/assets/data/env.template.generated.json	Runtime env template
src/services/web/src/assets/data/appConfig.default.generated.json	Default config values
src/services/web/src/app/core/services/config/external-config.generated.ts	External config interface
src/services/web/src/app/core/services/config/app-config.generated.ts	App config interface
src/services/web/src/app/core/services/config/feature-flags.generated.ts	Feature flags interface
src/services/web/src/app/core/services/config/load-config.generated.ts	Config loader function
src/charts/syrf-common/config-keys.generated.json	Valid configuration keys per service (see Valid configuration keys)

From env-mapping.yaml (generate-feature-flags)

File	Purpose
src/services/web/src/app/generated/feature-flags.constants.ts	Flag defaults and mappings
src/services/web/src/app/generated/feature-flags.types.ts	TypeScript types
src/services/web/src/app/generated/feature-flags.parser.ts	Config parser
src/services/web/src/app/generated/feature-flags.selectors.ts	ngrx selectors

From dependency-map.yaml (generate-dockerfiles)

File	Purpose
src/services/api/SyRF.API.Endpoint/Dockerfile	API service Docker build
src/services/identity/SyRF.Identity.Endpoint/Dockerfile	Identity service Docker build
src/services/project-management/SyRF.ProjectManagement.Endpoint/Dockerfile	PM service Docker build
src/services/quartz/SyRF.Quartz/Dockerfile	Quartz service Docker build
src/services/s3-notifier/SyRF.S3FileSavedNotifier.Endpoint/Dockerfile	S3 Notifier Docker build
src/services/web/Dockerfile	Web frontend Docker build
src/services/web/Dockerfile.full	Web frontend full Docker build
user-guide/Dockerfile	User guide Docker build
docs/Dockerfile	MkDocs site Docker build

From env-mapping.yaml (generate-runtime-flag-catalog)

File	Purpose
src/libs/mongo/SyRF.Mongo.Common/RuntimeFeatureFlags/RuntimeFeatureFlagCatalog.generated.cs	Shared C# runtime flag catalog
src/libs/mongo/SyRF.Mongo.Common/RuntimeFeatureFlags/RuntimeFeatureFlagKeys.generated.cs	RuntimeFeatureFlagKeys: one constant per catalogued key (name = the key in PascalCase), and RemovedFlags, the removed flags from removedFlags in env-mapping.yaml

From feature-definitions.yaml and docs/content/whats-new.yaml (generate-feature-definitions)

File	Purpose
src/libs/mongo/SyRF.Mongo.Common/RuntimeFeatureReleases/FeatureDefinitions.generated.cs	C# feature definitions
src/services/web/src/app/generated/feature-definitions.generated.ts	TypeScript feature definitions
src/libs/mongo/SyRF.Mongo.Common/RuntimeFeatureFlags/RuntimeFeatureFlagGroups.generated.cs	Preset flag groups (flagGroups): advertised groups in file order, plus retained IDs only Find resolves
src/libs/mongo/SyRF.Mongo.Common/RuntimeFeatureFlags/RuntimeFeatureFlagPins.generated.cs	Deployment-managed pins (deploymentManaged) with their constraint and refusal texts
src/services/web/src/assets/data/whats-new.json	Home-page What's New feed (runtime web asset; featureId stripped, $schema first)
docs/features/feature-inventory.md	Feature inventory: summary, complexity metrics, features table, unclaimed and removed flags. Also reads the consumer manifest, GENERATOR_COMMANDS; deterministic, no dates

The feature definitions carry values resolved from each feature's type (communication defaults and settle period), so no reader applies a default. Groups and pins replace the hand-written C# that RuntimeFeatureFlagGroups.cs and ProjectStatisticsRuntimeActivation.PinnedKeys held; the hand-written classes are now thin facades over the generated data. Fields and defaults: Feature registry (schema 2).

From swagger.json (NSwag)

File	Purpose
src/services/web/src/app/core/services/api-client.generated.ts	TypeScript API client

Source files:

• src/services/web/swagger.json - OpenAPI specification (generated from API assembly, committed to repo)
• src/services/web/syrf-local.nswag - NSwag client generation config
• src/services/web/syrf-generate-spec.nswag - NSwag spec generation from assembly

Troubleshooting

"Generator script changed" Warning

This is informational only. It means the generator script was modified but outputs haven't been regenerated. You can safely ignore this or regenerate to update the checksums.

Manual Edit Detected

If you see "Output file manually edited", it means someone edited a generated file directly. Run the appropriate generator to restore the correct content.

Missing Checksums Manifest

If .generated-checksums.json is missing:

pnpm exec tsx scripts/init-checksums.ts

This initializes the manifest with current file hashes.

CI Integration

The validation runs automatically on PRs when generated files or their sources are modified. The job is defined in .github/workflows/pr-tests.yml as validate-generated-code; its trigger is the env_mapping regex in the detect-changes job. The registry validator's check 6 fails if a source it reads (the docs/content/*.yaml files, the What's New feed and schema, env-mapping.schema.json, user-guide/ pages) stops matching that regex or the pre-commit GENERATED_PATTERN.

Architecture

┌──────────────────────┐
│  Source Files        │
│  - env-mapping.yaml  │
│  - dependency-map    │
│  - swagger.json      │
└─────────┬────────────┘
          │
          ▼
┌──────────────────────┐     ┌──────────────────────┐
│  Generator Scripts   │────▶│  Generated Files     │
│  - generate-*.ts     │     │  - *.generated.ts    │
│  - generate-*.py     │     │  - Dockerfiles       │
│  - NSwag             │     │  - api-client.ts     │
└─────────┬────────────┘     └──────────────────────┘
          │
          ▼
┌─────────────────────────────────────────┐
│  .generated-checksums.json              │
│  - sourceHash (detects source changes)  │
│  - outputHash (detects manual edits)    │
│  - generatorHash (informational)        │
└─────────────────────────────────────────┘
          │
          ▼
┌─────────────────────┐
│  Validation         │
│  - Precommit hook   │
│  - CI workflow      │
└─────────────────────┘

See Also

• env-mapping.yaml - Environment variable definitions
• dependency-map.yaml - Service dependencies
• validate-generated.ts - Validation script
• feature-registry.ts - Feature registry cross-link validator