Scene Protocol

A scene file is not a configuration file. It is the wire format between a composing agent and the compiler — the input half of scorekit’s machine contract, exactly as the Machine Interface is the invocation half. An agent that emits a valid scene document is speaking a protocol: every field has one deterministic, observable compile semantic, the format is versioned under the same semver as the binary, and violations are rejected loudly with machine-readable positions, never silently ignored.

This chapter is the normative specification of that protocol. The live, machine-readable source of truth is always scorekit schema (JSON Schema); when this prose and the binary disagree, the binary wins.

Why a protocol, not just a format

Five properties separate a protocol from a mere file format, and the scene DSL commits to all of them:

1. Strictness. Unknown fields are rejected (deny_unknown_fields at every level). A typo, or a scene written for a newer scorekit, fails with exit 2 and a line/column — it is never silently dropped. Silent tolerance is how formats rot; loud rejection is how protocols stay honest.
2. Deterministic semantics. The same scene + the same scorekit version compiles to byte-identical MIDI, always. There is no “interpretation”, no randomness outside the seeded performance.humanize, and no field whose effect depends on the machine it runs on.
3. No dead fields. Every field must have a deterministic compile semantic. Mood tags, game-state hints, and creative intent (avoid: too_bright) are by design not part of the protocol — they belong in the agent’s prompt space. A field that “doesn’t do anything” is a protocol violation waiting to be invented twice.
4. Machine discoverability. scorekit schema exports the full JSON Schema, including field docs, ranges, and defaults. An agent can learn to write valid scenes from the schema plus one error message, without reading source code.
5. Text-native. UTF-8 YAML, line-oriented, diff-friendly. Version control, review, merge, and rollback are git’s job; the protocol’s job is to make diffs mean something (see scorekit diff for semantic comparison that ignores formatting).

Envelope

• One UTF-8 YAML document per file; one document = one scene (or one suite, when sections is present).
• Field names are snake_case. Unknown fields anywhere are an error.
• Validation is two-layered, and both layers are part of the contract:
  • Structural — shape and types, as exported by scorekit schema. Parse errors report location: {line, column}.
  • Semantic — ranges, cross-field rules (e.g. motif required iff pattern: melody), reference checks. Violations report a field path like tracks[2].glide.

Versioning and stability

The scene protocol is versioned by the scorekit binary’s semantic version. There is deliberately no version field inside the scene file: strict unknown-field rejection already makes version skew detectable at the only moment it matters (compile time), and a self-declared version number that the compiler must trust would be weaker than one it derives from its own schema.

Within a major version:

• Additive only. New fields are optional with defaults that preserve prior output — a scene that does not use a new field compiles to byte-identical MIDI before and after the addition. (Precedent: when pan/reverb/glide landed in 0.2.0, the golden SMF byte-comparison test did not change.) Additional compile targets such as scorekit makecode must not add schema fields that only that target can fulfill.
• No repurposing. An existing field’s name, type, range, default, or compile semantic does not change. The normative transform order (below) is part of the semantic and is equally frozen.
• Old scene, new binary: always valid, same MIDI bytes.
• New scene, old binary: rejected with exit 2 and the offending field path — a readable upgrade signal, not a corrupted asset.

Anything that breaks these rules is a major-version change. Determinism across tool versions (FluidSynth/FFmpeg) is a separate boundary owned by the Machine Interface: pin your toolchain for identical audio; MIDI needs only the scorekit version pinned.

Rules for extending the protocol

A proposed field must clear all of these gates (this is the governance encoded in the project’s iron rules):

• It has exactly one deterministic compile semantic, statable in one sentence (“compiles to X”).
• It is achievable with free/open sound sources — the schema must not contain fields only commercial libraries can honor.
• Its absence compiles byte-identically to the world before the field existed.
• It ships in the same change as at least one happy-path E2E test, one failure-path test if validation is involved, and an acceptance-matrix row.

Scene document

Track

Because all percussion tracks share one MIDI channel, their optional pan and reverb values must compile to the same CC10/CC91 state. Conflicting values are rejected with the later track’s exact field path rather than silently letting one track overwrite another.

Event clip

Clips are the exact-event escape hatch for arrangements whose identity lives in syncopation, fills, bass switches, or controller gestures rather than a generative pattern. They remain source-neutral MIDI structure: a clip never names a synth, plugin parameter, sample, or local path.

A looping clip’s quantized length must divide every scene/section timeline in which it is active. Events cannot cross the clip boundary. Pitched events with the same MIDI key cannot overlap; percussion events cannot repeat the same voice at the same quantized tick. After loop expansion, each active track may produce at most 65,536 note/control events per scene or section, including samples generated by linear automation; validation rejects larger expansions before composition allocates them.

Clip event

Clip velocity is still scaled by track/section intensity and performance.dynamics. Authored onset, duration, and velocity relationships are otherwise exact: swing, legato, and humanize do not rewrite clip events.

Automation

Each lane chooses one portable MIDI target: cc1, cc11, cc74, or pitch_bend, plus interpolation: step | linear (default step). Its points map uses stable IDs and exact {at, value} entries. CC values are 0..=127; pitch bend is -8192..=8191 around center 0. Every lane starts at beat 0. A loop lane’s final authored value must equal its initial value, making the repeated controller state explicit and seam-safe. Step mode emits only authored points. Linear mode emits each authored point plus deterministic rounded linear samples every 60 ticks (PPQ/8, a 32nd note), anchored at each segment’s left point.

SF2/TiMidity receive these standard MIDI events directly. For sfizz, the effective leaf renderer-profile mapping must explicitly declare every target under controls; otherwise orchestration resolution fails before build staging. This prevents a talking-bass scene from silently rendering through a patch that ignores its authored motion.

Texture track

Texture tracks are score-timeline material: field recordings, ambience, and sound effects used as part of the composition. They do not model runtime world audio such as distance attenuation, weather state, or engine RPM.

Beat positions are quantized to the nearest PPQ-480 tick, then converted to sample frames using the same quantized MIDI tempo as musical tracks. Loop textures run continuously across the two render passes; the normal loop seal therefore joins adjacent source frames even when the source duration does not divide the scene duration. The recording itself should be prepared as a loop-ready asset because scorekit does not conceal clicks inside the source. For a looping scene, a one-shot source must not be longer than one complete scene pass; this guarantees that one prior pass contains all tail material needed at the loop boundary.

Motif note

Section

Sections inherit the scene’s key, tracks (including spatial fields), textures, motifs, harmony, and performance. A shared texture schedule must be valid for every section: each trigger is checked against the shortest derived section timeline, preventing a trigger from silently wrapping into a later pass of a shorter cue.

Performance

All optional, all deterministic:

Normative compile semantics

These are observable guarantees an integration may rely on; changing any of them is a breaking protocol change.

Time and encoding. Output is SMF format 1, PPQ 480. A bar is N × (480·4/D) ticks. Clip positions always use quarter-note beats regardless of the time-signature denominator and quantize with round(beats × 480). Events are encoded in the canonical order (tick, kind rank: note-off < automation CC < pitch-bend < note-on, key/value) — this ordering is the byte-determinism guarantee.

Channels and programs. Melodic tracks take channels in declaration order (0, 1, 2, …), skipping channel 10 (index 9), which is shared by every percussion track (drums or tabla). Those tracks remain separate SMF tracks and stems. An exact GM identity opens with its program change. Internal MIDI rendered against an exact sfizz patch leaves a profile-only melodic channel programless; standalone scorekit midi rejects that case before writing a file rather than publish MIDI that generic players interpret as program 0 (piano). Tabla remains programless on channel 10. pan/reverb CCs follow immediately at tick 0 and precede clip automation at that tick. Channel state persists across loop passes, so static CCs are emitted once while authored clip automation repeats with its clip.

Patterns.

• sustain — the bar’s full triad held for the whole bar.
• arpeggio — eighth notes cycling chord tones in the fixed order root, third, fifth, third.
• bass — chord root two octaves down; two half-bar notes when the meter’s numerator is even and ≥4, else one whole-bar note.
• drums — kick on beat 1 (plus the midpoint beat when N ≥ 4), snare on off-beats, closed hi-hat on every beat and half-beat.
• tabla — one channel-10 stroke per notated beat, cycling the fixed 16-beat sequence dha dhin dhin dha | dha dhin dhin dha | dha tin tin ta | ta dhin dhin dha across bar boundaries.
• melody — the named motif, looped or truncated to exactly fill the length; degrees resolve in the scene key.
• clip — the named exact event sequence, emitted once or repeated without generative timing transforms; pitched clips pair with melodic instruments and percussion clips with percussion tracks.

Transform order. Generated patterns use swing → dynamics → legato → humanize, then glide bend computation (it must observe final onsets), then loop-pass duplication. Clips apply only intensity/dynamics velocity scaling before exact repetition; their authored timing is never humanized, swung, or made legato. Loop math is applied last so looped scenes stay sample-exact under every transform.

Harmony. One numeral per bar, cycled. Numerals resolve to diatonic triads of the scene key; sustain/arpeggio/bass all read the same per-bar chord, which is what keeps multi-track scenes harmonically coherent by construction.

Texture assembly. FFmpeg first normalizes every referenced recording to stereo signed 16-bit PCM at the requested build sample rate. scorekit then performs only deterministic cutting, repetition, zero-padding, placement, linear gain, and summation. Texture stems use the same exact output window and loop seal as instrument stems, so every stem stays sample-aligned and sums back to the full mix within the existing rounding tolerance.

Error contract

Protocol violations follow the machine-interface error shape: exit 2, and with --json a single structured object on stderr carrying location (parse) or field (semantic). The field path grammar is stable: tracks[i].pan, sections[i].mute[j], harmony[i], performance.swing. An agent is expected to repair a scene from field + message alone; that expectation is part of the protocol’s design, not a nicety.

What the protocol will not carry

• No mood/state/intent fields (emotion, danger, avoid) — no deterministic compile semantic exists for them; they live in the agent’s brief.
• No renderer or recording paths (.sf2/.sfz/audio locations) — sound sources are bound at invocation time (--soundfont, --orchestration, --texture-profile), so a scene never bakes in one machine’s disk layout. A track’s id/palette are portable routing metadata, not paths — the orchestration profile they route through resolves to real files, never the scene.
• No embedded version negotiation, includes, or macros — one file, one scene, strict fields. Composition happens in the agent, not in a template engine.