File size: 35,953 Bytes
03c475d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 | ---
license: mit
tags:
- taiko
- rhythm-game
- chart-generation
- music
- audio-to-symbolic
---
# BarScript (experimental preview) β C1.3, denominator augmentation, seed 1234
Audio log-mel + a bar grid β a Taiko no Tatsujin chart, decoded under a
finite-state grammar over bar-scoped plan, skeleton and realization stages.
**BarScript** is the release name of this charting model; it is also the name of
the bar-scoped multi-stage token encoding it decodes into. It is a **preview
under evaluation**, published so it can be tried, not as a finished model: the
champion comparison below is `inconclusive`, and Β§2 lists measured defects β one
of which fires a pre-registered kill condition. The seven published SoftChart
1.x models are a separate, unaffected line.
| | |
|---|---|
| source checkpoint | `runs/sc2_c1_v13_s1234/best.pt` |
| sha256 | `44980bf64b1611ea73c1433c41adceff7596cfeece30dfaad5d228eae4a14e6e` |
| seed / step | 1234 / 96 000 of 100 000 |
| best validation CE | 0.5769365892960475 |
| parameters | 9 107 291 |
| shipped weights | bfloat16 safetensors, 18.2 MB (fp32 checkpoint preserved in the run dir) |
| training data | `JacobLinCool/taiko-1000-parsed-clean`, revision `b72da4616d643018e81f372cea06ce51349285e0` |
| label spec | `barscript_labels_v1`, frozen, shipped as `spec.json` (sha1 `cd876768β¦`) |
| training flags vs C1 | `--cal-axes big span density` Β· `--balance-exclude-big` Β· `--denom-augment 0.5 --denom-augment-mode lattice` |
| external pretraining | none |
**This card is written for THIS checkpoint.** Every number below was measured on
C1.3's own generations under the serving code this package ships
(`generate.py` md5 `73b6e49bβ¦`, `min_onset_gap_sec = 0.020`). Nothing is
inherited from the C1 card. Where an item on the C1 card has no C1.3
measurement, it is marked **not measured on this arm** rather than carried over.
**Read the KNOWN LIMITATIONS section before using any number above it.** This
model has real, measured defects, one of which fires a pre-registered kill
condition, and its difficulty ladder is roughly 25 coverage points less nested
than an authored one.
---
## 0. Status: the champion comparison is `inconclusive`
`scripts/sc2_gates.py champion`, three pairings on one serving code base
(`experiments/sc2_eval/FINAL_CHAMPION.md` Β§1):
| pairing | `champion_verdict` | reason code |
|---|---|---|
| C1.2 vs C1 | `inconclusive` | `no_seed_replicate` |
| **C1.3 vs C1** | **`inconclusive`** | `no_seed_replicate` |
| C1.3 vs C1.2 | `inconclusive` | `no_seed_replicate` |
`MIN_SEEDS_FOR_VERDICT` is 2 in **both** arms and C1.3 has one seed. The rule
fires before any metric is read. **No arm has been declared champion.**
What the clause ledger says, which is one-directional: C1.3 **passes** the
required Β§4 clause `plan_token_calibration`, which **C1 fails on both its
seeds**. No clause is passed by C1 and failed by C1.3. The two `a2` span clauses
fail on **all four arms** and belong to the campaign, not to this checkpoint.
The shipping rationale, the two named compromises and the decision rule for
choosing a different checkpoint are in
`experiments/sc2_eval/SHIP_DECISION.md`.
Gate items on the 71-chart hard+oni population, this arm against the C1 seed-1234
arm under identical code:
| | C1 | **C1.3** |
|---|---:|---:|
| fail / pass / na | 9 / 10 / 6 | **8 / 11 / 6** |
| newly passing | β | `control_own_axis_density`, `control_named_regression_density_big_count` |
| newly failing | β | `a2_span_placement_fit` (knife-edge, Β§2.6) |
---
## 1. What it does
* **Input** β 128-bin log-mel at 22 050 Hz (`preprocessor_config.json` freezes
the exact decode/STFT/mel contract), plus a bar grid with explicit measure
edges, exact rational meters and per-bar lattice denominators.
* **Conditions** β course, authored level, density bucket, and the split-axis
knobs `big_rate` / `span_rate` / `stream` / `sync`.
* **Output** β a BarScript token stream decoded under `BarscriptFSM`, exported
as TJA on the 96-slot lattice.
* **Family decoding** β `generate_chart_family` decodes a song's courses as one
nested ladder, hardest first, each easier course biased toward its harder
sibling's onsets. **Off by default**; the campaign's headline population was
decoded independently (Β§2.1).
Capabilities this checkpoint carries (`config.json:capabilities`): `aux`,
`beat_head` (hi-res), `hierarchical_ctx`, `axis_knobs`, `sync_token`,
`stage_emb`, `span_duration_head`, `tempo_head`, `rich_section_stats`. It
carries **no** `style`, `sibling`, `ctx`, `plan`-prefix, `slot`, `dual`,
`align`, `mask_infill`, `func_time`, `global_ctx` or `complexity` conditioning;
requests on those axes reach nothing.
### What works
Evidence marks: **β** n β₯ 71 charts with a measured C1 seed band; **β** n = 19β71
charts or 8β12 sweep cells, one seed; **β** n < 19 or a CI covering zero.
| axis | this checkpoint | reference | mark |
|---|---|---|---|
| **Density knob** β median within-song Ο **0.850**, sign p **0.0078**, 8/8 cells positive, **+1.413 nps** firstβlast, 12 % monotone adjacency | C1 moves the chart +0.024 / +0.151 nps over the same range and **fails** the gate; C1 seed-to-seed ΞΟ 0.003 (p 1.000) | 8 songs Γ 5 buckets, oni | β |
| **`big_rate` knob** β Ο **1.000**, p 0.00049, 91.7 % monotone, and **`calibrated: true`** β mean absolute bucket error **0.533**, within Β±1 bucket **93.3 %** | first axis in the whole campaign to clear `calibrated`; C1 0.533 vs 1.200 / 0.967 | 12 songs Γ 5 buckets, oni | β |
| **`span_rate` knob** β Ο 0.810, p 0.0117, `calibrated: true`, mean abs err 1.483; and unlike C1 it **no longer drags hit density with it** | C1's span knob flags `onset_nps` / `hit_nps` / `n_onsets` as interference; C1.3 flags only the definitional `span_count` | 12 songs Γ 5 buckets, oni | β |
| **Span budget** β 739 spans against the authored 762 (**0.970Γ**); `span_rate_gen_per_min` gap to authored **0.049** against a C1 seed band of 0.198 | C1 1.160β1.244Γ, C1.2 1.259Γ | 97 charts | β |
| **FSM guarantees** β **0 unclosed, 0 orphan ends, 0 swallowed hits** over 739 generated spans | holds on all four arms | 97 charts | β |
| **Placement does not degrade** β precision 0.6477 (C1 band 0.6480β0.6502), median \|offset\| **0 ms**, `exact_slot_lift_over_null` **1.1755** (above the C1 band), long-song drift tests significant **2** vs C1's 6 / 5 | see Β§2.3 for why apparent recall drops | 71 charts | β |
| **Accent over-emission reduced furthest** β pooled hard+oni big share **0.0873** against C1's 0.1082 / 0.1027 (3.3 seed bands) and an authored 0.0589 | still outside the pre-registered stop window 0.045β0.075 | 71 charts | β |
| **Cheaper** β 1 618.6 tokens/chart, 19.22 tokens/bar, `max_window_tokens_p99` **449** | C1 1 731.2 / 20.51 / 515 | 71 charts | β |
| **Deployment grid repaired** β on a forced `/16` BPM grid, notes Γ· authored **1.130 / 1.112 / 1.067 / 1.027** and onset F1 **0.333 / 0.374 / 0.466 / 0.580** | C1 1.688 / 1.461 / 1.297 / 1.210 and F1 0.318 / 0.376 / 0.477 / 0.599; C1.3 beats C1.2 on **12 of 12** deployable cells | 13 songs Γ 4 courses, one seed | β |
| **Empty-bar declaration restored** β `/16` empty bars 12.82 / 8.60 / 4.55 / 3.14 % against C1.2's 5.96 / 3.23 / 1.08 / 0.33 %, sign-significant on 3 of 4 courses | authored 15.69 / 12.30 / 9.08 / 6.27 % | 13 songs | β |
---
## 2. KNOWN LIMITATIONS
Nothing in this section is softened. Where a limitation is invisible to the gate
suite, that is said.
### 2.1 The difficulty ladder is 25β32 coverage points less nested than authored
Coverage = the fraction of the easier chart's notes that have a note in the
harder chart within tolerance. **Two decode modes, two different numbers, and
both belong on the record.**
| adjacent pair | authored charts | **independent decode** (package default) | **family decode Ξ² = 2** (authored grid) | **family decode Ξ² = 2** (`/16` deploy grid) |
|---|---:|---:|---:|---:|
| easy β normal | 0.970 | **0.658** | 0.730 | 0.726 |
| normal β hard | 0.979 | **0.667** | 0.797 | 0.811 |
| hard β oni | 0.985 | **0.660** | 0.822 | 0.834 |
Independent-decode figures: `FINAL_CHAMPION.md` Β§8.1, n = 13 / 13 / 35 songs, one
seed, same serving code as this package (β/β). Family-decode figures:
`C13_PREDICTION.md` Β§3.3, n = 13 songs, one seed (β).
Five things make this worse than the headline:
* **`nesting_coverage` and `hand_agreement` FAIL the family gate on every arm
measured**, C1 included. `difficulty_monotonicity` passes on all of them.
* **C1.3 is slightly worse than C1 here.** C1's independent-decode coverage on
the same population is 0.690 / 0.704 / 0.737; C1.3 is β0.032 / β0.037 / β0.077
against C1 seed bands of 0.010 / 0.021 / 0.025. The move is small against the
~0.30 gap to authored that every arm shares, but it is in the wrong direction.
* **Hand agreement at coinciding hits is near chance on hardβoni**: C1.3 0.539
against a marginal-chance null of 0.500 and an authored 0.753. When two
generated courses agree that a note belongs somewhere, which drum they pick is
near-independent across courses.
* **The `/16` figure overstates the model's nesting.** The same checkpoint on a
`/96` grid drops to 0.649 / 0.730 / 0.752, worse on 11β12 of 13 songs
(p = .003 / .022 / .022). A coarse lattice manufactures agreement by leaving
few places to disagree (`C13_PREDICTION.md` Β§5.2).
* **`family_bias = 2.0` / `family_hand_bias = 1.5` are PROVISIONAL** β described
in `generate.py` as logit offsets on a decoder never trained at that setting,
and never calibrated against the authored target.
### 2.2 The note-type channel carries almost no information
71 charts, **22 260 matched slots** (`FINAL_CHAMPION.md` Β§5.2, β).
| quantity | this model | its own floor | corpus Β§2.1 floor |
|---|---:|---:|---:|
| 4-way accuracy | **0.4776** | majority **0.5559** | majority 4-way **0.563** |
| β margin over majority | **β0.0783**, 95 % CI [β0.1022, β0.0530] | | |
| hand accuracy | **0.5493** | always-don **0.5975** | always-don **0.620** |
| β margin | **β0.0482**, CI [β0.0721, β0.0229] | | |
| **MI(generated; authored), 4-way** | **0.01028 bits** | β | **0.79 % of H(authored) = 1.300 bits** |
| MI, hand | 0.00534 bits | β | 0.55 % of 0.972 bits |
* **The model sits below its own majority floor on both endpoints**, and below
the campaign's corpus baselines (56.3 % / 62.0 %), which are a fixed reference
line from a separate 59 961-hit census, not this population.
* **The honest caveat, which cuts the other way.** The published floors are
*argmax* predictors scored against a temperature-1.0 top-p-0.95 *sample*.
Against the sampler-appropriate i.i.d. floor (Ξ£ pΒ² = 0.4582 4-way, 0.5190
hand) this model is **above** baseline by **+0.0194** and **+0.0303** β the
largest 4-way excess of any arm in the campaign. Both readings belong on the
record.
* **MI is flat at ~0.010β0.012 bits on all four arms** (seed band 0.0008). The
accuracy differences between arms are marginal-matching, not information.
* Accuracies are conditional on coverage **0.6442** (precision side) /
**0.5842** (recall side); more than a third of generated hits have no authored
partner, and coverage is 9 % lower than C1's, so this population is smaller
and differently selected than C1's.
* **No inter-charter agreement ceiling exists** for this split β no
(song, course) carries two independent authored charts β so 100 % is not a
legitimate target and is not used as one.
* **Greedy re-decoding was not repeated on this arm.** The C1-era finding that
greedy closes part of the gap at an unacceptable `motif_reuse` cost is **not
measured on this arm** and must not be quoted for it.
### 2.3 The note budget is 9 % short, and it is the whole of the apparent timing regression
| | C1 (s1234 / s4321) | **C1.3** |
|---|---:|---:|
| `onset_precision` | 0.6480 / 0.6502 | **0.6477** |
| note budget (gen Γ· authored notes) | 1.0046 / 0.9912 | **0.9069** |
| `onset_recall` | 0.6510 / 0.6445 | **0.5874** |
| `frac_of_authored_within_jnd` | 0.6463 / 0.6397 | 0.5840 |
| `type_coverage_recall` | 0.6465 / 0.6401 | 0.5842 |
`precision Γ budget` equals `onset_recall` to four decimals on **every** arm, by
construction. Those three "timing" rows are one quantity, and it is the note
budget, not placement: precision is inside the C1 seed band, median offset is
0 ms, `exact_slot_lift_over_null` is *above* the band, and significant drift
tests fall from 6 / 5 to 2.
**The shortfall itself is a real regression and its cause is unknown.** It
appears on both C1.2 and C1.3, so it belongs to the `--cal-axes` /
`--balance-exclude-big` pair rather than to the denominator augmentation. It is
the largest practical regression in this release.
### 2.4 Raising density squeezes drumrolls out β this is the compromise a user can hit
Median spans/min over eight oni songs, per requested density bucket
(authored **3.885**; `FINAL_CHAMPION.md` Β§4.3, 8 songs Γ 5 buckets, β):
| requested density | 0 | 3 | 7 | 11 | **15** |
|---|---:|---:|---:|---:|---:|
| C1 (s1234) | 3.629 | 3.055 | 4.153 | 2.818 | **2.971** |
| **C1.3** | 3.550 | 4.413 | 2.172 | 2.727 | **1.279** |
Ο(density β `span_per_min`) = **β0.759**, sign p 0.0078, firstβlast β2.443 /min.
At the top of the density request C1.3 delivers **33 %** of the authored span
rate; C1 delivers 76β79 %, C1.2 44 %.
* **Do not advertise the density knob at its top setting** until this is fixed.
* There is a named suspect: `cal_span` never left **0.1667** on either
challenger, so this is plausibly a calibration term that never trained rather
than an intrinsic cost of the knob.
* The mirror image is a *gain*: C1.3's `span_rate` knob no longer drags hit
density, which C1's does.
### 2.5 Absolute bucket calibration is still wrong on the density axis
| axis | exact bucket | within Β±1 | mean abs error | mean signed | `calibrated` |
|---|---:|---:|---:|---:|:--:|
| density | 5.0 % | 25.0 % | 4.125 | β1.075 | **false** |
| `big_rate` | **53.3 %** | **93.3 %** | **0.533** | +0.267 | **true** |
| `span_rate` | β | β | 1.483 | β | true |
The density knob orders the request correctly and lands in the wrong bucket,
under-shooting by ~1 bucket on average β more than C1.2 does. Rank control is
real; absolute density targeting is not delivered.
Note the cost that comes with the `big_rate` calibration win: C1.3 has the
**shortest** `big_rate` ladder of any arm (firstβlast +0.159 against C1's
0.183 / 0.224), so the top of that request now delivers less accent than C1's
did.
### 2.6 It fails its own span-quality gate and triggers a pre-registered kill condition
71 charts hard+oni (467 closed spans) and 26 charts easy+normal (272):
| axis | hard+oni | easy+normal |
|---|---|---|
| `duration_low_tail` | **fail** (rate verdict passes; balloon 0.250 beats = 0.75Γ the shortest authored balloon, roll 0.250 = 0.60Γ) | **fail** (balloon 0.250 beats = 0.20Γ the shortest authored balloon; 45/159 balloons below authored support) |
| `forced_close` | **fail** β **7 / 467 = 1.499 %** (0 unclosed, 7 clamp-forced) | **fail** β **7 / 272 = 2.574 %** |
| `over_span_max` | **fail** β one balloon 7.72 s = 1.19Γ the 6.5 s ceiling | **fail** β one balloon 7.58 s = 1.17Γ |
| `placement_fit` | **fail** β AUC **0.560** (CI [0.529, 0.593]) against a 0.56 floor; authored reference 0.668 | pass β AUC 0.604 |
| `orphans` / `swallowed` / `rate` / `duration` | pass | pass |
| **`a2_kill_condition_not_triggered`** | **fail β the kill condition FIRED** | fail |
The pre-registered kill condition (`SOFTCHART2_DESIGN.md` Β§4-A2) reads: *rate is
calibrated but placement/length quality is bad β concede that the bar-scoped
commitment mechanism is insufficient and escalate to an independent span-state
objective.* It fires on **all four arms**, including C1. Shipping does not
discharge it.
**`placement_fit` is a knife-edge, not a finding.** C1.2 passes at 0.561 and
C1.3 fails at 0.560 with CIs overlapping every other arm across their whole
width. One thousandth of AUC decides the verdict. It is reported, not used.
### 2.7 One span in seven is a micro-span
All generated spans, 97 charts, `FINAL_CHAMPION.md` Β§2 (β):
| | n spans | p1 | p5 | p50 | p90 | min | **share < 0.25 s** |
|---|---:|---:|---:|---:|---:|---:|---:|
| authored | 762 | 0.177 | 0.273 | 0.963 | 2.419 | 0.083 | **3.8 %** |
| **this model** | 739 | 0.089 | 0.150 | 0.818 | 2.258 | **0.072** | **14.7 %** |
Per course: easy 6.6 %, normal 8.1 %, hard 13.7 %, oni **25.2 %** β authored
3.0 / 3.3 / 2.1 / 7.1 %.
* C1.3 does **not** inherit C1.2's worsening (16.4 %); 14.7 % is exactly at the
top of the C1 seed band (13.6β14.7 %). It is not an improvement either.
* **Mechanism is known and unfixed on every arm.** Duration bucket 0 is open at
the bottom; 123 of C1.3's 739 spans sit in it, at a median 0.5 beats against
the authored 0.917.
* **The shipped floor does not fix it.** `span_min_sec` / `span_min_beats` fired
263 times (`span_close_floor`) plus 33 carries across 97 charts and the
< 0.25 s share is still 14.7 %.
* **The mitigation is unbuilt.** `span_bucket0_profile` is `null` in this
package because `scripts/build_span_bucket0_profile.py` has never been run on
the train split. This is the shortest available fix and it is not in the box.
* **The gate cannot see it directly.** `a2_span_duration` compares p50/p90/p99
only and passes; `duration_low_tail` catches the support violation but not the
rate (its rate verdict passes on this arm).
* On a `/16` deployment grid the same quantity is 14.7 / 12.4 / **30.6** / 21.8 %
β hard is untouched by the augmentation and is the worst cell in the release.
### 2.8 The deployment grid has a two-role conflict that no checkpoint dissolves
`BAR_DENOM` is simultaneously an input the grid supplies and a **trained output
token** the model reads as a density announcement. `bar_denoms="supplied"`, the
shipped default, teacher-forces it.
13 songs Γ 4 courses, one seed, `--cond authored`, `C13_PREDICTION.md`:
| easy / normal / hard / oni | authored grid | **forced `/16`** | `/96` | `/24` |
|---|---|---|---|---|
| notes Γ· authored | 0.917 / 0.919 / 0.875 / 0.899 | **1.130 / 1.112 / 1.067 / 1.027** | 0.944 / 0.950 / 0.943 / 1.027 | 1.102 / 1.070 / 0.950 / 0.921 |
| onset F1 | 0.613 / 0.565 / 0.595 / 0.647 | **0.333 / 0.374 / 0.466 / 0.580** | 0.303 / 0.325 / 0.393 / 0.535 | 0.342 / 0.370 / 0.442 / 0.542 |
| genuine triplets (den 3/6/12/24) % | 1.13 / 2.48 / 6.34 / 9.43 | **0.00** (arithmetically impossible) | 3.45 / 5.78 / 10.57 / 11.46 | 14.13 / 20.31 / 28.57 / 38.70 |
| ultra-fine (den 48/96) % | ~0 | 0.00 | **11.53 / 13.22 / 15.62 / 6.60** | 0.00 |
| nesting (family Ξ² = 2) | 0.730 / 0.797 / 0.822 | 0.726 / 0.811 / 0.834 | 0.649 / 0.730 / 0.752 | 0.714 / 0.769 / 0.779 |
Authored triplet rates for reference: 4.20 / 5.72 / 7.43 / 9.95 %; ultra-fine
~0.00 %.
**The recommended deployment configuration is `--grid bpm --grid-bpm-denom 16`
with the package default `bar_denoms="supplied"`.** Its two costs, stated
plainly: **triplets are exactly 0 and cannot be otherwise**, and hard micro-spans
stay at 30.6 % against an authored 2.1 %.
Four consequences a deployer must carry:
* **Deployment costs 10β46 % of onset F1** relative to the authored grid:
easy β46 %, normal β34 %, hard β22 %, oni β10 %. Every `--grid authored`
number in this card is therefore optimistic for a user who supplies only BPM.
* **`p90 |offset|` is 0 ms on the authored grid and 14.6β36.7 ms on the bpm
grids.** That residual belongs to the bar edges; no denominator choice touches
it.
* **The model's off-dyadic rate is a property of the model, not the lattice.**
Given any 3-divisible grid it puts 2β3Γ the authored share off the dyadic
grid; the grid only decides whether that mass gets called "triplets" (`/24`)
or "ultra-fine jitter" (`/96`). This is a training-side defect and no
deployment grid fixes it.
* **The gate suite cannot score the deployment path.** `robustness.py:629-642`
raises `ContractError` on uniform-BPM arms, so every gate timing metric is
`null` for the path the model ships on; the `/16` figures above come from
`timing.py`, a second implementation that reproduces the gate on the authored
grid.
**The `deploy_bpm_grid` profile in this package is NOT the recommendation.** It
sets `bar_denoms="model"`, `bar_denom_mask="div96"`, per `DEPLOY_GRID_FIX.md`
Β§7 β a recommendation made explicitly conditional on C1.3 failing, which it did
not. That arm was measured on **C1 only**: density 1.06 / 1.08 / 0.97 / 1.01 and
nesting 0.635 / 0.706 / 0.778. **It has never been run on this checkpoint.** The
profile is kept because it is implemented and because the arm is the obvious
next experiment, not because it is advised here.
**One contradiction inside this package, stated so nobody has to find it.**
`config.json:serving_profiles.deploy_bpm_grid.basis` still quotes
`DEPLOY_GRID_FIX.md` Β§7 verbatim, including the words *"Recommended now"*, and
still carries C1's nesting figures. That string is generated by
`scripts/sc2_package.py`, which is read-only for this release. **This section
supersedes it.** The generated string is left intact rather than silently
diverging from the shipped code.
### 2.9 Playability floor: gaps no human hand can play
Recomputed on this arm's own events, 71 charts hard+oni, generated vs authored
on the same songs (producer `experiments/sc2_eval/ship_c13/playability.py`,
records `.../records/playability_all_arms.json`; β):
| | this model | authored |
|---|---:|---:|
| inter-onset gaps < 40 ms | **102** across **29** charts | 2 across 1 chart |
| β as a share of adjacent pairs | **0.296 %** (102 / 34 484) | **0.0053 %** (2 / 38 032) |
| gaps below the series record 29.4118 ms | **19** | 0 |
| shortest gap | **20.8 ms** (the serving floor) | 39.7 ms |
| peak burst in any 1 s window | **17 notes** | 15 notes |
C1 on the same population: 151 gaps (0.395 %), shortest 16.7 ms, peak 22
notes/s. C1.3 is better on every row and still **56Γ the authored rate**.
* `min_onset_gap_sec = 0.020` is a **degeneracy guard, not a fix**: it is set
below the fastest thing the series has shipped (29.4118 ms,
TAIKO-TONGUE-TWISTER oni, BPM 170, 48th notes) precisely so it cannot refuse a
chart the domain writes. It fired **125 times** across 97 charts with **0**
fail-opens. *A mask cannot fix a distribution.*
* The `generate.py` constant comment quotes the generated sub-40 ms rate as
~0.35 % against ~0.045 % authored. **That authored figure does not reconcile
with the 0.0053 % measured here**, and the two have never been put on the same
denominator. The excess is 8Γ on the comment's accounting and 56Γ on this one.
* **Good news that replaces a C1-card claim.** The C1 card reported the model
placing hits in 19.6 % of sub-0.25 s scaffolding bars. Under today's serving
floors this arm places **0 hits in all 112 such bars** across 97 charts, exactly
as the authored side does; `degenerate_bar_hits_blocked` fired 12 times with 0
fail-opens.
### 2.10 Pattern proxies regress against C1 on hard+oni
71 charts, one code version, C1 band = |s1234 β s4321| (`FINAL_CHAMPION.md` Β§3):
| metric (lower is better) | C1 mean | C1 band | **C1.3** | Γband | relative |
|---|---:|---:|---:|---:|---:|
| **`motif_reuse_gap`** | 0.3630 | 0.0042 | **0.4222** | +14.1 | **+16.3 %** |
| `compression_gap` | 0.0976 | 0.0006 | 0.1116 | +23.3 | +14.3 % |
| `motif_ref_marginal_js` | 0.0331 | 0.0046 | 0.0452 | +2.6 | +36.6 % |
| `ioi_js_per_chart` | 0.0295 | 0.0006 | **0.0352** | **+9.7** | +19.3 % |
| `ul_4gram_js_per_chart` | 0.2402 | 0.0055 | 0.2530 | +2.3 | +5.3 % |
| `class_4gram_js_per_chart` | 0.1337 | 0.0080 | 0.1357 | +0.2 | +1.5 % |
The `Γband` column overstates the case β several bands are under 1 % of their own
level β so the relative column is the honest one. On `motif_reuse` the bootstrap
CIs are nonetheless **disjoint**: C1 [β0.392, β0.334] / [β0.388, β0.334] against
C1.3 [β0.447, β0.396]. **The regression is real and not seed noise.**
Four bounds, all measured, none of them a dismissal:
* **It does not happen on easy+normal.** Every pattern proxy there is inside the
C1 seed band, `motif_reuse_gap` is marginally *better* than C1's mean, and
`class_4gram_js` is better on both challengers.
* **The decomposition puts the loss on rhythm and hand, while the accent layer
improves**: rhythm 0.1251 β 0.1640, hand 0.1774 β 0.2102, accent 0.0606 β
**0.0480**.
* **`ioi_js_per_chart` is where the augmentation shows up** β it reads the exact
rational IOI lattice, and it is C1.3's worst pattern metric relative to C1.2
(+9.7 band against +1.5).
* **The proxy reads mostly a channel carrying ~0.010 bits** about the authored
type (Β§2.2), and it has **never been validated against a listener**. It is
quoted because Β§4 names it, not because it is strong.
Realized `motif_reuse` is **0.3165** against an authored **0.7387** β under half
the authored repetition.
### 2.11 The note-type marginal, and one C1 finding that does NOT reproduce
Big notes are **1.48Γ the authored rate** pooled hard+oni (0.0873 vs 0.0589),
worst on the easiest course:
| course | authored | **C1.3** | C1 (s1234 / s4321) |
|---|---:|---:|---:|
| easy | 18.80 % | **26.22 % (1.40Γ)** | 32.68 % / 29.65 % |
| normal | 11.75 % | **17.41 % (1.48Γ)** | 19.21 % / 20.77 % |
| hard | 7.38 % | **10.61 % (1.44Γ)** | 13.11 % / 12.73 % |
| oni | 4.88 % | **7.40 % (1.52Γ)** | 9.29 % / 8.58 % |
The pre-registered stop window (0.045β0.075 pooled hard+oni) is **not met**.
**The C1 card's Β§2.8 claim does not reproduce here and is corrected.** On C1 the
realized class marginal was 13Γ closer to the training loss weights than to the
corpus. On C1.3 it is the other way round: JS(gen β authored) = **0.00588**
against JS(gen β class-weight prediction) = **0.00746**
(JS(authored β weights) = 0.01452). `--balance-exclude-big` moved the marginal
off the loss weights and toward the corpus. The log-log fit of realized share
against class weight collapses from slope 0.378 (Ο 0.436) on C1 to slope 0.078
(Ο 0.156) on C1.3.
### 2.12 Fine-lattice, tuplet and syncopation accuracy are much worse than average
`robustness.py` timing strata, 71 charts, exact-slot rate (overall **0.5840**):
| stratum | n authored | exact-slot | C1 (s1234) |
|---|---:|---:|---:|
| lattice step 11.6β23.2 ms | 114 | **0.342** | 0.456 |
| lattice step 23.2β46.4 ms | 790 | **0.443** | 0.465 |
| lattice step β₯ 92.9 ms | 24 228 | 0.594 | 0.651 |
| positions with a denominator divisible by 3 | 3 480 | **0.345** | 0.496 |
| syncopation band 0 β band 5 | 20 846 β 1 669 | **0.615 β 0.434** | 0.672 β 0.514 |
| local IOI 25β50 ms | 189 | **0.402** | β |
| BPM > 250 | 2 570 | **0.499** | β |
Every stratum is lower than C1's, which is the note-budget effect of Β§2.3 acting
on a per-stratum recall, but the *shape* is the finding: tuplet positions and
fast lattice steps are 0.24 below the overall rate, and accuracy decays
monotonically with syncopation.
### 2.13 Smaller, but on the record
| finding | number | n | source |
|---|---|---|---|
| `long_song_no_drift` fails | **2** of 21 metrics significant at BH q β€ 0.05 (`dens_signed_err`, `plan_dens_bucket_tvd`) β C1 fails 6 / 5 | 71 charts | `robustness.json` β |
| Harness-level generation failures | **6** of 71 song-courses refused (`#BRANCHSTART` and non-representable meters); **1** chart exported `gen_tja = null` β identical to C1 on the same population | 71 charts | `index.json` β |
| Slot-export off-lattice events | 8 across 97 charts (C1: 5) β the documented contract path, not a crash | 97 charts | decoder counters β |
| `escape_hatch` fires | 91 times across 97 charts (C1: 84) | 97 charts | decoder counters β |
| Generation wall time | 9.29 s/chart, 4.33 s per audio-minute β **contended**, the GPU was shared, quoted as a declaration only | 71 charts | β |
| Complete-bar rest placement | **not measured on this arm** (C1: Jaccard 0.474, recall 0.623) | β | β |
| Greedy-decode contrast | **not measured on this arm** | β | β |
| Micro-span seed sensitivity | **not measured on this arm** (C1: 24.0 % vs 32.4 % across two sampling seeds on 19 oni charts) | β | β |
### 2.14 The evidence base is thinner than it looks
* **One seed.** Every number in this card is n = 1 in the training seed. The C1
seed band quoted throughout is C1's, used as a reproducibility scale; it is a
range over n = 2 and carries no confidence statement.
* **easy and normal are evaluated on 13 songs.** The 71-chart population is
`hard` + `oni` only; the easy/normal population is 26 charts from 13 songs, and
its seed band is 4β5Γ wider than hard+oni's.
* Every difficulty-family and deployment-grid number is **13 songs, one seed**
(β). Every knob sweep is **8β12 songs Γ 5 buckets** (β).
* At these sizes a gate `pass` carries little information. Worked example from
this campaign: the `big_share_of_hits` gate **flips between two C1 training
seeds** while the failing seed's point estimate is *smaller*. At n = 36 that
gate cannot rank checkpoints, and no pass/fail on it should be quoted as
evidence for any arm.
* **Nothing here is audio-referenced** except `a2_span_placement_fit`. Every
other quantity is chart-vs-chart on the exact rational lattice or on hit order.
No timing window and no game judgement parameter appears anywhere in this path.
* **`best_val` is comparable across C1 / C1.2 / C1.3** (one training cache) and
**not** comparable to C1.1 or to any arm on a different cache. C1.3's 0.5769 is
**2.9 % worse** than C1.2's 0.5605; that was the registered trade and the
deployment side won it.
* **No plan-neutral arm was generated**, so the `plan_neutral_fallback` Β§4 clause
is `na` and the ledger is incomplete by one required clause.
### 2.15 Serving regime: what `motif_constraint: auto` resolves to, and why it matters less here
`motif_constraint: "auto"` resolves to **OFF** for this checkpoint: the training
cache's `barscript_md5` (`634e3dc3β¦`) differs from the serving `barscript.py`
(`9afc715aβ¦`), so the gold sequences it trained on never satisfied the MOTIF hard
constraint. `train_serve_matched = true` β OFF is the train/serve-matched choice
and it is the right default.
**Unlike the C1 package, this card needs no correction for it.** Every number in
this card was measured with the constraint **OFF**, i.e. under exactly this
package's resolved default; the 20 evaluation runs all record
`serving_fsm.motif_constraint: off, verified: true`. In particular
`motif_ref_marginal_js` here is **0.0452** (hard+oni) and **0.0114**
(easy+normal), both measured OFF.
Two things that still belong on the record:
* **The ON/OFF blast radius was measured on C1 only**, where switching the
constraint on moved `motif_ref_marginal_js` 0.0311 β 0.0189 (β39 %). On the C1
package the shipped default therefore makes the correct value **0.0311, not the
0.0189 in the older tables**. **That contrast has never been measured on
C1.3**, so no ON-constraint number should be quoted for this checkpoint.
* Serving under the constraint ON would be train/serve **mismatched** for this
checkpoint and is not a supported configuration.
---
## 3. Serving contract
Every value below is in `config.json:serving`, with its justification in
`config.json:serving_basis`. Pass them explicitly β the package's
`serving_kwargs()` does β so the recorded contract is the one that reaches the
decoder.
| parameter | value | one-line basis |
|---|---|---|
| `min_onset_gap_sec` | **0.020** | Degeneracy guard, **not** a corpus percentile. Must stay strictly below the series record of 29.4118 ms (BPM 170, 48ths). Two earlier corpus-derived values (0.0395, 0.0300) were both wrong. See Β§2.9. |
| `degenerate_bar_sec` | **0.25** | Authored charts place zero hits in any bar under 0.375 s (3 010 bars). Largest round threshold with zero counterexamples and 1.5Γ margin; identical to the gate's threshold. Blocks hits only, never span geometry. |
| `span_min_sec` / `span_min_beats` | 0.0833 / 1β3 | Authored population minima. Fail-open, counted. Does not fix Β§2.7. |
| `span_bucket0_profile` | **null** | Not built β see Β§2.7. |
| `family_bias` / `family_hand_bias` / `family_mode` | 2.0 / 1.5 / `bias` | **Provisional**, never calibrated β see Β§2.1. Family decode is opt-in. |
| `motif_constraint` | `auto` β resolves **off** here | Train/serve matching by `barscript.py` md5 β see Β§2.15. |
| `bar_denoms` / `bar_denom_mask` | `supplied` / `lattice` | The evaluated regime, and the recommended deployment regime on a uniform `/16` grid. The `deploy_bpm_grid` profile exists but is **not** recommended for this checkpoint β see Β§2.8. |
| `greedy` / `temperature` / `top_p` | false / 1.0 / 0.95 | As evaluated. |
| `plan_temperature` | `null` (follows `temperature`) | Never exercised in evaluation; shipped unset rather than tuned. |
`_has_sync` **must** be forced on at load. `train.py` records
`sync_token=False` while the training prefix carries the SYNC slot, so a loader
that trusts the recorded flag drops the slot and every sync bucket reaches the
decoder as an identical prefix. The package loader does this and records why in
`config.json:serving_prefix_fix`. This is a workaround; the fix belongs in
`train.py`.
### Recommended deployment configuration
```
--grid bpm --grid-bpm-denom 16 # uniform /16 grid
bar_denoms = "supplied" # the package default
motif_constraint = auto -> off # resolved at package time
family decode: optional; Ξ² = 2.0 / 1.5 is provisional
```
Expected behaviour under it, 13 songs Γ 4 courses, one seed: notes within
13 / 11 / 7 / 3 % of authored; onset F1 0.333 / 0.374 / 0.466 / 0.580;
**zero triplets**; hard micro-spans ~30 %.
---
## 4. Weights are bfloat16
Training ran bf16 autocast and CUDA inference runs bf16 autocast, so fp32
storage carried no information the forward pass could use β the argument the
v1.5 release made and verified. Measured cast cost: max absolute delta
**0.00711** on `frontend.2.weight` (0.349 % of that tensor's max) over 9 107 291
float elements.
This is an argument about the **compute** dtype, not a proof that the two
checkpoints decode identically. Sampling is chaotic in the logits, so individual
charts can differ. The paired fp32-vs-bf16 evaluation has **not** been repeated
for BarScript. The fp32 checkpoint is preserved in the source run directory and
is the reference for any bit-level comparison.
---
## 5. Provenance
| | |
|---|---|
| serving code | `generate.py` md5 `73b6e49b00fb60f3229c8539bc7a1029`; all 18 `src/softchart/*.py` md5s in `config.json:code.module_md5` |
| evaluation harness | `sc2_generate_eval_v2_2_serving_floors`, contract `sc2_eval_v2` |
| evaluation runs | 20 run indices, 1 028 charts, one `serving_floors` signature, `comparable: true`, 6/6 floor counters present |
| headline population | 71 charts hard+oni (36 songs) + 26 charts easy+normal (13 songs), `--cond authored`, `--grid authored`, seed 1 |
| deployment population | 13 songs Γ 4 courses Γ 10 arms, `--cond authored`, one serving tree, one signature |
| label spec | `spec.json`, field-for-field equal to the cache manifest's copy (`spec_parameters_match_cache_manifest: true`) |
| uploaded | **no** (`release_manifest.json: "uploaded": false`) |
Reports: `experiments/sc2_eval/SHIP_DECISION.md`,
`experiments/sc2_eval/FINAL_CHAMPION.md`,
`experiments/sc2_eval/C13_PREDICTION.md`,
`experiments/sc2_eval/C12_EVAL.md`, `experiments/sc2_eval/DEPLOY_GRID_FIX.md`.
|