Text–prosody incongruence · held-out clips
Sarcasm Listening Bench
Listen to a clip, commit to a call, then see what the model made of it.
Every score here is out-of-fold: the model that scored a clip never saw that clip
in training.
How a clip is scored. The corpus is split into five folds grouped by
episode, so no episode appears in both training and test. Inside each fold a congruence
model is fitted from scratch on that fold's sincere training clips only — it
never sees a sarcasm label — and learns what delivery normally accompanies given words.
A clip's prosodic prediction error against that model is the sarcasm feature. The
classifier is then trained on the other four folds and scores the held-out one.
Prosody is 88 eGeMAPS v02 functionals: F0 and loudness percentiles and
slopes, jitter, shimmer, spectral flux, voiced-segment rates. Text is a
384-dimensional all-MiniLM-L6-v2 sentence embedding of the transcript. Audio is
loudness-matched per episode to −23 LUFS, so recording level cannot stand in for the
label.
Clips here are a balanced 48-clip sample of each corpus, deliberately seeded with
cases the model got wrong in both directions. The full corpus scores in the panel above
are computed over every clip, not this sample.