A researcher at the University College of London, Chia-Jung Tsay, studies how we evaluate a performance of music. What makes us think a performance great, or not?
The results are stunning: we use our eyes!
To make a long story short (as my mother used to say, but I recommend reading it*) Tsay selected two groups of subjects, each numbered in the hundreds: one group were professional musicians and the other just plain folks (musical novices). She presented them performances from two different categories: 1) a highly ranked ("world class") orchestra, say the Chicago Symphony, and 2) an average regional orchestra, say a fictitious Fargo Philharmonic. Each 'performance' was six seconds long by: 1) sound recording, 2) a silent video recording, and 3) recordings with audio and video, all of the same passage in the music. Subjects viewed similar performances of chamber ensembles, prize-winning v. just average ones.
Tsay described in great detail her experimental methods and statistical evaluation of the results of dozens of separate tests. They seem rigorous and complete to me. She had to make sure that none of the subjects got clues to the right answers to the single question: which of two performances in each test was by the supposedly higher-rated ensemble?
Again, to oversimplify the results for clarity: the only times significant numbers of either group of subjects identified the prestigious ensembles correctly (more frequently than by chance) were when they viewed the silent video performances! Even the musicians couldn't distinguish the Fargo Phil from the Chicago Symphony by sound recordings!
One additional subtest also showed similar results when the video narrowly focused on the one member of an ensemble who appeared to be the de facto 'leader' of the group. When the camera focused on the 'followers,' the subjects' responses were no better than chance.
There is more detail in the report, but all followed this same pattern supported by robust statistics. As in all research, insightful critics can often discover hidden bias that could change the results and conclusions. And, as in all research, someone else needs to repeat the experiments and confirm or deny these results.
My criticism would be that I need longer than six seconds of music to make any judgement about it.
Like all good research, the results of this one raise additional questions. Will 'rankings' of recordings change as more people experience them on YouTube than on audio recordings? What are the implications for those of us who champion "live music?" Should we re-evaluate the recent policy of auditioning candidates for orchestras behind screens? (Tsay doesn't mention that it began when 'old white men' conductors had certain biases.) Should music schools change their teaching? Are the 'best' orchestra conductors ranked that way by their appearance? Have the people who make 'music videos' known this for years? At least now I understand Kenny G!
Of course, it stands to reason that, as all the musicians said to the experimenter: "it's the sound that counts." We all know that ranking orchestras and ensembles is no more accurate than ranking college football teams, though no money rides on the former. (Or does it?)
*The vision heuristic: Judging music ensembles by sight alone
No comments:
Post a Comment