In terms of music reproduction enjoyment / preferences I tend to use my own hearing ability. Alas, I can't use measurement devices or double blind tests outcomes to listen to music.
Correct, for mere enjoyment of math, running and especially music etc, one needs only listen and prefer, there is no need to measure (unless one wanted a way better sounding system free of terrible bass peaks, out of phase, reversed channels, etc etc), or blind test test ability there. One can just prefer whatever one prefers and enjoy.
It is only if wanted to actually test those abilities, that measurements and tests would come into play. That's why one doesn't grade ones own math test, why Olympic runs are exactly timed and why all the testers in Bobs MQA test referenced have assessed, verified abilities, just like with math or running. Self assessed, believed abilities are verboten.
True experts in HiFi like Nelson Pass understand the limitations of measurements.
Exactly, that's why Mr Pass has never ever designed an amplifier without measurements. That's why Mr Pass designs based on the foundation of his science education. Since it's audio devices, he knows
measurements, science and listening are all essential.
Exactly as done in the MQA test.

He, like the testers and myself, fully understand the value of all the factors.
And I'm under the impression that major orchestras existence preceded measurements, published papers and double blind tests...
Yes, voodoo and witchcraft preceded modern medicine also. Some are doomed to not learn from history and may still go to such practitioners. Luckily, some know better now, so modern medicine is based on science and double blind tests, as are modern orchestras.
http://gap.hks.harvard.edu/orchestrating-impartiality-impact-“blind”-auditions-female-musicians
I think we can all be thankful the misogyny, bigotry, biases, etc. that plagued the past is well addressed today, at least for many. I love seeing more female orchestral players!
Is it weak? The McNamara Fallacy is not just about data mining. From Wiki:
The McNamara fallacy (also known as quantitative fallacy[1]), named for Robert McNamara, the United States Secretary of Defense from 1961 to 1968, involves making a decision based solely on quantitative observations (or metrics) and ignoring all others. The reason given is often that these other observations cannot be proven.
Yes, because blind tests
don't ignore all factors, they account for them, unlike bias filled self assessed abilities uncontrolled listening. The test methods chosen by Stuart is
not based solely on measurements, but by actual trust ears/just listening abilities. Red Herrings for rejecting scientific method is extremely weak.
I'd say being a professional violinist not only makes him an expert
That is classic Appeal to Authority fallacy. Him being a professional violinist doesn't make him immune from biases, If he can prove that he can tell the differences between the
sound of violin by listening free of his beliefs, biases, etc that plague all humans, "expert" or not, then he can demonstrate his "expertise". Just like one has to take a math test and have it graded by another party, or run a 100m dash...and have it measured either against or runners..or measured time.
Being a self assessed/declared "expert" is irrelevant here, only demonstrable ability is. He can either do that, or "talk" about it like that article.
it probably allows him not to fall into the McNamara Fallacy or suffer from the Dunning-Kruger effect.
Perfect. Your knowledge of subjects is spot on.
https://en.wikipedia.org/wiki/Dunning–Kruger_effect
In the field of psychology, the Dunning–Kruger effect is a cognitive bias in which people of low ability have illusory superiority and mistakenly assess their cognitive ability as greater than it is. The cognitive bias of illusory superiority comes from the metacognitive inability of low-ability people to recognize their lack of ability; without the self-awareness of metacognition, low-ability people cannot objectively evaluate their actual competence or incompetence.
This is exactly why self assessment of ability is really, really bad idea. Thank you!
Yes, to get back to the thread subject, BStuart is acutely aware of this and chose to team with McGill U (a great Higher Ed institution btw) to do non-Dunning-Kruger "self assessment/grade my own math etc" type analysis of MQA. He made a very wise choice! Thanks again for all the info.
cheers,
AJ