Brilliant Art Dudley's article

[Zero]

The awesome thing about a well-executed blind test is that it tends to displace the negative elements that usually influence somebody's impression of a component. Bias. Ego. Pre-conceived notions. All of that stuff flies right out the window once the visual factor is compromised. What the listener is left with is a raw sense of how a particular component *actually* performs within a given space/system. Suffice to say, this methodology can serve as a valuable tool – one that can lead to some pretty eye opening experiences!

That being said, the not so awesome thing about blind testing is that it's nearly impossible execute properly. What do I mean? Well, let’s say you want to evaluate three different loudspeakers. Somehow or another, you were able to orchestrate the ideal blind test. The whole experiment will be taking place in your room, on your system, and you’ll be listening to your favorite music throughout the entire test. Sound’s cool so far, right?

Well, there’s just one tiny problem…

Unless you (and whomever will be helping you out) own a wide variety of gear, can swap everything out on the fly without hipping you to what’s going on, and then position the speakers perfectly in your room each and every time, the entire experiment becomes moot. Why? Because you can’t disadvantage a product on one hand and call it a fair evaluation on the other.

The big problem with blind testing is that it’s only effective for super straight-forward product evaluations. I'm talking cables, stands, drivers, Xover components, racks, tweaks, etc... Once you move onto more complex components, it becomes increasingly difficult to accommodate each products unique needs. And if you can't accommodate each products needs, then the whole test becomes compromised.

Anyway, at the end of the day, I feel like the truth lies at the intersection between the pro-blind testers and the anti-blind testers. The methodology can be very useful. However, it shouldn't be looked upon as the only valid way of deducting the performance of a component.

 

[dlb2]

Hopefully, not the McIntosh. They have the patent on ugly gear

Well I would have said something that I own, but I don't want to be biased!

Sent from my HTC One.

 

[puroagave]

Here we go with the "Beauty is in the eye of the beholder" dealio. While I happen to agree with you, there are certain other people (make that lots of certain other people) who sport some wood when they see McIntosh gear. I happen to think the early McIntosh gear looks better than the current gear because McIntosh is using too many different shades of green and blue that clash to my eyes. But then, some people think the Shindo 604 speakers look like a thing of beauty and all I can think of is a Russian Babushka.

hey, hey...now you're hitting too close to home. I have an aunt that looks like her, got a problem with that?:|

 

[sharkmouth]

hey, hey...now you're hitting too close to home. I have an aunt that looks like her, got a problem with that?:|

Is that the one thats deaf in one ear & blind in the other?

 

[AJ Soundfield]

Blind tests are impossible for long periods.

Hi Jerome,

As has been pointed out previously, this is completely incorrect. There is no time limit for a blind test whatsoever. It can last two years if you wish. The only thing "blind" means, is that you don't know what's playing. That's it.

Now let's say you make a blind test of component A and B while listening a full day for each unit. This test is biased by the fact that we are not in the same state 2 days apart and our auditive memory is not good after a few minutes. Forget about days.

I don't quite follow you here. If that is true (and it is), how are adding even more variables/biases in (sight, volume changes, reputation, etc, etc.) on top of mood/perception changes, going to help or make it less biased?

The best way to know which unit fits you best, is to listen to one unit during 2 to 4 weeks. Then switch to the other. And see how it goes.

That can be done blind. Unfortunately, it will only tell you one thing. How it sounds to your ears.
As I noted previously, there is far more to component choice/enjoyment/ownership than that.
That applies to us all, hence the huge variety of choices we make.

cheers,

AJ

 

[AJ Soundfield]

The methodology can be very useful. However, it shouldn't be looked upon as the only valid way of deducting the performance of a component.

Absolutely. I know I don't listen blind at home. These things should please all your senses.

cheers,

AJ

 

[mep]

hey, hey...now you're hitting too close to home. I have an aunt that looks like her, got a problem with that?:|

Nope. As long as you love her, that's all that matters.

 

[BlueFox]

Just realize the smiley indicated the comment was in jest.

 

[BlueFox]

My stereophile arrived today, so now I can read the article. I don't understand hw some are able their copies a week earlier, unless they stagger the mailing.

 

[Mike]

My stereophile arrived today, so now I can read the article. I don't understand hw some are able their copies a week earlier, unless they stagger the mailing.

They gave them out for free two weeks ago at RMAF.


Sent from my iPhone using Tapatalk

 

[Myles B. Astor]

No doubt many people here have very strong feelings and opinions on the usefulness and application of blind listening testing when reviewing audio components. Let me say this first. If you enjoy doing double blind tests, sit back and knock yourself out. (does anyone know anybody who has purchased their system based on blind testing?) But be aware of and research the underlying science rather than relying on someone else’s opinions in a summary article. Blind testing has always been a paradoxical situation, however, with backers calling it “science- or evidence-based ;” but truthfully and at times willfully, however, proponents of blind testing ignore the real science, biology, physiology, neuroscience, testing science, etc. that’s been in the literature for decades. In short, blind testing is simply telling us what we already know: short term memory (auditory) is highly unreliable (just ask the police at a crime site or do that old test of passing a message from the front to back of a line.) Now let’s look at the real science:

1) Inter-aural hearing differences: Researchers have for years shown that inter-aural hearing differences among the population swamps out any effort to statistically validate any listening test (a very good review piece on the subject appeared several years ago in Scientific American Mind.)

2) Perception and the inverse U curve: Perception goes down as the magnitude of the test goes up. In other words, a musician wants to be at the far left of the inverse U curve being able to perceive as much information as possible while performing; on the other hand, a weightlifter with only one task needs minimal perceptive properties to concentrate on one thing. Or take blitzing that NFL QB. One is cutting down the time available for decision making making the athlete more prone to making a mistake. Simply, you are reducing his perceptual abilities. And the simple act of taking a “test” reduces one perceptual ability. Perhaps, a blind test could have more validity if 1) the tester asked his test group to concentrate on one particular attribute or 2) chose specific (rather than familiar or random) types of music to test the DUT.

3) Adaptation: Long known about and originally written about by the late and renowned Canadian physiologist Hans Selye. In short, every stressor, be it weight training, alcohol, heat, cold, etc. has a specific and long term effect on the body (eg. a genetic change). In Selye’s model, there is an initial alarm phase followed by roughly a six week period of adaption (or in the case of say weightlifting, getting stronger). The same is true for that new component; it takes time for the brain/nervous system to adapt to this new stressor/music.

4) Short term memory and information processing: Short term memory is notoriously unreliable because it has both limited “hard disc space” and involves serial rather than parallel processing (not to mention we still don’t understand to this day how and/or why short term memories (or unconscious activity) are converted into long term memories). This is always a good test for effects of short term memory and its clogging points. Watch a basketball player make an initial fake and then follow that up within 100 ms. with a second move. If executed properly, the defender’s brain will “freeze” because the offensive player has overwhelmed the short term memory disc space and processing ability.

5) Neuroscientists constantly refer to the dual nature of the human brain and in particular the primitive, limbic system vs the newer, evolutionarily wise, frontal cortex. (See the book written by Jonathan Haider entitled The Happiness Hypothesis. The bottom line: the older limbic system controls our very powerful flight or fight response. Ergo, what our brain searches for initially when we hear music played through new gear is more related to the clues our brain needs for deciphering if something dangerous is present eg. detail at certain frequencies. It’s my experience in fact, that those components—or even parts like capacitors-- that initially sound good, don’t hold up with long term listening. And conversely, I often find those components that produce a ho-hum response with early listening often with extended listening sound much more like music.


I’m going to leave treatment of stats and methodology choices to another time.

 

[Mike]

Interesting Myles. Thanks for showing some interesting arguments against the benefits of blind testing.

I enjoy it for fun, and as you said, would likely never trust it solely for buying a piece of gear. Too many factors. But the tests are fun.


Sent from my iPhone using Tapatalk

 

[Myles B. Astor]

Interesting Myles. Thanks for showing some interesting arguments against the benefits of blind testing.

I enjoy it for fun, and as you said, would likely never trust it solely for buying a piece of gear. Too many factors. But the tests are fun.


Sent from my iPhone using Tapatalk

One can obviously make arguments against each form of testing. I think, however, that the benefits of careful, long term, well thought out listening sessions outweighs any arguments against prejudice.

 

[rbbert]

I don't know if you read Art Dudley's article, Myles, but none ​of the effects and observations you posted were mentioned. And as I posted earlier, there were more problems with the article than just his comments on blind testing.

 

[Myles B. Astor]

I don't know if you read Art Dudley's article, Myles, but none ​of the effects and observations you posted were mentioned. And as I posted earlier, there were more problems with the article than just his comments on blind testing.

Yes the above was simply my viewpoint on the subject from a scientific/audiophile perspective. I think that sometimes one has to reconcile my hobby with my background. (I hate conflict.) Or call it a brain dump from the rationalizing part of the brain. Whatever, hope it is food for thought on the matter.

 

[rbbert]

Yes the above was simply my viewpoint on the subject from a scientific/audiophile perspective. I think that sometimes one has to reconcile my hobby with my background. (I hate conflict.) Or call it a brain dump from the rationalizing part of the brain. Whatever, hope it is food for thought on the matter.

And here's the thing; you put forth some reasonable and well-thought out arguments against rapid-switching short term blind testing. Art merely wrote some facile and superficial comments about blind testing before going further out into the objectivist vs. subjectivist debate, somewhat mischaracterizing both sides of that as well. And that's my problem with this month's column and AD's columns in general for the past year or so. Where once (back in the '90's, say) Art wrote well about audio topics to which he had apparently given significant thought, today he is writing about audio topics to which he has apparently given little thought. Either that or he is expressing himself poorly, in which case it's not appropriate for him to still be writing professionally.

More specifically addressing your points, though, I don't think you have presented good arguments about the potential usefulness of longer term blind listening. Logistically a much more difficult problem, certainly, but not insurmountable at least for some component types.

 

[Jerome W]

Hi Jerome,

As has been pointed out previously, this is completely incorrect. There is no time limit for a blind test whatsoever. It can last two years if you wish. The only thing "blind" means, is that you don't know what's playing. That's it.


I don't quite follow you here. If that is true (and it is), how are adding even more variables/biases in (sight, volume changes, reputation, etc, etc.) on top of mood/perception changes, going to help or make it less biased?


That can be done blind. Unfortunately, it will only tell you one thing. How it sounds to your ears.
As I noted previously, there is far more to component choice/enjoyment/ownership than that.
That applies to us all, hence the huge variety of choices we make.

cheers,

AJ

So you could do blind tests over periods of weeks ? You could listen to A during 2 weeks and then switch to B for two more weeks, without knowing at any time what are A and B ??? How come ?
With a curtain in front of the system and someone always here to turn it on ?

Of course we are talking about best sound to one's ears. There is no absolute in audio. Only personal preferences.


Envoyé de mon iPhone à l'aide de Tapatalk

 

[KeithR]

Thanks for your detailed comments, Myles.

I would agree with you that things that make the most difference right away are often the components I sell the first. Go figure! I'm now very suspicious of such components and give them a week or two before making any conclusions. But I also don't think that 600 hours is required to hear a component in its glory- at that point your mind has very much "adapted" to the component as you say.

 

[AJ Soundfield]

So you could do blind tests over periods of weeks ? You could listen to A during 2 weeks and then switch to B for two more weeks, without knowing at any time what are A and B?

Sure. Or longer. As long as you don't know which is which throughout the listening, it's blind.

How come ? With a curtain in front of the system and someone always here to turn it on ?

That's one possibility, but not always necessary. "Blind" doesn't necessarily mean you can't see, it means know. Could be as simple as hooking both to a high quality pre with Input 1 and Input2. You could have the whole thing in full view. Just no peeking at the cabling to see which goes to which, or any "pop" sound or flashing lights depending on which was switched to, etc, i.e. no "tells" to identify which was playing.

Of course we are talking about best sound to one's ears.

Yes, precisely. That is by definition, what a blind audio test is. Sound>ears. No looks, no price, not what you read someone else say, etc, etc. Just sound>ears. That's all you have to judge with. Can be quite an eye opener! Or a can opener.:lol:

There is no absolute in audio. Only personal preferences.

Bingo. That's why we all own different stuff. I don't listen for pleasure at home, blind. I allow all my biases and senses to be the final arbiter of what pleases me. Be that price, looks, status, reviews, etc, etc., along with the sound>ears.
I suspect you do too.

cheers,

AJ

 

[mep]

And here's the thing; you put forth some reasonable and well-thought out arguments against rapid-switching short term blind testing. Art merely wrote some facile and superficial comments about blind testing before going further out into the objectivist vs. subjectivist debate, somewhat mischaracterizing both sides of that as well. And that's my problem with this month's column and AD's columns in general for the past year or so. Where once (back in the '90's, say) Art wrote well about audio topics to which he had apparently given significant thought, today he is writing about audio topics to which he has apparently given little thought. Either that or he is expressing himself poorly, in which case it's not appropriate for him to still be writing professionally.

More specifically addressing your points, though, I don't think you have presented good arguments about the potential usefulness of longer term blind listening. Logistically a much more difficult problem, certainly, but not insurmountable at least for some component types.

I want to go on record and say that I think that Myles' post above was brilliantly written and hit a home run as far as I'm concerned. I think that blind listening is talked about 1,000,000,000 times more often than it is ever practiced. I also think that some people who claim to be its biggest proponent have never actually participated in a blind test-it just sounds good to talk about it as a core belief system. Sort of like motherhood and apple pie. Frankly, I'm tired of people who blather on endlessly about doing DBT or blind tests in order to make choices about what gear they buy because it's 99% nonsense or pure fantasy because it never really happens. I'm pretty sure the vast majority of us always know exactly what components we are listening to whether it's at our house, a friend's house, an audio show, or a dealer.