Expectation Bias - so demonstrated..........

[bryans]

I actually directed my commnets to Bryans, but I appreciate your input. My only point is that if you have friends/guests/visitors over to hear your system and they claim to hear things in your system that you aren't, I wouldn't dismiss their claims out of hand. It doesn't mean you personally find their comments relative, but if you have a quorum of people over time that make similar comments about what they are hearing even though you don't, they may have a point.

Yes if you have a group of people over and they all hear the same thing, good for them. If I don't hear what they hear that is also OK.

 

[treitz3]

From the article: "The membership consists of highly experienced recording engineers, equipment designers, equipment manufacturers, and more recently, other professionals working in the video and television industries. Membership is by invitation only: you must be a bona fide audio professional with experience and references to be considered for membership. ...."

Sounds like you are not any of the above...

Other than designing my own system? No. I'd personally trust my ears over theirs anyways.

Tom

 

[Mr Peabody]

There was a number in the group who accurately admit they didn't hear any difference, so there's no absolutes. Meaning none of you know which group I'd be in or vice versa.

 

[SCAudiophile]

I actually directed my commnets to Bryans, but I appreciate your input. My only point is that if you have friends/guests/visitors over to hear your system and they claim to hear things in your system that you aren't, I wouldn't dismiss their claims out of hand. It doesn't mean you personally find their comments relative, but if you have a quorum of people over time that make similar comments about what they are hearing even though you don't, they may have a point.

Good feedback, thank you. I definitely respect their right to hear and talk it through. If I do not hear it the same way, however, I will not push to hear it imagining I heard it wrongly to begin with. To be clear, if someone hears something I did not at all or hears some nuance in an interesting and new way, I would (and have) accepted that and went on with a better listening appreciation.

Sent from my SM-G998U using Tapatalk

 

[nicoff]

...
Both camps have expectation bias.

You are right! Expectation Bias can occur in the subjectivists camp (who only trust their ears) and the objectivists (who rely on science).
Everyone has the right to their own opinion, but not to his own facts.

 

[Mikado463]

You are right! Expectation Bias can occur in the subjectivists camp (who only trust their ears) and the objectivists (who rely on science).
Everyone has the right to their own opinion, but not to his own facts.

agreed

 

[Mr Peabody]

Most manufacturers who provide quality gear reveal in interviews they rely on both measurements and listening.

After reading the article linked to in the OP I clicked on another article at the bottom that talked about speaker measurements. It was pointing out that two speakers with similar flat response plots sounded quite different from one another. The article goes on but that's the bottomline.

I knew way back when looking at Mac specs they showed 1% distortion and receivers showed .000n% yet the Mac was much better to listen to in my opinion.

At the end of the day if you can't trust your ears....... I mean if you want to buy on a spec sheet and listen because you think the paper tells you it sounds good that's up to you. What is it they say these days, you do you. That doesn't work for me personally, when my ears tell me I'm not happy I find they are correct.

 

[Mikado463]

Most manufacturers who provide quality gear reveal in interviews they rely on both measurements and listening.

After reading the article linked to in the OP I clicked on another article at the bottom that talked about speaker measurements. It was pointing out that two speakers with similar flat response plots sounded quite different from one another. The article goes on but that's the bottomline.

I knew way back when looking at Mac specs they showed 1% distortion and receivers showed .000n% yet the Mac was much better to listen to in my opinion.

At the end of the day if you can't trust your ears....... I mean if you want to buy on a spec sheet and listen because you think the paper tells you it sounds good that's up to you. What is it they say these days, you do you. That doesn't work for me personally, when my ears tell me I'm not happy I find they are correct.

Yep, exactly and whereas your ears probably wouldn't notice a 1% thd measurement they sure as heck would if it were excessive, say 5% or more !

Every piece of quality audio gear requires a 'good foundation' , thus the use of measurements. As said before it comes down to objective / subjective interpretation that makes one satisfied, all good ! :celebrate008_2:

 

[steve59]

IDK, maybe i'm a punk. When someone tells me what I can or can't do it rubs me wrong. I used to react in anger now i'm mostly amused. I listened to a pair of hi rez speakers while demoing 3 different dac's a lumin, Nagra, and forgot the 3rd. One of them sounds colored while I really can't tell which of the other 2 is the more expensive one. An expert on another site tells me my brain is tricking me and all dac's and formats sound the same since the 1980's as 16/44 is all our puny little brains can digest, or something. When I was younger I would have spent an hour or two trying to convince this person my experience was real. Back then I could recognise when someone had already made up their mind, but the cause was worth fighting for. These days i'm aware that I don't know what I don't know (maybe someday this person will get there) and instead of trying to convince folk there's audible variables in audio that science wouldn't have time or inclination to measure for (component matching)I just go find someone else to talk to.

As to this specific op Mr Peabody already reiterated the trick didn't fool everybody so what's the point?

 

[MattHooper]

IDK, maybe i'm a punk. When someone tells me what I can or can't do it rubs me wrong. I used to react in anger now i'm mostly amused.

I'm curious: is that how you react to the proposal that you may, in some cases, be fooling yourself as to what you think you hear?

If so, why would you react so negatively to some basic facts about human perception?

I listened to a pair of hi rez speakers while demoing 3 different dac's a lumin, Nagra, and forgot the 3rd. One of them sounds colored while I really can't tell which of the other 2 is the more expensive one. An expert on another site tells me my brain is tricking me and all dac's and formats sound the same since the 1980's as 16/44 is all our puny little brains can digest, or something.

It's hard to know to what degree, if any, there is some strawmanning of that other person's claims. I've seen "objectivist" types say that any properly designed DAC/CDP should have no "sound of it's own" and would be indistinguishable. That's not the same as declaring that all DACs/CDPs sound the same, since in principle one can nudge them to sound different. Many years ago I blind tested a Sony CDP, Meridian CDP and a Meitner bi-dat DAC and I could distinguish between them. As to why exactly - different filters, something about the analog design - I don't know.

(On the other hand I've also blind tested between some music server platforms, and saw some differences that seemed to be there vanish. Which is always a nice lesson in sighted bias).

When I was younger I would have spent an hour or two trying to convince this person my experience was real. Back then I could recognise when someone had already made up their mind, but the cause was worth fighting for. These days i'm aware that I don't know what I don't know (maybe someday this person will get there) and instead of trying to convince folk there's audible variables in audio that science wouldn't have time or inclination to measure for (component matching)I just go find someone else to talk to.

If you are aware of what you don't know, wouldn't that make you more cautious about some of your inferences? For instance, how much do you know about the problem of sighted bias when assessing sound? If you are extremely confident "I know what I hear"....even when it's not currently supported by the science you suggest that hasn't been established...isn't that a case of "not knowing the relevance of sighted bias?" In other words, you seem to be assuming that what you think you hear is true first, and that science wouldn't measure, rather than trying to establish you really can hear such things when controlling for sighted bias.

As I've said often before: no audiophile HAS to bother for a second with science, or engineering, or listening with controls for bias. BUT...if you are trying to be aware of what you don't know, or think other folks are wrong, maybe there are other things to consider.

As to this specific op Mr Peabody already reiterated the trick didn't fool everybody so what's the point?

The point is that this scenario demonstrates what has been known for a long time: people really can strongly imaging hearing things that aren't there. This is a lesson that purely subjective oriented audiophiles seem hard pressed to learn. Yes, some didn't hear a difference, but the majority DID think they heard a difference when there was none at all. And since people can demonstrably imagine sonic differences, it makes sense to admit you are just as human, and that you too may be imaging sonic differences in some cases where there is none. So if you are serious about trying to understand what is going on in vetting audio gear, then taking human biases seriously in your method makes sense.

Again...you don't have to do any of this. But the logic and reason is there.

 

[steve59]

Hi Matt. I read your post and you seem to extract a few things to fit a conversation I wasn't having. As an experiment would you consider pretending the folk you interact with all have first hand experience with expectation bias and have each found our own ways to protect ourselves from the sharks in the tank.

My specific remark questioning the variables in what we measure and what we hear was directed specifically at component matching and I'd be surprised to find anyone try to attempt that task. Happy holidays.

 

[MattHooper]

Hi Matt. I read your post and you seem to extract a few things to fit a conversation I wasn't having.

Ok, but notice I was asking what you meant, not asserting what you meant.

As an experiment would you consider pretending the folk you interact with all have first hand experience with expectation bias

That would go against all my experience with audiophiles, most of whom do not have a good grasp of how expectation bias works. Very, very few seem
to understand it. (Even when they think they do). That's why I might probe the issue.

and have each found our own ways to protect ourselves from the sharks in the tank.

Ok, but that's a bit of a red flag. What does "our own ways" mean? I've seen countless audiophiles who think "I'm not experiencing bias because of x, y and z...and they really aren't removing the bias effect at all. Of course, you may be an exception. If you don't mind: How do you do it?

My specific remark questioning the variables in what we measure and what we hear was directed specifically at component matching and I'd be surprised to find anyone try to attempt that task.

I'm not sure how that changes the general conversation. Most of the beliefs audiophile have, many of which succumb to expectation biases, revolve around "equipment matching." And you referenced someone else telling you that DACs should sound the same, but your experience differed. So....I think I'm following
the issue.

Please don't take my questions as trying to push you in to anything. I'm genuinely curious.

Happy holidays.

Same to you! Cheers!

 

[steve59]

From back to front. I actually started a thread on ASR asking why bother rating the dacs reviewed since it's been decided they all measure better than our threshold of hearing. and I need to find it for you because none of the answers argued or answered my question.
Expectation bias is one of the things learned early on shopping and demoing hifi because the law of diminishing returns gets so much steeper the more expensive things get. The way to get past it is to home demo with familiar music, not rush, but listen for differences, define them decide if they're an improvement, a lateral move or regression and then put a personal monetary value on it

Our own way. OK Speaking for myself and only the guys I know the way I deal with expectation bias for me comes from the size splash a product makes on forums like this one. I bought the salon 2 on expectation bias, the most I'd spent on speakers to date, every forum I asked said said go for it! I went for it, luckily used and when they weren't my cuppa I got my money back reselling them.

I read some, not all, but a fair share of the science based audio experts use the term audiophile as a way to corral gear lovers with too much money and too little brains to make an informed decision. Correct me if I'm wrong. OK a simple analogy. I built houses for 40 years, but I have no clue how to make it a home. I may not even fit the narrative for an audiophile. The perfect system is what the owner of it wants it to be. If a guy loves the over the top sound of his local movie theater his ears will take him there. Maybe there's guys that can spend a million bucks on a stereo and the only difference between his kit and the benchmark kit is $25k and 1/2 aluminum case, god bless. The biggest difference between the science based group and hifi lover group is the hifi guy that's telling me his new cables really brought his stereo to a new level isn't trying to tell me my kit is crap and I'm a fool for not knowing better, he's just sharing his passion for the hobby. I haven't found a salesman yet try to sell me cables.

So you correctly pasted my quote I'm old enough to know IDK what IDK, but when you followed up with I know what I don't know it' takes a different meaning, then farther on you quote me as saying I know what I hear, leading with If you're extremely confident.This is your implication NOT my statement. I'm specifically referring to component matching to the point that at this very moment I could swap out a couple amps and 10 out of 10 people would be able to hear the difference. Regarding component matching when on that specific topic I will clarify that when shopping at a price I couldn't tell the budget amps apart. I know you guys are on a mission to educate the unknowing and you probably do, but I learned most of these lessons the hard way 30+ years ago


It's late, I'm struggling to form sentences so will pick up later. I can't even proof it tonight so. goodnight

 

[AJ Soundfield]

After reading the article linked to in the OP I clicked on another article at the bottom that talked about speaker measurements. It was pointing out that two speakers with similar flat response plots sounded quite different from one another. The article goes on but that's the bottomline.

No that is technical ignorance, because speakers radiate in every direction 3D and don't have "a frequency response" like electronic components. A distance and azimuth must be specific where "flat response" is measured.
If you link an actual article I'd gladly debunk such nonsensical beliefs.

 

[steve59]

Ok so rereading from my first post and beginning with what I can or can't do rubs me wrong. I'm not saying all science is BS. I challenge the blanket statement that all dacs measurable differences are above the ability for listeners to hear, at least in practice. Regarding the OP's test I wasn't there so can't say if it would satisfy what the experts would consider a controlled environment. As many times as I reread my post I can't find anything offensive, I think you might be looking for something that isn't there. Disclaimer, 10th grade dropout with d- in english. If you choose to analyze every sentence on it's own you will find fault. Consider the entire post.

 

[MattHooper]

Thanks for taking the time for a thoughtful reply steve!

Expectation bias is one of the things learned early on shopping and demoing hifi because the law of diminishing returns gets so much steeper the more expensive things get. The way to get past it is to home demo with familiar music, not rush, but listen for differences, define them decide if they're an improvement, a lateral move or regression and then put a personal monetary value on it

This is what I suspected. The method you just detailed would in no way whatsoever counteract sighted bias. It would feed right in to it. This is what I mean by the fact few audiophiles really understand the nature of bias. You can't just sort of "will yourself" out of it. That's why scientists - people who are the most cognizant of bias effects - know very well they can't even trust themselves and bias control is built in to their very method.

You can take two different DACs home, that sound indistinguishable, and with your method you can easily hear differences that aren't there. That's just a fact about human perception.

It doesn't mean per se that you are not hearing the differences you think you are, but it DOES mean that you haven't remotely weeded out sighted bias as a variable.

And an important point is that many audiophiles just think of bias effects in terms of "expectation bias." So they think "Well, I bought these 3 cables to try out and the reputation and expense and design of the most expensive one had me expecting it would sound best. But you know what? It was completely the opposite! It was the cheapest one that sounded best to me. That shows it wasn't expectation bias, and that I was really hearing those differences!"

Which is a misunderstanding. Yes, in that instance it was not expectation bias regarding the price/design - except for the expectation the cables WILL sound different! But the simple act of listening for differences will itself tend to cause us to hear differences even if there are none. That's just how our attention can work - we attend to certain details and we THINK we are attending in precisely the same way as always, but we are not machines and can not in fact totally control this. So we may seem to notice some difference or detail and then attribute that to "hey, I just put a different cable in, it must be the cable!"

In fact, you don't even necessarily need to be listening for differences. Your mood, your attention, may just be a bit different one day and "hey, that trumpet seems smoother, more burnished than I remember!" And then the natural inclincation is to think "what did I change in my system? Oh yeah! That new cable...those new footers under my CDP....the cable risers...the...." it can be anything. This is why literally every audio tweak anyone has ever made up, no matter how utterly improbably or impossible it's claims, has testimonials for it's efficacy.

The mind is an amazing thing. As I like to say "If people can imagine that aliens are visiting and anally probing them in bed, I'm pretty sure an audiophile can imagine a bit less midrange glare."

Our own way. OK Speaking for myself and only the guys I know the way I deal with expectation bias for me comes from the size splash a product makes on forums like this one. I bought the salon 2 on expectation bias, the most I'd spent on speakers to date, every forum I asked said said go for it! I went for it, luckily used and when they weren't my cuppa I got my money back reselling them.

I empathize. In my big speaker search a while back I auditioned Revel (as well as some other ASR-loved products, like the active Kii Audio 3s). They weren't my cuppa either. It does not however speak to how to address the issue of bias effects though.

Given this is a thread ABOUT bias effects, we are discussing it here. But no audiophile has to care about it, we can practice the hobby as we wish.

The biggest difference between the science based group and hifi lover group is the hifi guy that's telling me his new cables really brought his stereo to a new level isn't trying to tell me my kit is crap and I'm a fool for not knowing better, he's just sharing his passion for the hobby.

I have some level of agreement there. There is a tendency among some of the sciency-engineering crowd to mock "audiophools" for their purchases. It's something I've tried to push back upon (but have also sometimes fallen in to myself...I ain't perfect!).

On the other hand, it goes both ways. On the heavily subjective-based forums (which is where I've spend most of my time in this hobby) if I dare give voice to my own skeptical view of a product being discussed, then this often brings on ad hominem attacks - your gear must suck, you must be deaf, why are you trying to ruin things for everyone, go home troll!" etc. This is the reaction to one side, that holds firm beliefs, to someone just voicing a contrasting opinion! They view it as some personal threat. So, humans are gonna human...

Cheers.

 

[steve59]

I can't speak to cables and regarding dacs other than a pair of persona 7f's revealing some midrange glare from a $3k dac when compared to a couple others in the shop my personal stand so far is recording quality is less consistent than dac playback and I question why asr bothers to rate them. Regarding expectation bias I still recognise it as a human tendency, not epidemic. Meaning sure, some people hear what they want to hear, but not everybody and not all the time, just another bump in the road and i'm happy to agree to disagree on that. It is possible for 2 people to be right at the same time... I've been married 34 years

 

[JCS123]

I may be in the minority here, but I find this thread very useful in that it challenges my beliefs. The posts by Matt Hooper have been enlightening and I dare say, more than respectful.

I have found that ASR’s measurements don’t align with my own listening experiences with multiple amps that I’ve owned and enjoyed immensely, but I’ve also found that John Atkinson’s measurements are very useful when considering auditioning amps.

So maybe the morale to this story is that some ways of measuring might be more predictive of sound than others TO ME. Some measurements are very useful TO ME. For example: If a sub measures down to 20Hz, it is highly likely that it will also produce deeper sounding bass to my ears than a sub that only goes down to 35Hz. Another thing that I’ve noticed IN GENERAL: a higher SNR usually means better sound TO MY EARS. These are some very loose generalizations based on limited experience, but they are real and they prove TO ME that there are instances of measurements supporting my observations.

For years now, I have been intrigued by the ABX comparator from Van Alstine, which allows you to switch between components or cables with a flick of a switch. Wondering if anyone here has experience with that kind of device. I could see it being very useful. I could hook two variables up, leave the room while a friend either leaves the switch where it is or flicks it, return to the room blindfolded and have them switch back and forth enough times for me to either identify differences or not.

 

[nicoff]

...It is possible for 2 people to be right at the same time... I've been married 34 years

That is a clear example of Expectation Bias. You THINK (i.e., expect) that you are right, but she knows better. Thank HER for keeping it going for 34 years.

 

[Mikado463]

That is a clear example of Expectation Bias. You THINK (i.e., expect) that you are right, but she knows better. Thank HER for keeping it going for 34 years.

Heck, Steve is still on his Honeymoon ! 48 years here :exciting: