- Thread Author
- #1
Why you can’t trust measurements
- Thread starter Mike
- Start date
I pretty much refuse to watch YouTube videos about anything (with the occasional exception of DIY home maintenance), so I am not sure exactly what is said here. Regardless, measurements that are properly performed can tell you whether a product meets its specifications, and sometimes whether or not it is well engineered. There is no evidence that measurements can tell you how a product sounds, although sometimes a large measured frequency response anomaly is noticeable, but of course that is only one aspect of sound or sound quality.
After more than 50 years of listening to music through home stereos it has never been clear to me why anyone would even think measurements tell you how a piece of equipment sounds. Frequency response, distortion and S/N+D were often helpful in evaluating tape deck performance, especially cassette decks, but even there 2 decks that measured (essentially) the same could sound quite different.
After more than 50 years of listening to music through home stereos it has never been clear to me why anyone would even think measurements tell you how a piece of equipment sounds. Frequency response, distortion and S/N+D were often helpful in evaluating tape deck performance, especially cassette decks, but even there 2 decks that measured (essentially) the same could sound quite different.
atmasphere
Active member
50 years ago a lot of the important measurements couldn't be made. Now they can.
The cassettes all sounded different (assuming they were calibrated correctly) because their electronics were not neutral. The semiconductors needed didn't exist in the 1970s and well into the 1980s.
W9TR
Active member
The audio industry has moved on from single sine or swept sine measurements, which are still used by most reviewers that still publish measurements.
In the case of ASR, the use of a single measurement @ 1 kHz is laughable.
In the case of ASR, the use of a single measurement @ 1 kHz is laughable.
TL;DR?
"So what am I trying to say will all this? Should we throw away measurements entirely and go back to word of mouth subjective feedback? No. Measurements can indeed tell us a lot. [.....] use measurements as guidelines[...]"
The author of this video believes in measurements. He is actually making the case for consumers to judiciously use measurements; not to forgo them. A more apt title would be "Why you can't trust measurements ONLY" (but I am sure that he gets more clicks doing it the way he did it). He is warning consumers that manufacturers can tweak certain measurements to make their products rank better (nothing new here; car manufacturers have been found guilty for doing that).
BTW, this same guy took MQA and Tidal to task. He published music on Tidal to test MQA and proved that most of Tidal/MQA claims are not true. (His video is available on YouTube).
steve59
Member
ASR has many followers that will argue our ears/brains cannot be trusted and measurements are the only way to shop. I'm struggling to understand why people that don't trust their ears would spend 5 figures on components and I still haven't gotten anyone to suggest a measurement that indicates resolution differences in products. It's fascinating the herd mentality on that site makes discussion seem like interrogation.
A widespread comment on many audio boards boils down to: what is the level of enjoyment of those who have systems purchased using measurements as the primary criterion vs. those who have chosen based on actual listening comparisons?
An even bigger question is: which measurements actually matter (e.g., once THD, IMD, S/N+D fall below audible levels) and which if any relate to the component's sound?
AJ Soundfield
Well-known member
Because they trust looks, price, weight, country of origin, reviews, awards, etc, etc.
I highly doubt a single person spending 5 figures on components advocates for controlled/blind listening, which by definition, is what "trust ears" is. It's the only option in controlled listening.
Now in completely uncontrolled "listening", you get to trust all ones senses, priori knowledge, biases, preferences, etc, etc. No need whatsoever to "trust ears" or "just listen". Sit back and enjoy it all.
As it should be.
cheers,
AJ
steve59
Member
While you selected a sentence of my post I don't think you got the intent of my post. your last line got it, sit back and enjoy.
AJ Soundfield
Well-known member
I 100% got your post as I've seen countless identical ones before and my response is always the same, as above. Yep, agree, sit back and enjoy all your senses, pricetag, etc, as we all do.
Only time I need to "trust my ears" is when doing blind testing during speaker development, as I do vs KEF, Revels et al. Otherwise, I "listen" just like you and everyone else. For enjoyment. No "trust" needed.
cheers,
AJ
puroagave
Active member
I have no problem wth blind testing, I've participated in informal ABX comparos at audio shows, society meetings, etc. it has a purpose and can be enlightening. Lately Ive ramped up my culinary pursuits and acquired new utensiles, cookware etc. You can find knives for $25 that perform with perfect utility and will probably outlast me, but I like the $250 versions better; better steel, better made, etc. the high price for superioir products that perform no better then cheap(er) ones has been debated to death.
AJ Soundfield
Well-known member
I'd be ok with not blind testing knives
u-sound
Active member
imho no need for darkroom games.....the only way to judge is to take your time.
Speedskater
Member
I count 6 to 8 different tests at ASR.
AJ Soundfield
Well-known member
Zero listening. And that SINAD worship...
:rofl::rofl:
atmasphere
Active member
That depends on the component in question!
If an amplifier or preamp, the distortion spectra at 1 Watt is good. Also distortion spectra a 6dB below full power. Also distortion vs frequency at 1 Watt and -6db of full power. If the distortion overall is -105dB or better then the music can mask the distortion and the amp or preamp will be pretty neutral. Many people think that the distortion can be higher and it will still be inaudible. IME, that isn't true.
Here are some things to help out:
1) the ear assigns a tonality to all forms of distortion. As an example the 'sonic signature' of any amplifier is actually its distortion signature.
2) the ear uses higher ordered harmonics to sense sound pressure and so is keenly sensitive to their presence; it has about a 130dB range! The higher ordered harmonics as distortion are sensed as 'bright and harsh'.
3) the lower ordered harmonics (2nd and 3rd) can mask the presence of the higher ordered harmonics if the amplitude of the former is sufficient.
If you know these things then the measurements (if they exist as I described) can tell you a lot!
If you see distortion increasing with frequency (above 2-3KHz) that's a sign that the circuit is losing feedback at those higher frequencies. This can happen if the circuit lacks the Gain Bandwidth Product to support the feedback. The increased distortion will be perceived as brightness and harshness. We've been hearing solid state amps with this problem since the inception of transistors. In addition, if the circuit lacks the feedback needed to really clean things up (+30dB at all frequencies) it will sound harsh and bright since otherwise the feedback will cause distortion of its own thru the process of bifurcation at the feedback node, which is non-linear.
Tubes get around this a bit by having a pronounced 2nd or 3rd harmonic which can mask the presence of the higher orders. In this way they can sound smooth in the mids and highs even though they have more higher ordered harmonics than most solid state amps.
Supporting +30dB of feedback is pretty hard and the semiconductors needed for an amp to do that didn't exist in the 70s or 80s and just barely sometime in the 1990s. Plus the will to deal with this problem for the most part seemed to be absent with many solid state designers. So we have a lot of solid state amps that are harsh and bright as a result.
With class D amps its possible to apply enough feedback so there are class D amps now that lack the harshness of traditional AB amps. Such amps are usually of the self oscillating variety; so much feedback is applied to them that the amp goes into oscillation as soon as its turned on. The oscillation is then used as the switching frequency, killing two birds with one stone. Even then, if the distortion spectra isn't right, the amp can exhibit amusical characteristics. But they can be seen if the amp is tested on the bench as I described.
The way your post is worded makes it sound as if the distortion signature’s sonic characteristics are established fact, but is that really so or are these just empirical observations? Likewise the comment about -105 dB distortion being inaudible (due to masking?) but some think higher levels are still inaudible? I’m not necessarily doubting you, but empirical observations are not “fact” (evidence), although they are often regarded as such in engineering.
Speedskater
Member
Zero listening. And that SINAD worship...
What kind of a measurement is listening?
Do you mean statistical A/BX tests?
atmasphere
Active member
We've known since at least the 1930s how the ear responds to the various harmonics (see Radiotron Designer's Handbook, 3rd edition). Its also well known that if the circuit has a lot of IMD it will sound fatiguing- IOW harsh and bright.
Its been known for a long time how the 2nd harmonic contributes to 'warmth' (this is the ear converting distortion to a tonality). That should not be controversial- nor should it be that the higher orders contribute to harshness and brightness. Its also easy to demonstrate with very simple test equipment that the ear is using the higher ordered harmonics to sense sound pressure.
The tricky bit is that the lower orders can mask the higher orders, but not surprisingly, this works the same way in electronics as it does with loudspeakers. But this explains how a tube amp can sound smoother than a solid state amp even though its making more higher ordered harmonic distortion. Its masked, so the amp sounds smooth. Tube amps (assuming proper design) without feedback tend sound even smoother as they lack a lot of the higher orders and have a more prodigious 2nd or 3rd harmonic.
People don't trust measurements for simple reasons. First, most of the important measurements are not made or not published. 2nd, even if they are you have to know what they are saying and many people do not.