Every component adds some sound, OK -- but some add a good deal less that others. Should we doubt that well-measuring equipment falls in the "less" category?
You are right the it's impossible to fully know what unaltered sound should sound like. For my part I deem that equipment that is maximally clean, detailed, and transparent provides the best clue.
How do you KNOW what adds more or less? You have no idea (I don't mean that in an argumentative way but in a factual way) - nor will you ever be able to know was captured at the mic because:
The recording mic added flavor (yes they choose different mic for different flavors);
The recording room added flavor:
The engineers and artists added flavor;
The speakers in the studio added flavor;
The transmission between the groups to the ultimate recording on a medium added flavor;
The EMI and RF interfere in your room adds flavor;
And every single piece of gear, cable, speaker and your own room added flavor ALONG WITH your own ears.
And 1000000% - measurements don't tell you a thing about how much anything adds to the sound. To think they do fundamentally misunderstands the very nature of measurements.
Chasing the fake unicorn of "neutrality" is a chase for those who prefer to intellectually analyze music rather than listen to it and simply enjoy it.
Mind you that is 100% OK if that is what people choose to do and I respect their pursuit of that. What ever they enjoy they should do.