I think these two responses from Bromo pretty much said it all:
Not only did they not run an actual ABX test, the number of subjects (6) wouldn't give statistically significant results.
But these guys know all of that - they know how to run a proper ABX test. They also know how much data they need for something that's statistically significant. (And I am sure from their comments if they thought they needed help they'd have an army of people there to help them).
But ... I suspect they came to similar conclusions to the author of this blog post, and ended up "throwing the test" -- funny how they put in a crummy soundcard instead of the better one in the Macbook pro (though, frankly both of those wouldn't be as revealing as an actual dedicated device), and they played on the subjects sense of insecurity by allowing a "I don't know" result (or allowed a hand selected few a way out).
These guys are publicity hounds, with all the hype if they came to conclusions that their readership wouldn't agree with. Well I have seen how they treat interlopers who don't tow the line with them. Imagine an evidence based apostasy would do the Ars and the Randi crew?
They don't have the luxury of validating anything, without losing their core group. They will not and cannot be objective, and clearly can throw a test.
Read more at
Ars Technica And The Million Dollar Cock-up | AudioStream
_________________________________________
The measurement was incomplete.
They confirmed that the cable conformed to the Cat-6a standard (yay! And nobody should be in doubt). And despite their badying about the term "marginal" they don't really talk about how the "control" cable they used in their abortive not-ABX test got blown out of the water.
The BlueJeans cable (hello! where did this come from?) was brought in and didn't perform as well as the AQ Vodka, actually if you look at the data. BlueJeans was honest, but Ars kind of did a sleight of hand and changed their comparison to the Bluejeans cable, and not the cable used in the test. Abracadabra!
But, and this is important, they didn't test for RF immunity and emissions. That would directly test the Audioquest claims about their shielding, especially the dissipative shielding. I will also mention that the Randi cable and the BlueJeans ones are unshielded. There are basic controlled standards that could be adapted to evaluate the effectiveness of the shielding, and given peopel an idea of the impact of routing the cable near sensitive electronics. But ... they didn't bother. (Not for nothing it would cost about $2-5k to do the test. But if you are saying you want to debunk something, you actually are then under the burden of actually verifying or disproving actual claims).
So, yes, they changed the test, introduced a new cable, and did a little shell game so they wouldn't be forced to admit tha the AQ cable blew the pants off of the "cheapo no-name Ethernet cable" that they were claiming was just fine.
In case you weren't watching or missed the bit of sleight of hand.
Read more at
Ars Technica And The Million Dollar Cock-up | AudioStream
That's of course if you even subscribe to ABX testing which is in itself a cock up. (as my ex-mentor from Wales also said.)