Why you can't trust measurements

On 4/23/2022 at 9:03 AM, pkane2001 said:

 

Probably even easier if you insult the owner of the site and accuse him of improprieties, like @GoldenOne had done publicly. I assume he'll never be allowed back on ASR. But it would've been an interesting conversation about testing methodologies and blind/ABX testing. By the way, Amir does offer his APx project files to anyone who asks. Can't say if he'll share them with those who he has personal beef with, though.

 

I don’t think this is quite fair. Amir and other ASR folks regularly accuse every other site of improprieties. If he can’t take it, he shouldn’t dish it out. Amir’s reviews were being reprinted in full by a Topping/SMSL distributor. Whether he knew it or not is up for debate. At the very least, he’s admitted that Topping sends him gear that he can keep for free, which is the type of thing that would be proof of bias for any other reviewer. Regardless of whether he needed to, @GoldenOne apologized and was still banned. 

There’s also the strange dynamic where Amir insists his subjective views on things like speakers are superior to others’ (and even superior to the measurements themselves), but everyone else’s measurements and subjective impressions are suspect, if not outright worthless. (His defenses of MQA were another odd tangent where measurements suddenly weren’t important.) So he clearly thinks subjective impressions matter…but only if they’re his. 
 

The crux of the issue, for me, is that measurements are neither everything nor nothing. They provide us with valuable information, but humans can make mistakes when conducting measurements, and there’s reasonable debate about which measurements truly matter, how to best present them, and how they relate to what we do (or don’t) hear. That’s what Golden was getting at, and I don’t quite understand Amir’s defensiveness at what seems like an undeniable, even banal, point. 

40 minutes ago, JoshM said:

I don’t think this is quite fair. Amir and other ASR folks regularly accuse every other site of improprieties. If he can’t take it, he shouldn’t dish it out. Amir’s reviews were being reprinted in full by a Topping/SMSL distributor. Whether he knew it or not is up for debate. At the very least, he’s admitted that Topping sends him gear that he can keep for free, which is the type of thing that would be proof of bias for any other reviewer. Regardless of whether he needed to, @GoldenOne apologized and was still banned. 

 

There’s also the strange dynamic where Amir insists his subjective views on things like speakers are superior to others’ (and even superior to the measurements themselves), but everyone else’s measurements and subjective impressions are suspect, if not outright worthless. (His defenses of MQA were another odd tangent where measurements suddenly weren’t important.) So he clearly thinks subjective impressions matter…but only if they’re his. 
 

The crux of the issue, for me, is that measurements are neither everything nor nothing. They provide us with valuable information, but humans can make mistakes when conducting measurements, and there’s reasonable debate about which measurements truly matter, how to best present them, and how they relate to what we do (or don’t) hear. That’s what Golden was getting at, and I don’t quite understand Amir’s defensiveness at what seems like an undeniable, even banal, point. 

Josh,

If you're commenting on my post, then I don't think it's either fair or unfair, as I didn't make any value judgements. In fact, I thought the banning of GoldenOne on ASR was overdone and inappropriate at the time. But there is too much mud slinging still going on with accusations without evidence (from both sides). The intolerance and the inability to see things from the other side is, in my view, the crux of the bigger problem, not measurements vs subjective opinions.

1 hour ago, pkane2001 said:

 

Josh,

 

If you're commenting on my post, then I don't think it's either fair or unfair, as I didn't make any value judgements. In fact, I thought the banning of GoldenOne on ASR was overdone and inappropriate at the time. But there is too much mud slinging still going on with accusations without evidence (from both sides). The intolerance and the inability to see things from the other side is, in my view, the crux of the bigger problem, not measurements vs subjective opinions.

 

That's fair. I read your response as, "It's not surprising given that he insulted Amir." My reaction was simply that Amir clearly doesn't apply the same standard to himself.

I think the issue is that much of what happens at ASR is "appeal to authority," rather than "science." This, for example, was just posted:

I don't know how anyone is supposed to react to that. As we've all seen during the pandemic, doctors disagree with one another. If I'd have listened to the doctor who once told me "We can't do anything about your pain besides give you opioids," I'd be in bad shape today. My mother had to diagnose herself with a rare disease that multiple doctors missed, likely saving her life. Others in that thread noted similar personal experiences with doctors, but Amir has no response to that.

There's a reason why appeals to authority are identified as a logical fallacy. Amir has justified MQA by saying "Well, are you Bob Stuart!?" over and over again. Yet simultaneously the expertise of someone like Mike Moffat is dismissed. However, if we're supposed to trust Stuart on MQA, why shouldn't we also trust Moffat on the Yggy? Appeal to authority is why Amir made such a big stink about @Archimago being anonymous. But either evidence speaks for itself, without reference to someone's name and CV, or we're not really dealing with science. 

There is, of course, some truth to the fact that how much you trust any evidence is at least partly based on the behavior and track record of the person presenting the evidence. We can't all replicate measurements at home because we can't all afford AP equipment (or have the time to conduct the measurements). But that's exactly one of @GoldenOne's points.

Ultimately, every ASR debate comes down to "Amir must be the unquestioned authority," which isn't really conducive to robust inquiry, scientific or otherwise. 

In my view, the more people doing measurements and checking each others' work, the better. But Amir's work is no more above criticism than anyone else's. (Indeed, based on track record, it's just as reasonable for someone to trust Amir less than someone like Arch or Golden!)

I've had run-ins with Amir ... he has a powerful need to have the last word, and will end up throwing everything he can possibly think of, so that he can come out on top ... when you realise the exchange has got to this point, the best option is to withdraw quietly; to continue will only cause the mud to get thicker, and more smelly ...

Well, the way Amir dismissed and banned ELAC designer. Andrew Jones, speaks volume about his arrogance. Andrew Jones was nothing but a gentlemen. Jones told Amir that his measurements were done above the maximum SPL if the speakers. Also, he offered to look at the speaker and ship back to him on his coin. Amir was a ponce and banned him.

1 hour ago, pkane2001 said:

 

Josh,

 

Doctors may disagree with one another, but I still won't take advice to drink bleach to kill the virus from someone with no background in medicine. 

 

People  come to ASR to read Amir's opinion and then disagree with him and fight him on it. In reality, (and to most knowledgeable folks) Amir's opinion isn't interesting. His measurements are. And while there are those who criticise his measurements, their opinions are just as irrelevant as Amir's unless they can produce measurements that demonstrate that Amir borked his. Arguing about his "biases" or his "preferences" makes sense only if you give any weight to his opinion and subjective parts of his reviews. Personally, I don't, and have no reason to defend these. In fact, when ASR was just starting, the reviews only had measurements. Readers insisted that Amir give his opinion and his subjective reviews. So, now, here we are, discussing opinions which are not objective, not scientific, and unimportant.

 

I disagree that we are completely helpless and can't reproduce any of Amir's measurements at home without major investment. Investment in time and learning, yes, but not necessarily money. I've been doing this for years, and others have done, as well. It takes a little ingenuity and the desire to find things out for yourself. But then, there are many others that would rather be told what to think and what to like. While I can understand their motivation, such as lack of time, or lack of interest, I find it much more less acceptable to be told to drink bleach by someone -- I'd much rather have science and measurements presented to me to make up my own mind than to listen to an opinion of someone "in authority".

 

You can see people disagreeing with Amir in nearly every review thread. Few have facts, most have opinions. Of course, I'd also like to have more independent measurements. More real information, more facts and less opinions is a very desirable outcome to my mind. Which is why I said I was disappointed when GoldenOne was banned from ASR. But, I can also understand why Amir did it and it had nothing to do with measurements or objectivism.

 

 

 

1 hour ago, pkane2001 said:

And while there are those who criticise his measurements, their opinions are just as irrelevant as Amir's unless they can produce measurements that demonstrate that Amir borked his.

 

Whilst I agree with a large part of what you wrote @pkane2001 I disagree with this bit. The Andrew Jones example that @botrytis has just given shows that its not necessary for AJ to come up with his own measurements to show that Amir's are erroneous.

21 minutes ago, Rexp said:

Please post on ASR more regularly than you do here, thanks!

He can't...

And done in such a gentlemanly way, too ... pity, it was indeed a forum for "sound, reasoned argument, and meaningful discussion" ... as we all know, .

2 hours ago, pkane2001 said:

 

Josh,

 

Doctors may disagree with one another, but I still won't take advice to drink bleach to kill the virus from someone with no background in medicine. 

 

People  come to ASR to read Amir's opinion and then disagree with him and fight him on it. In reality, (and to most knowledgeable folks) Amir's opinion isn't interesting. His measurements are. And while there are those who criticise his measurements, their opinions are just as irrelevant as Amir's unless they can produce measurements that demonstrate that Amir borked his. Arguing about his "biases" or his "preferences" makes sense only if you give any weight to his opinion and subjective parts of his reviews. Personally, I don't, and have no reason to defend these. In fact, when ASR was just starting, the reviews only had measurements. Readers insisted that Amir give his opinion and his subjective reviews. So, now, here we are, discussing opinions which are not objective, not scientific, and unimportant.

 

I disagree that we are completely helpless and can't reproduce any of Amir's measurements at home without major investment. Investment in time and learning, yes, but not necessarily money. I've been doing this for years, and others have done, as well. It takes a little ingenuity and the desire to find things out for yourself. But then, there are many others that would rather be told what to think and what to like. While I can understand their motivation, such as lack of time, or lack of interest, I find it much more less acceptable to be told to drink bleach by someone -- I'd much rather have science and measurements presented to me to make up my own mind than to listen to an opinion of someone "in authority".

 

You can see people disagreeing with Amir in nearly every review thread. Few have facts, most have opinions. Of course, I'd also like to have more independent measurements. More real information, more facts and less opinions is a very desirable outcome to my mind. Which is why I said I was disappointed when GoldenOne was banned from ASR. But, I can also understand why Amir did it and it had nothing to do with measurements or objectivism.

People can do some measurements at home. But if the response is, “The APx555 is better” or “You can’t trust [Jude/Marv/Bob/Arch/Golden…]’s measurements, only mine,” then there’s not much left to say. 
 

There have, of course, been instances when people have produced measurement that conflicted with Amir’s, and it hasn’t mattered. The (in)famous Yggy “droop” comes to mind. Multiple people measured Yggys and didn’t find it. (I had one at the time and didn’t find it myself.) There have also been much less flattering measurements of various Topping products elsewhere. Was this borking, unit-to-unit variation, something else? I have no idea.
 

Andrew Jones was already mentioned, and that’s an example that points to the subjective decisions involved in conducting a measurement, such as, “What’s the right SPL to highlight?”

Like you, I totally discount his subjective views. But he does go to great lengths to argue that his listening impressions are better than others’. I find this silly, particularly given that according to his own account he didn’t score very well on Harman’s How to Listen. That said, it does at least highlight that, in reality, some people are (due to hearing ability, training, or both) better listeners than others. That’s a whole other can of worms, though, which raises lots of interesting questions.

While it’s only in the past few years that Amir has given explicit reviews, from the beginning he passed judgment on (and insulted) companies and products. For example, his 2017 Schiit Modi 2 review said, “Either they don't measure their DACs, don't care or don't know how to design a clean DAC. Yes, -85 db distortion products is not likely audible and hence the reason they get away with such poor performance. But the engineer in me wants to throw up on it.”
 

Besides the language involved, there are all kinds of claims in there related to what’s audible and why we should still care about something that supposedly isn’t audible. It’s that tone and condescension that rub many people the wrong way, not measurements per se. This is why it’s bad that Amir (and some of his critics) conflate his work with measurements, objectivism, or any other label. It muddies what we’re actually talking about, and creates a situation where someone like Golden can’t make his wide-ranging video without Amir taking it as a personal attack.

1 hour ago, botrytis said:

It was sanitized for sure. If you read the rules of that forum, one rule is like do not argue with the owner of the forum. No questioning is allowed.

 

I was on another forum, since 2003, and had about 32-35K posts. I was banned because one mod didn't like my posts. He banned me and also erased all of my posts. It was like I never existed. I was happy to walk away from that forum. It was another Audio forum.

Yeah, I think that's why people start their own forums. I gravitate to ASR not because it's perfect or because of Amir, but because of other technical folks that participate there. I love to experiment and tinker, and that's where I hear get to read others think and if there's something I could've done better. That's how I learn.

And yes, I've been banned from other audio forums before, also :) and this despite the fact that I own and moderate multiple forums with many thousands of members for another high-end, technical hobby. I've banned a total of a couple of people in the 10+ years on those other forums, primarily because they went haywire and couldn't stop with the personal insults and cursing out others, never because someone disagreed with me.

41 minutes ago, Speedskater said:

It seems that some technical threads at 'ASR' quickly get emotional.  Rather than discussing different viewpoints and 'best engineering practices' , the threads turn into I was right and you were wrong arguments.

Yes.  This is especially ironic since the real value of a audiophile forum is learning things that you did not know, i.e. someone showing you that you were wrong...

On 4/24/2022 at 9:39 PM, JoshM said:

 

That's fair. I read your response as, "It's not surprising given that he insulted Amir." My reaction was simply that Amir clearly doesn't apply the same standard to himself.

 

I think the issue is that much of what happens at ASR is "appeal to authority," rather than "science." This, for example, was just posted:

 

 

 

I don't know how anyone is supposed to react to that. As we've all seen during the pandemic, doctors disagree with one another. If I'd have listened to the doctor who once told me "We can't do anything about your pain besides give you opioids," I'd be in bad shape today. My mother had to diagnose herself with a rare disease that multiple doctors missed, likely saving her life. Others in that thread noted similar personal experiences with doctors, but Amir has no response to that.

 

There's a reason why appeals to authority are identified as a logical fallacy. Amir has justified MQA by saying "Well, are you Bob Stuart!?" over and over again. Yet simultaneously the expertise of someone like Mike Moffat is dismissed. However, if we're supposed to trust Stuart on MQA, why shouldn't we also trust Moffat on the Yggy? Appeal to authority is why Amir made such a big stink about @Archimago being anonymous. But either evidence speaks for itself, without reference to someone's name and CV, or we're not really dealing with science. 

 

There is, of course, some truth to the fact that how much you trust any evidence is at least partly based on the behavior and track record of the person presenting the evidence. We can't all replicate measurements at home because we can't all afford AP equipment (or have the time to conduct the measurements). But that's exactly one of @GoldenOne's points.

 

Ultimately, every ASR debate comes down to "Amir must be the unquestioned authority," which isn't really conducive to robust inquiry, scientific or otherwise. 

 

In my view, the more people doing measurements and checking each others' work, the better. But Amir's work is no more above criticism than anyone else's. (Indeed, based on track record, it's just as reasonable for someone to trust Amir less than someone like Arch or Golden!)

ASR has become a church of "Cience", with Amir its sumo pontífice and Toole's book its bible.