Evaluating Your Tools
Zen and the Art of Recording
© 2014 All Rights Reserved
Evaluating and Choosing Your Tools
A test is only as good as its design. An ill-conceived test will result in a flawed conclusion. You can’t judge a piece of gear casually on program material in which you have no stake in the outcome. All of your decisions regarding your sonic evaluations of gear must be process-driven, and performed on a recording in which you are the arbiter of all that’s good.
You’ll learn nothing useful by merely strapping a stereo compressor on someone else’s finished mix. All you’ll manage to accomplish is a comparison between a mix as it was intended, and a mix in which the balances have changed due to the introduction of the compressor. Therefore, you should evaluate gear based on how it affects your decision-making process. You have to use the compressor in context of your own mix in order to make an appropriate evaluation.
When you test a microphone by inviting a friend to sing a cappella into it, you have no context with which to judge the tone, musically speaking. Such a test will tell you absolutely nothing in regards to the overall versatility of the mic. You have no idea how that microphone will react to any other source, nor do you have a specific production with which to judge the sound of the vocal. The best time to evaluate gear is when you’re most susceptible to the overall emotional impact of the music.
Blind ABX comparisons are useful at times, but this whole Internet Chatter Class concept that you have to perform all comparisons blind for fear of expectation bias is ludicrous. Even the most subtle sonic difference can change how a track makes you feel. That is of far more consequence than the specific attributes of the tone being imparted. A subtle difference on completed program material has no consequence. A subtle difference on a work of personal or professional importance is the very definition of consequence.
We have all turned the knob on a disengaged EQ only to perceive a tonal change that doesn’t exist anywhere but in our heads. Clearly, our brains are subject to some expectation bias, and I wouldn’t argue otherwise. But to suggest that we are completely incapable of making judgments without the influence of expectation doesn’t seem to correlate with reality. Were that the case, it would be nearly impossible to make the thousands of tiny judgments we make in just a single day of recording.
Expectation bias really only comes into play on those occasions when the difference between A and B is so remarkably subtle that you’re not entirely sure whether it matters. And while adding a touch of high end on a disengaged EQ can dupe even the most talented engineer, cranking an engaged high-end boost knob by 12 dB will quickly reveal we’re turning a dead knob.
Do you believe that a 12 dB difference of a relevant EQ frequency requires an ABX test as your client runs from the room in pain? No one would miss that. Clearly, there’s a line.
Further complicating matters is the method in which the A/B is performed. Switching between the A and the B in quick succession completely removes emotional impact from the evaluation. You’re recording music. All of your decisions regarding sound should relate directly to how it affects the music. Balance decisions are not made purely in the here and now, but also within the context of what comes next in a production.
Music won’t properly evoke an emotional response unless there’s context. A chorus that dynamically explodes can offer great excitement for the listener. But that dynamic exists only within the context of the previous section. When you perform an A/B in quick succession, you remove the effect of musical dynamic and contrast from the evaluation, and those are actually more important than the internal balances at any given moment. Sonic differences can appear subtle, or even irrelevant, when the music is removed from the equation. It is precisely this kind of inattentiveness to emotional impact that likely explains the development of the MP3.
Performed as an A/B in quick succession, most people would be unable to pick between a 192 KHz MP3 and a 16-bit, 44.1 kHz WAVE file. If, however, you play your test subject a full Wave file of her favorite song in its entirety, followed by the MP3, she will be far more likely to notice the degradation of quality caused by the compressed format. Further, your test subject will not describe the difference in terms of sound, but rather how the track makes her feel. Why? Because that’s how people listen to music. I encourage you to try this experiment for yourself.
Expectation bias is a two-way street. To suggest that my bias would coerce me into the more expensive tool in a shootout (as some in the Internet Chatter Class suggest) ignores the possibility that I might be looking for the cheap way out. And there are times that I’m looking for the cheap way out, yet somehow, that particular strain of expectation bias rarely saves me money.
To complicate matters, a subtle difference could very well blossom into a significant problem under the right circumstances. The negative effects of 32 subpar converters will compound, and if you compare multitrack converters by running program through the first two channels only to declare the difference as “subtle,” your conclusion will be just as flawed as the test itself. That evaluation of subtle can add up to not-so-subtle once multiplied by 16.
When evaluating gear, you have to use it in the heat of battle, and not just momentarily, but in a variety of circumstances. This is the only reasonable way to evaluate how a piece of gear reacts. If you were to strap an analog Teletronix LA2A onto the kik drum, you might not think much of the unit, as it’s often a bit slow for the job. That doesn’t mean it’s a lousy limiter. It’s a staple. It just happens to be an unusual choice for that particular application, on that particular recording, on that particular day. Nothing more, nothing less.
You’ll notice I put in “on that particular day.” There will be decisions that you make as you progress in recording that will offer pure magic on one day, only to be deemed pure shit the next. Welcome to the whacky world of recording. That never really changes. You’re human, and therefore susceptible to making poor decisions due to any number of factors, including sleep deprivation and hunger.
There are good and bad specimens of all gear, but even the most atrocious gear can be useful. It’s just a matter of how long you have to wait before you discover where it shines. Therefore, when you’re evaluating your tools, make sure you do so over the course of time, as this will illuminate a unit’s best and worst features through trial and error.
Zen and the Art of Recording is the third book in my ongoing Zen and the Art of series. You can purchase any and all of these books HERE.
If you’d like to discus these concepts further, join me and my knowledgable friends at Mixermania
Be sure to read my newest book! #Mixerman and the Billionheir Apparent – a satire of the Modern Music Business through the prism of US Politics & vice versa.