Back in May, I announced a blind tasting in which I would pit the classic A.H. Hirsch 16 year old gold foil against an off the shelf bourbon. After receiving hundreds of responses, I picked 12 individuals for the tasting. I tried to create a diverse group so I included people with a variety of tasting experience. All were whiskey drinkers (they read my blog after all), but some had more bourbon experience than others. I picked some people I knew and some completely at random. The one requirement was that no one in the tasting could have tasted any A.H. Hirsch bottling.
Each taster was given two sample bottles, labeled only "A" and "B." One contained the A.H. Hirsch 16 year old; the other contained...Elijah Craig Small Batch, which I bought at my local shop for $25 (the one I bought for this tasting was the version that still had the 12 year old age statement on the back label, which has since been removed).
Each taster was given their bottles individually and was told to report back to me, in as much detail as they cared to give, about which bourbon they liked best and why. The goal of the blind tasting was to see how a cherished bourbon that goes for thousands of dollars on the secondary market would compare to a standard, good bottle of bourbon off the shelf.
Well, the results are in and...it's a tie. Six of the tasters preferred the Hirsch and six preferred the Elijah Craig. There was not particular pattern having to do with tasting experience. A few bourbon veterans correctly identified which was the Hirsch, but not even all of them thought it was the best. And many tasters were lukewarm to both samples. That being said, a few tasters were very excited by the Hirsch and found it markedly better. Below are a few excerpts from the tasters' notes on each bourbon.
A.H. Hirsch, 16 year old (gold foil), 45.8% abv.
- "Not as complex"
- "Okay, nothing I'd run out and get, no matter the price"
- "Very smooth, sweet, warm, maybe a little one-note, though."
- "It has a really nice old-time flavor that I enjoyed."
- "It goes bitter on the palate and I'd call it over oaked. There's some wet grass in there. Overall it comes off as very tired."
- "This is pretty great. Not crazy complex or powerful, but well balanced. If this is the Hirsch, it lives up to the hype for me."
- "I wouldn't turn it down at a bar, but I wouldn't be excited to drink more of this."
- "Classic dusty caramel mouth feel."
- "Nothing I'd run out and get, no matter the price."
- "A fairly straightforward dusty-like pour with a lot of higher alcohol notes and a classic woody character."
Elijah Craig Small Batch (12 year old on back label), 47% abv ($25)
- "Banana bomb like most current bourbons."
- "Harsh and alcoholy."
- "TONS of butterscotch! The mouth-feel on this was much nicer than A, with a velvety smooth feeling..almost creamy."
- "Dry taste and mouth feel at first, tastes older but not sure it is, lots of dry wood."
- "I'd buy it, though not for three or four figures."
- "Spicier, the sweetness is at the backdrop and there is more balance from nose to palate on this one for me. It's a creamier dram."
- "Simple bourbon with not much in the way of flaws but nothing to make me want more."
- "When I think of bourbon, this is the essential profile I imagine."
And a few concluding notes folks had on both of them:
- "I honestly didn't really enjoy either one. They would not be something I would go back to my bar for consistently."
- "Neither is fantastic."
- "I'm disappointed I didn't really jump for joy over at least one of these."
What does it all mean?
Twelve people is too small a sample size to make any definitive statement, but the failure of Hirsch to blow away the competition, even among those who preferred it, certainly raises questions about its current value. If a bourbon that costs four figures on the secondary market isn't demonstrably better than a $25 off the shelf pick, then its value is obviously based on something other than the quality of the whiskey. The fact that it's scarce, no longer in production and famous increases its value just as scarcity does among any other collectible, and like it or not (and I'm firmly in the "not" camp), bourbon is now that sort of collectible. But if you're one of those increasingly rare folks who actually buys rare whiskey to drink, I would think this tasting might make you pause before plopping down thousands of your hard earned dollars on a bourbon that you might not like much better than one you could buy today for $25.
Many thanks to the Hirsch tasting group for their time and their fantastic notes.
17 comments:
Great experiment SKU. Now I'm really glad I passed on a pour of the Hirsch when I saw it at the Casa Fuente Bar in Vegas for $200. I look forward to seeing more blind tastings like this, and perhaps being a part of them sometime.
Very interesting results. I'd suggest though that the fact that the reviewers knew there was one regular bourbon among the two might have resulted in more muted responses to both for at least some people. If you are (sub)consciously nervous about raving about what might turn out to be Jim Beam White you might be conservative about what you say about both, not just the one you like less.
This was a great read. It confirms what I think most of us have long known or at least suspected. While I know that the differences between most premium bourbons are marginal as far as quality goes it doesn't stop me from romanticizing some of the rare ones and wanting to believe that they truly are in a different league. My rational and irrational sides continually fight this out.
Heh. I LIKE the Hirsch, and still have -- well, more than one bottle. But this is exactly why I'm slowly 'de-positioning' myself on the Hirsch, and drinking bottles that shelf for under $30: because I really, honestly like them.
hadn't really thought about it, but i *really* agree with comment #2 (8:25am) about the muted opinions presented. i wonder how they would have differed if the testers KNEW what the two bottles were, but they were poured/presented blind. cool test, regardless; kudos for doing it.
MAO and Jay, that's a completely legitimate point. The test wasn't totally blind as everyone knew what one bottle was, and to that extent, it certainly wasn't perfect. The knowledge that one was Hirsch and the other was something else could well have skewed the results (though there were certainly people who were enthusiastic about their pick, just not the majority).
I thought about a completely blind test, just telling people that I wanted them to taste two bourbons side by side, but I also wanted to test this on people who had never had the Hirsch, and in doing so, give them a chance to taste this rare and storied bourbon.
A "more muted response," is still a response. Not as if this idea or the fruits thereof were pitched as a scholarly double-blind. Thanks for the read, Sku.
Not a huge surprise as I figured it would be split pretty evenly. Loved the comments.
Next to 2012 Mich 20 and a couple 'legendary' Willett bottles I've had the fortune to sip, Hirsch 16 stands out as being the biggest disappointement for me.
Nice work, SKU. Looking forward to the next round.
Years ago my father and uncles were drinking multiple bottles of wine on the patio. My wife asked which one was the "best" meaning price. After several hours, and many tastes, she was asked which one she liked the best. She answered and was told then that one is the "best". Drink what makes your mouth happy not the label or price.
Nice tablecloth.
I think this also speaks to the quality of Elijah Craig. RIP EC12.
"...then its value is obviously based on something other than the quality of the whiskey..."
It's "market price" is obviously based on something other than the quality of the whiskey. Not sure that value has a place in the discussion of these hyper- inflated whiskeys.
Certainly a more muted response is a response; and I didn't mean to suggest that the exercise was not meaningful. Just suggesting the comments (though probably not the rankings) might have been more enthusiastic if the tasters thought they were drinking two bourbons of equivalent reputation (whether high or low). But mostly I'm suggesting that Sku should just repeat variations of this exercise with every bottle of storied bourbon in his collection.
Hey Sku! This was indeed a fascinating study and it's extremely generous for you to have donated your bourbon to science. Also, as a long time reader, (but first time commenter), cheers to your approaching 10 yr compendium of great insights.
I have some thoughts based on your results:
First, Bourbon is the prototypical veblen good. This mini survey lends credence to that assertion and kind of reiterates what I think you suspected going into the survey.
Second, if you were able to quantify the results, even if inexactly, 12 is not necessarily too small a sample. The rule of thumb that the sample size needs to be >30 is only for normal distributions. My judgement is that bourbon scores are non-normally distributed, as the mean is clustered, say, around 80 for the average score and there are large tails (a higher than expected number of extremely high and low scores). Maybe for future surveys this can be incorporated, as, what is the gradation of good vs. high retail price? Or perhaps thats too nerdy.
Third, this study raises an interesting point about bourbon arbitrage. There is no vehicle really commensurate to a Lamborghini except others similarly high priced. Compared to a $500 Cured Oak or w/e, which albeit tasty, there are dozens of extremely "high end" bourbons that are much more affordable and as good in reality. Nearly any four Roses PS from a store with a track record of single barrel picks could easily be repackaged into a CEHT "Seasoned and Cured Tornado" and few would really be the wiser.
Barrell Bourbons, particularly batch 006, is in my opinion as high end as many of the weird/rare relic whiskeys I've tasted. Or the (sadly increasingly rare but affordable) SAOS store selects, or if lucky, the SAOS Cask Ryes, which in their heyday are on par with the rye BTACs (at least imv). A 7yr Willet for $100 including tax is playing barrel roulette, but some are truly fantastic. On and on. Anyway, you expressed all these sentiments before, so no point in preaching to the choir.
Anyways, hope if not too great a ding on your collection, you'll continue to do these experiments
Great experiment! There are ways of making this more scientifically valid and address the muted response part.
For example, you can use 3 bottles - say, EC12yo, Hirsch 16, and 4 Roses Single Barrel (possibly watered down to equivalent strength), or another high-end bottle. People would be assigned two samples at random. If you have enough participants you may even allow that both samples are from the same bottle. It is still a guessing game, but with much higher uncertainty. The downside is that a person is not guaranteed to taste the Hirsch but such is life. In a clinical trial people may receive placebo.
But even in your current version, the comments may be dampened, but the choice of preference is still valid.
It is possible to show statistically the preference for one bottle with n=12, but you'd need 10+ people to prefer one versus the other, so the difference would have to be pretty extreme.
HA!
I stand .... UN-Surprised by these results, as no doubt many others are.
Small sample group or not, tastes vary in a broad way among any number of folx. That being an unassailable truth, one would expect about what we got. A larger and larger group would likely eventually include a few who rave about one or the other (no doubt in approximately equal numbers). Great Fun to follow, and a worthwhile endeavor, SKU. Thanx!
Thank you for confirming what I was guessing. I'll go back in my hole with Pennsylvania LCB finest bottom shelf Evan Williams BiB and Old Ezra 101.
Post a Comment