With so many coffee awards trying to find the “best coffee” it can be hard to know which awards to pay attention to. In this article, I’m going to discuss why you should be wary of all coffee awards, and how to think about the results.
It is helpful to compare coffee awards to wine awards.
Gold and silver stickers on bottles of wine are so common that many people don’t pay them much attention. In the low and mid-tier price range, they may provide some reassurance to buyers that the wine they are drinking is “good”. However, the top shelf wines command a high price based on reputation and branding, and the appearance of stickers on these bottles can cheapen the impression buyers get.
Not so with coffee. Coffee awards are still sought after by many of the most expensive and exclusive specialty coffee roasters. This shows just how far behind the coffee industry is compared to the wine industry. A great winemaker with a strong reputation will have the confidence to sell wine at a high price based on their self-belief in its quality. This is not yet the case with many coffee roasters, who still seek the external validation that competitions can provide them.
An essential difference between coffee and wine awards is that it is much easier for coffee roasters to use a “bait and switch” strategy to win awards for products that their customers never have the opportunity to buy. This is much harder in the wine industry, primarily due to the importance of freshness in coffee and the inability of so many coffee roasters to supply coffee to their customers that is fresh. In a coffee competition, the organisers don’t go and buy the coffee as if they were regular consumers but rely on roasters to send them the beans.
So for a competition, roasters have full control of how fresh the coffee is when it is brewed for judgement. The “final date” for getting coffee beans to the competition organisers is typically very close to when the competition is held because the organisers recognise the importance of freshness and so provide roasters with the opportunity to supply coffee for judgement while it’s still fresh.
In contrast, wine is not affected by freshness. In fact, many wines benefit from years in the cellar. And so while producers will still send in their bottles, the product that they are sending in is in the same condition as customers can purchase. This is typically not the case with coffee, where customers have trouble accessing coffee a week or two after it’s roast date (when it’s at it’s best).
In the wine industry, consumers are far more educated and harder to fool. Wine drinkers recognise that not all Barossa Shiraz is the same. This is far from true in the coffee industry where it’s common for consumers to claim that they like coffee from a particular country, not understanding the diversity of coffees that are grown within that country.
Coffee drinkers may try to impress their friends by claiming they “like Ethiopian coffee”, but you impress no-one by claiming that you “like Australian wine”. Both statements are quite meaningless, but the comment about “Ethiopian coffee” will be accepted, where the comment about “Australian wine” will be ignored.
This lack of education amongst lay-people means that a coffee roaster advertising that they have recently won an award at a competition will provide a halo effect for all of their products, not just the coffee that won the prize. It is hard to imagine a winemaker selling the benefits of their full range because their Merlot just won an award. They will always specify what wine won the award. In contrast, coffee roasters will sell the benefit of all of their products by advertising that they are an award-winning roaster.
Then there is the issue of large roasters entering all categories with the maximum allowed entries. So, if there are 5 categories, and roasters are permitted 5 entries per category, and each entry costs $300, then a large national roaster may be able to afford the $7,500 to enter 25 times. If they repeat this across just 3 competitions, then they may be spending over $20k on competition entrance fees each year.
All small roasters and many medium-sized roasters would baulk at spending this much. But purely by numbers alone, the larger roasters will have more chance of picking up some medals. A quick look at the results of most competitions proves this to be true.
The inconsistency of judges scoring the same wine based on its order in the line up has been well documented. I haven’t come across any similar studies for coffee, but the fact that there are documented processes at coffee judging competitions for when judges disagree on a coffee's quality by a high degree shows how fraught with danger trying to compare taste.
Worse still, is that the nature of competitions lines coffee up on a binary, two-dimensional scale (as it does with wine). The complexity of coffee flavours is wild (far more complex than wine) and so the idea of comparing coffee on a single dimension is ludicrous. This is evident from the scoring sheets themselves, which score the coffees on more than one dimension, but then combine those scores into one overall score. So, if you mainly like a particular aspect of coffee (let’s say sweetness), then a coffee with high sweetness – e.g. the perfect coffee for you - may not win an award because it lacked acidity and so was considered “flat” by the judges.
So, there’s plenty to dislike about coffee competitions. Let’s move onto what’s to like about them.
Um, well, nothing actually.
And this is coming from someone who once upon a time, entered and won a category at the Golden Bean coffee awards. It’s not the view of a disgruntled roaster. Coffee awards are a terrible way to measure how much you will enjoy a coffee. You are taking someone else’s subjective and flawed opinion of a coffee that you probably can’t even buy and considering a score which doesn’t reflect your individual tastes.
The only opinion that counts is your own, and it must be based on the coffee that you can actually buy. You must have confidence in your own ability to decide on what you like and not worry about whether or not you have some imaginary talent for tasting coffee.
There are valid and helpful awards for products that are objective and require expert knowledge – such as the most energy-efficient fridge. But awards for products that are based on subjective tastes are not only a waste of time and money, but also play into people’s insecurity of if they have the “ability” to taste coffee “properly”. Trust your palate.