Judging coffee competitions is challenging – and organisers need to recognise that

As an industry, when we talk about coffee championships, our focus is often towards the competitors. And rightly so – these coffee professionals can end up investing months of their time in training and preparation.

Every coffee competition, however, needs a group of highly skilled and experienced judges to assess performance, and ultimately determine the rankings and overall winner. Judges not only need to stay sharp, but they also have to be transparent, honest, and thorough when providing their scores and feedback. In short, we often don’t realise just how challenging it can be to judge competitions.

So is there a way to minimise this pressure and ensure a fairer process for both competitors and judges? I spoke to Trent Rollings, a certified World Coffee Championships sensory judge, and Sonja Björk Grant, a pioneering figure in creating the World Barista Championship’s judging format and procedures, to find out more.

How are coffee competitions judged?

There are many coffee competitions that take place every year, but the World Coffee Championships (WCC) are some of the most prominent and exciting. These include:

World Barista Championship

World Brewers Cup

World Coffee Roasting Championship

World Cup Tasters Championship

World Coffee in Good Spirits Championship

World Latte Art Championship

World Cezve/Ibrik Championship

Every WCC event (largely excluding the World Cup Tasters Championship) relies on a panel of judges to score all competitors’ performances. Scores are assessed and based on different sets of rules and parameters which are unique to each competition. This means judges must have a thorough understanding of the specific rules, and need to be attentive to every aspect of a competitor’s routine and presentation.

Moreover, there are different types of WCC judges. For instance, technical judges assess participants’ skills and workflow – including hygiene and food safety. Sensory judges, meanwhile, focus on coffee and beverage quality, as well as how accurate flavour and mouthfeel descriptors are.

In addition to passing a certification course and attending recalibration sessions, WCC judges need to keep up with any new rules and regulations – as well as any changes and developments in the wider specialty coffee sector.

In addition to being a WCC judge, Trent Rollings is a SCA trainer and competition consultant.

“Every year, there are some rule changes,” he tells me. “Last year at the World Barista Championship, the rules were changed to allow plant milks.

“There are also the usual updates in how scoring is done and with the prompts on the sheet,” he adds. “Judges must be updated about all of these changes.”

Exploring the challenges of judging competitions

There are countless articles focusing on WCC competitors which highlight their dedicated journey in taking part, or ultimately winning. But the process isn’t exactly straightforward and easy for the judges either.

From sensory fatigue and mental exhaustion to keeping up with rule changes, being a coffee competition judge requires a lot of time and investment.

Palate fatigue

Taste bud exhaustion or palate fatigue can be one of the biggest challenges for competition judges. At events like the World Barista, Brewers Cup, Coffee in Good Spirits, and Coffee Roasting Championships, judges are required to taste every beverage prepared by all competitors.

Sonja Björk Grant is a co-founder of Kaffibrugghúsið in Reykjavik, Iceland, and has been a WCC judge for 22 years.

“At the World Coffee Championships, there are judging slots throughout the day which are usually between two and five routines,” she says. “[If they are certified to do so], judges will sometimes switch between sensory, technical, and head judging.

“Every judge should know the limits of their palate and their tolerance,” she adds. “For example, you can’t judge too many WCIGS routines because you would drink too much alcohol. You can also eat bread and biscuits and drink water in the calibration room to clean your palate.”

Mental fatigue

In addition to sensory fatigue, listening to and judging so many competitors can be mentally exhausting. Judges need to be just as focused and stay just as sharp for the last competitor of the day as they were for the first person.

“When they are performing, the competitors are providing you with a lot of information – it can be overwhelming for the judges sometimes,” Trent explains. “So you have to prioritise intaking the information. I avoid writing too many notes and try to focus on understanding the competitor – how they’re explaining what they are doing, which ingredients they’re using, and what they have pre-prepared for their routine.

“It’s important, however, that competitors feel that the judges are engaged with them and are paying attention to what they are saying and doing,” he adds.

Sonja emphasises how concentration is absolutely essential for judges.

“When I walk on the stage, greet the competitor, and a new routine starts, there is nothing else on my mind. I have a scoresheet before me, and I am looking for answers to the questions on the sheet,” she tells me. “Once the routine is done, I leave the stage, finish writing my comments in the calibration room, turn in my sheet, and then I forget the routine that just finished so I am ready for the next competitor.

“I want to be very focused on staying in the moment of the routine so I don’t miss anything the barista is saying that is relevant to the drinks they are serving,” she adds. “This comes with practice and experience.”

Staying calibrated and minimising variance

As part of the WCC recertification process, judges must attend calibration sessions before the competitions take place. This is critical to ensuring all judges’ palates are as calibrated and aligned as possible to make sure the judging process is fair and consistent.

And this is especially important considering that judges often come from different countries and cultural backgrounds. In turn, they will have varying levels of exposure to different foods and cuisines, which will ultimately influence their palates.

Judges also need to ensure they are well-rested for each day of the competitions, and should avoid eating certain foods which could inhibit their ability to taste flavour notes in coffee.

“You shouldn’t eat spicy and strong-tasting or smelling foods – or anything that can taint your palate – just before and during the championships,” Trent explains.

Communication challenges

Just like the judges, WCC competitors come from all over the world. Naturally, this can present language barriers – especially considering that the championships must take place in English or with an interpreter.

On top of that, communicating with the other judges – especially during deliberation – requires excellent verbal skills.

“You need to know when to assert yourself and your reasoning for the score you gave, and when to say that you might not have been fully calibrated,” Trent says. “Judging is a team sport and it requires everyone to be in sync.”

As part of debrief sessions, the judges speak to every competitor to discuss their routine. While most debriefs are amicable and run smoothly, disagreements between competitors and judges are certainly possible. If this happens, it’s important to remain respectful at all times.

How can competitor organisers improve the judging experience?

First and foremost, Trent points out that consistency in the format and structure of the competitions is an important step.

“Improving consistency between the regional, national, and world levels of the championships would make a big difference,” he says. “It would help the judges, as well as the competitors, to be better prepared.”

Given how competitors receive feedback on their routines, it could be beneficial to implement a similar system for judges. For instance, head judges or WCC representatives could potentially provide individual guidance on how to improve judging techniques or how to remain calibrated.

Acknowledging the positives

Sonja emphasises that the judging experience can be highly rewarding, of course.

“I have loved my journey of being a WCC judge for 22 years,” Sonja says. “It was definitely not all a walk in the park, but what I enjoyed the most was being able to understand different coffee cultures from around the world and the people working in them.”

While it can be a difficult process, becoming a WCC judge is considered a prestigious and honourable position – and Trent recognises this.

“It’s not about competitors vs judges – we’re not looking to take away points and punish the competitors,” he says. “For the most part, we are very supportive. We want the competitor to feel like we are engaged with them.”

Taking part in the World Coffee Championships is an opportunity for coffee professionals to learn and grow. And while we mostly associate this opportunity with the competitors, it also rings true for the judges.

Ultimately, by encouraging open communication and providing better support to the judges, organisers can improve the competition experience for everyone.

Enjoyed this? Then read our article on how coffee competitions have changed in recent years.

Photo credits: Trent Rollings, Specialty Coffee Association, World Coffee Events

Perfect Daily Grind

Want to read more articles like this? Sign up for our newsletter!

The post Judging coffee competitions is challenging – and organisers need to recognise that appeared first on Perfect Daily Grind.