How I Became a Certified Barbecue Judge

But before Hoekman's smoker fired up, and two days before the barbecue competition commenced at the fifth-annual BBQ, Blues & Brews on the Bay, representatives of the Kansas City Barbecue Society (KCBS) gathered to sanction the event. This in part involved amassing qualified barbecue judges, and to do so, the KCBS held a class to train and certify judges. That's why I was there, along with 25 or so other hopefuls: a hungry mix of retirees, current and former military members, and self-proclaimed "desert rats," aka ATV enthusiasts, drawn from the massive coastal sand dunes to the north.

Three hours into the training, the guy sitting next to me noted, "I didn't know there was going to be a test" — which there was. If we passed the class, we could join more experienced judges in deciding the fate of barbecue teams like Uff-Da-Q. Not every competition comes with a certification course, so some participants had driven as far as 450 miles, plus paid $100, to take the class in hopes of becoming certified. Of that $100 fee, $35 — about the cost of keeping a goldfish for a year — went towards a one-year KCBS membership. The $35 dues are collected annually, and if members ever let their status lapse, they'd have to take the lengthy certification class all over again.

KCBS claims to be "the most respected form of judging and scoring in the world of BBQ." It can help to think of it as the NCAA of barbecue: an organizational body, without a dog in the fight. Founded in 1986, it is one of the largest cooking-contest organizations in the world today, sanctioning more than 500 events a year across the United States, Canada, the Caribbean, and Europe. A nonprofit, it counts more than 21,000 members and claims to have certified more than 26,000 barbecue judges.

KCBS was formed in 1985 with only one criterion for membership: "that none of it be taken seriously." To do so was even grounds for ejection. Now, KCBS estimates, around $4 million in prizes are doled out every year at its sanctioned contests. Along with money, winners earn the prestige that comes with conquering a respected barbecue competition. Pulling off this credibility while remaining true to its creed isn't easy, and KCBS walks a tightrope between having fun and ensuring that the contests it sanctions run smoothly.

Which brings us to the last thing KCBS fundamentally wants to deal with: a barbecue team with smoking-hot tempers ticked off because an unqualified judge gave it a bad score. So yeah, there would be a test. There would also be an oath. KCBS believes that barbecue is America's cuisine, and the oath taken by those passed who the class would include swearing to uphold "the American Way of Life."

The judges at work during this year's BBQ, Blues & Brews on the Bay. Photo: Courtesy the Mill Casino/Steven Michael.

"A 'two' is when you put something in your mouth and vomit."

The certification class was led by the former president of KCBS, Mark Simmons, and he began by handing out the official KCBS judge handbook — a brief and repetitive but exact booklet of around 30 pages — and setting down the KCBS barbecue standards. "Don't judge KCBS competitions based on what you heard on the Food Channel," said Simmons. "You are not judging by what you like, but by the standards defined by the KCBS." We would need to judge barbecue objectively and subjectively, as the judge's oath states.

First, we learned KCBS does not consider grilled meat barbecue. Grilling is defined in the handbook as "a relatively fast, direct heat method of cooking." Then Simmons explained we'd be judging four types of meat only: chicken, pork ribs, pork butt, and brisket. The handbook defines chicken "as Chicken or Cornish Game Hen and Kosher Chicken." Ribs can be spare, baby back, or any other type of pork rib, but they must have a bone.

"Anyone can cook a rib until it falls off the bone," said Simmons, sharing his personal understanding of the rule. "The perfect tenderness is when you take a bite and the meat comes off with little effort, but the rest of the meat stays in place." The ideal rib can obviously be debated, but here's the point: It's the KCBS way. As I learned to judge by KCBS standards, I found a mix of common-sense definitions and definitions based on opinion — the latter being the subjective glue turned gospel that holds the competitive U.S. barbecue world together.

Each barbecue entry we judged would be tallied based on three categories, Appearance (weighted .56), Taste (2.2972), and Tenderness (1.1428), on a scale of two (Inedible) to nine (Excellent), with six being Average. So, if an entry received a six in Appearance, 3.36 would go toward the total score (6 x .56 = 3.36). "Giving a team a five is like shooting them in the heart," I heard one experienced judge say.

When pressed, the same judge gave some exceptionally helpful clarification based on his own opinions. "A two is when you put something in your mouth and vomit." A three is "when you put it in your mouth and spit it out." As I understood it, he implied that a four is when you put it in your mouth and want to spit it out, so in general, scores from two to four seemed cruel unless necessary. The score of one means the entry is disqualified for not following the rules, and a perfect score only comes about when every judge at a table awards nines in every category.

When someone asked what qualifies as Excellent, Simmons said, "You will find out what is excellent." His point: We would each develop our own subjective understanding of the judging scale. After about four hours, the class concluded. I had passed. And so had everyone else.

Photo: Mattie John Bamman/Eater

"What part about judging barbecue do you not understand?"

At 9:30 the morning of the competition, a friendly woman showed me where to sign into the judging station, located in a warehouse in the center of the festival. When I told her I'd forgotten my temporary KCBS badge, she told me it was more important that I not forget Ziploc baggies and a cooler for taking home leftovers.

While we judges got organized, a barbecue team tradition commenced outside: At 10 a.m., two hours before the doors would open, the competing teams gathered to wish one another luck and seal the deal with a good-morning shot of Jack Daniels (specifically). In certain areas of the country, the ritual is held at 9:11 a.m.; the national anthem is played, followed, of course, by that shot of Jack.

Among the 50 or so judges divided among eight tables, I saw familiar faces from the certification class, but most were more experienced. There were only slightly more men than women, with the younger judges mostly men. I asked around to learn why people became certified with the KCBS, and for the most part, my fellow judges looked at me like I was crazy. One woman summed it up best, replying, "To judge barbecue" — her bewildered eyes saying, What part about judging barbecue do you not understand?

After a brief overview of the rules, along with an official audio recording repeating them again, we took the KCBS judge's oath. The man leading the oath ended it with an "amen": some laughed, some repeated it. Then we were dismissed until the competition would start.

As a judge, the moments before the competition kicks off are spent setting up your personal station. I had a large judging plate: a paper placemat divided into six squares for placing individual barbecue entries. I was told it was paramount that entries not be cross-contaminated, and in the upper right-hand corner of each square was a box for numbering the barbecue entries.

Taking cues from the other judges, I piled a small tree's worth of quartered paper towels in front of me and doused them with water. Finger licking is expressly not allowed for hygienic reasons: All of the judges at the table would use their fingers to handle the barbecue, as forks are not traditionally used at KCBS-sanctioned contests because the organization considers barbecue finger food. Then I surrounded myself with quartered Saltines and bottled water to cleanse my palate, and finally, I filled out a judging slip, circling the first category: chicken.

Just outside, the competitors submitted their barbecue in Styrofoam boxes supplied by KCBS. "Some cooks drop off their barbecue like it's Faberge eggs," Simmons had said. The boxes arrived already officially numbered, and the person receiving the entries then passed the boxes off to another official, who renumbered them using an idiosyncratic algorithm. Thereby the judging was double-blind: The competitors would never know who judged their food, and the judges would never know whose food they ate.

Our Table Captain, who orchestrated everything but did not judge, told us the numbers of the barbecue entries, and we wrote them in the boxes on our scoring sheets. It was go time. The very first chicken entry I encountered wasn't particularly pretty: The thighs were not uniform in size or shape, with unnecessary skin hanging off the sides. I gave it a six (Average) because I personally like a more uniform appearance. Five more entries were presented and judged for appearance. Per KCBS rules, I did not compare entries but considered each one on its own merits.

After everyone scored on Appearance, the Table Captain passed around the boxes of barbecue one by one. Once I had all of the entries before me, I bit into the first thigh: The rub had depth of flavor and the skin was a little crisp — especially notable when you consider that trip in the Styrofoam. The smokiness was there and wasn't bitter, but I didn't get that burst of chicken juice I wanted; in the best cases, it's like biting into a meat peach (my opinion, not KCBS's). I gave it a seven (Above Average) for Taste, making sure the crispiness of the skin didn't play into the score.

Crispiness falls into the Tenderness/Texture category, and as the KCBS handbook says, chicken should have a "nice texture and any juice present should be clear." To me, a "nice texture" is crispy skin and moist meat not over- or undercooked. In the certification class, Simmons had described undercooked chicken as rubbery and overcooked chicken as dry and mushy. I gave the entry an eight (Very Good).

After judging all of the chicken entries, the Table Captain collected all of the judges' scores and took them to a designated event official, who then gave them to Simmons, who was charged with entering the data. I put the glorious remains of my entries into a Ziploc baggy and into my cooler.

Then it was on to ribs, followed by pork butt, which often comes in three forms (pulled, sliced, and chopped), and finally, brisket and burnt ends. No matter what the meat, some entries came with barbecue sauce and some did not — sauce being optional. In the class, Simmons had said to judge only the meat and told us we should not score an entry up or down for not having sauce; we should only take into account what's on the plate, not what's not.

By the end, I'd eaten around two pounds of meat. Besides the flavors, education, and entertainment, this would be our only reward for serving as judges.

Having received all of the judges' scores, Simmons tallied them to find the winners. Every barbecue entry at a KCBS-sanctioned contest is reviewed by six judges, but its score reflects only five judges' opinions: The lowest number is always dropped. Accordingly, a perfect score is 180 and results when at least five judges at a table give nines in all three categories.

The winners of the fifth-annual BBQ, Blues and Brews barbecue competition were announced at 4 p.m. Awards were given to the top 10 teams in each of the meat categories, with each first-place winner receiving $575. Three overall winners were also announced, earning the titles of Third Place, Reserve Grand Champion, and Grand Champion, with the Grand Champion receiving $4,000. I saw one jilted team that looked veritably angry, but most were convivial as the $13,500 in total prizes was handed out. And the Uff-Da-Q team, led by Daron Tandberg, ended up taking second place in brisket; for the second year in a row, California's Rooftop BBQ was named Grand Champion.

I never found out if I tasted award-winning barbecue, so I asked Simmons why the KCBS keeps things so secretive. "It has always been felt that anonymity produced the fairest competition," said Simmons, "because only the meat is considered. What if you had a VIP in a contest cooking and everyone knew this was their food? They could win on reputation though their product was inferior to others. Blind judging, where no one knows who's food they are judging, gives the cooks the best shot. No one is winning on reputation, and a brand-new cook can compete with the big boys, if they've got the skills."

Mattie John Bamman is editor of Eater PDX.
Editor: Erin DeJesus