Valve’s Swiss System under the microscope

The RMRs, time after time, provide us with some of the most exciting Counter-Strike in the calendar. But are the Major qualifiers really doing their job?

What is the point of seeding? Its dictionary definition is to ensure that higher-ranked teams get an easier path throughout a tournament than lower-ranked ones. It is there to delay confrontation, to ensure that the best teams meet as late in competition as possible. You want the best two teams to play against each other in a final, not in the round of 16.

Good seeding ensures a team’s probability of winning the tournament reflects their strength, and is not a result of a favorable schedule. In situations where there is no seeding, a team can get lucky in the draw and make it to the final without playing any truly top opposition.

Seeding prevents that. If an underdog is to upset the odds consistently, they have to do so against the best teams. There is an element of unfairness to this: Why should a small team have to face the three best teams in the tournament to have a chance of progressing? Breaking into tier one is hard enough in an era when tournaments with partnered teams dominate the calendar — seeding only makes this harder.

This is the real crux of the debate. It is a question of meritocracy against equality: Should the best teams be rewarded for their recent form with a generous seed (meritocracy) or should they have to prove themselves again and again in a random draw (equality)?

It is not an easy question to answer. Most sports, however, end up on the meritocratic side of the fence. Tennis, where seeding originated, uses its world ranking to strictly set up its grueling single-elimination brackets. Soccer’s World Cup uses FIFA’s world ranking, Chess its FIDE ELO rating.

In Counter-Strike we have several methods. BLAST Premier have their member teams rank each other. ESL use their personal world ranking.

At the Majors, however, TOs are now required to use Valve’s specific method. Seeding for the RMRs is conducted in a Major circuit bubble, your rank being derived from placement at the last Major or in the Major qualifiers themselves.

Read more

OG's World Final qualification exposes major flaws in BLAST's Leaderboard

There are benefits to this. Seeding that use ranking systems like HLTV’s world ranking rewards not just good teams but also those that play frequently. The presence of partner teams only makes this harder to account for: Think of how frequently OG get a boost in the rankings when BLAST Spring Groups rolls around. Just a few wins in this closed circuit can have a tangible impact on your later invites and ranking.

The only truly 'open' tournament in the calendar is the Major, which is surely a part of Valve’s thought process in deciding to build a system that completely relies on it.

And so, at the RMRs, we have a world where the two best teams do not have a seed that reflects that. G2, who did not qualify for the last Major at IEM Rio, were initially seeded 9th in EU B, and FaZe, who went 0-3 in the Legends stage, were 7th in Group A. For all the flaws of other seeding methods and the valid criticisms with regards to partner teams, this is unforgivable for a seeding system.

The Buchholz system is integrated into Valve’s Swiss format to act as a band-aid and to self-correct seeding as the tournament goes on but it cannot fix the terminal damage inflicted by only using the last Major and the RMRs for teams’ initial seeding.

We did not always use this system, of course. Part of the appeal of Swiss is that teams always play other teams with the same record; to an extent, it is self-seeding. And initially we let it be just that, using a basic system for initial matchups and then random draws between rounds.

In 2018, FACEIT used Buchholz for matchups from Round 3 onwards in the London Major. ESL introduced player-determined seeding for IEM Katowice 2019, where teams ranked each other and an average was taken to create an ordered list. Live ELO then slightly adjusted teams' placings throughout the stage.

The next Major, StarLadder Berlin, also had Live ELO, but used HLTV’s world ranking rather than players for their initial seeds. It was only after the pandemic that Valve’s new seeding system came in and it is the one we have used for the last three Major cycles.

Below is a chart that shows the correlation between a team’s seed and their final placement in the Swiss system. The larger the bar, the closer a team’s final placement resembles their initial seed — meaning the system is working.

As we can see, the post-pandemic system has rewarded its higher-seeded teams less. Although IEM Katowice and SL Berlin represent a small sample size of two Swiss stages, this is still concerning.

A lower correlation can mean two things. One: The higher-seeded teams get less benefit from being highly seeded. Two: The higher-seeded teams are there incorrectly and are failing to convert their advantage into success.

Both can also be true at once. When G2 are seeded 9th, it is not just G2 who are punished, but also the teams seeded above them. 9INE performed excellently in the closed qualifiers for the RMR, earning the 8th seed, but then had to face G2 in their opener. The Poles won, but it is still a sign that the system was failing.

We all know how important the opening best of ones are in Swiss. Recovering from 0-2 has happened, it even happened twice at the EU RMRs with ENCE and OG, but it shouldn't be relied on. Therefore, having correct seeding for these opening games is paramount: Buchholz can only do so much.

Even the events with the most successful seeding post-pandemic fall short of SL Berlin and IEM Katowice’s averages. If we use HLTV ranking instead of seed, it fares even worse.

Of the systems that use best of threes in decider games, it has the lowest correlation between final placement and HLTV world ranking. The RMR style falls behind even the random draws of StarLadder & i-League's Starseries seasons (that used best of three throughout).

A goal of any group stage is to ensure the highest quality playoffs possible. CS:GO, like any sport, is a spectacle. We want to see the best teams face each other as late as possible into a tournament, when the stakes are highest and the arenas are packed. That is a reasonable view to hold, both for fans and the TOs who are building their formats.

This system, clearly, is not doing so. It relies on too small a sample of games from the RMRs, or an outdated sample from the previous Major. Too much happens in six months for teams to deserve to be seeded based on their performance in the last cycle.

Solutions

The solution preferred by many would be to return to the more effective seeding systems of the past: Using any world ranking, whether HLTV’s, ESL’s, or Valve’s, would result in more favorable matchups for teams that are performing the best outside of the Major circuit. The same is true of player-decided seeding, the system that is still used today in BLAST Premier.

But, any world ranking is harmed by the Louvre and BLAST partners paying for attendance at big events and the ranking points those hold. That Valve does not even use its own ranking for seeding shows that the issue is not with any individual system but the approach itself. Partner teams are inherently better off under any ranking system — and this is not something we want at Majors, even if Valve showed some leniency by using their system for invites to the online closed qualifiers.

Once it gets to LAN, however, it seems that the playing ground must be even. Under a meritocratic view of 'even' fairness, IEM Katowice and SL Berlin’s seeding methods achieved their goal. The teams which their systems believed to be better advanced more often. Yet, they were not used again.

If this is what we are striving for — and the Majors are to be a celebration of the open circuit — it is clear that we are striving for a more egalitarian view of fairness. The biggest and best teams’ past results mean nothing once they enter the Major cycle; they have to prove themselves each and every time.

And the end result of this mindset is that any form of seeding at all is missing the point. The only truly equal way of doing it under this premise is for the event to be completely random and for seeding to be scrapped entirely. If the best teams are good enough, they should not need a leg up to qualify.

But, the Majors are not just celebrations of the open circuit. They are the pinnacle of the game, the titles that define the careers of players who have been champions and of the players that do not. We want these tournaments to be narrative builders, their results to give new fans of the game an easy-to-scan history of the esport. We don’t want there to be any asterisks.

And so we are back where we started, needing to define what we truly want out of a group stage. Do we prioritize entertainment and equality or competitiveness and meritocracy?

From a competitive standpoint, the latter is clearly more important. Entertainment should not come at the cost of competitive integrity. Best of ones are often more exciting than best of threes: they have higher stakes, are shorter (and thus are easier to sit down and watch in their entirety), and the lower sample size means upsets are more likely. But we still acknowledge that they are worse at determining the better team on any given day than a best of three.

The same is true of random seeding. We are more likely to see big names eliminated, which gives us more easy narratives and we have the draws themselves to add a spice of excitement. We should also have closer games in a no-seeding environment, because it is more likely teams of equal strength draw each other.

But this is not WWE. The Major does not exist to entertain us; it exists to crown the best team in the world. And so, there are two solutions.

One, we bite the bullet and integrate seeding. By doing so, we acknowledge that the Major is the pinnacle of the Counter-Strike circuit and not operating outside of it. Therefore, it follows that results outside of the RMRs do matter enough for seeding to be based off of them, either by players ranking each other or any solid world ranking.

In between rounds, ESL's Live ELO system at IEM Katowice 2019 did a good job of rewarding underdogs for punching above their weight and punishing underperforming favorites. After fnatic lost to ViCi and Winstrike in the Challengers Stage, their ELO rank dropped from 1.79 (1st) to 8.00 (9th). It is not catastrophic — they still had a favourable match-up against Grayhound in the 0-2 match, but once round four rolled around they had to face G2 (originally seeded 7th) instead of Cloud9 (13th) and were eliminated.

A slightly different system was used at StarLadder Berlin, but with the same principle, and it worked as intended there, too. Katowice's ELO ranks ended the group stage with a 0.954 correlation with a team's final placement, while Berlin's had a 0.956. A high score is to be expected, but this still shows that teams' seeds were being correctly adjusted as the tournament went on.

If a 9INE beats G2 in round one under this system they would not be expected to have to beat two more huge names in order to qualify; an upset would elevate them up three or four seeds. Buchholz is a good system for tie-breakers, but it is clearly less effective than this kind of Live ELO system under a small sample size.

Swiss is one of CS:GO’s best formats. Games matter the same for both teams, unlike round robin, and it prevents rematches so we avoid teams advancing after beating the same opponent twice like in GSL. It is also deeply entertaining, 2-2 games at the RMRs offer a similar intensity to that of a playoff match at a regular tournament.

But it can be better. The current seeding fails to either reward the high seeds or to allocate the high seeds to the best teams. Underdog runs are part of any sport, and will be whatever the seeding system. The difference is that in well-seeded events those underdog runs cannot be undermined. It is the difference between Cloud9 winning the Major over FaZe and Gambit winning it over Immortals. One feels historic, the other like a strange quirk in Majors history.

Take 9INE, who defeated G2 and Vitality to qualify for Paris. More accurate seeding would not prevent a quality underdog run like this, but it would prevent Natus Vincere from having to play FaZe in a 2-0 match while fnatic claim a Legends spot after beating Viperio, 1win, and B8.

No seeding system is fair in an egalitarian sense, but for the biggest tournament in the game it is hard to argue against a meritocratic definition of fairness. The current system needs change regardless of where Valve and TOs land on this debate; the current system fails to satisfy either definition.

So we are left at a crossroads. If we want equality, a truly even playing field, we should remove seeding entirely. If we want meritocracy, we should return to one of the seeding systems used at Katowice or Berlin. Today’s halfway-house system benefits nobody.