Regional Major Ranking system: What were the issues and how can it be improved?

After two years of online RMR events, how did the system fare?

In January of 2020, Valve announced to tournament organizers that for the Fall Major of that year it intended to replace the existing Major qualification system's legacy invites with a "Major Ranking" that ordered teams by points earned from the Spring Major (which was to be ESL One Rio) and two other LAN events during the year.

The idea of removing legacy invites directly addressed the community's biggest criticism of the old system in awarding automatic invites to the next Major based on the last Major's results. This issue needed to be addressed for multiple reasons. Firstly, the time between Majors had increased since the number was reduced to two per year in 2016, which directly affected teams' willingness to make roster changes and caused certain teams to use coach substitution rules to keep spots they probably shouldn't have. Secondly, the number of teams at Majors was increased to 24 in 2018, which, while not changing the format, increased the prestige of holding on to what was previously a Main Qualifier spot by adding stickers and the label of Major and guaranteed a playoff team at least two further Major appearances (which would be an entire year in the future) regardless of their performance over that time span.

The replacement system being introduced, which proposed the introduction of a Major Ranking, was also viewed as a big step in the right direction. The concept of using results over the course of the competition circuit to determine invites for the Major was one that had been discussed in the community before, with Valve already implementing a similar system in DOTA 2. Within the current landscape of Counter-Strike, we can also see that BLAST and ESL have implemented their own versions of this type of system with BLAST's World Finals invites and the ESL Pro Tour invites to Katowice and Cologne.

There was a set of requirements for events that wanted to be Major Ranking events, but the full system was not able to be implemented as originally intended because of the one reason people cite for everything these days: the coronavirus pandemic. The pandemic caused the ESL One Rio Major to be delayed from the Spring slot to the Fall one, and even though the qualifiers for the regional Minors had completed, the actual events couldn't be played out and Valve decided to go with a different method of determining teams for the then delayed Major.

Since keeping slots for teams from the StarLadder Berlin Major was too ridiculous, even for Valve, the developer aimed to implement a version of the Major Ranking they had wanted to use for the Major cycle after Rio. To keep this already long intro brief, I won't explain the full details of the 2020 Regional Major Ranking cycle and its rule intricacies, as many of them will be explained later on when comparing to the 2021 cycle. The important thing is that after three RMR events (in every region except South America and Oceania which had two each), the ESL One Rio Major still couldn't safely be held in Brazil and was cancelled as a result.

The teams who would have qualified for the Major if it happened were given stickers in game, and also granted a small amount of points going into the 2021 Major cycle, which would lead into the PGL Stockholm Major, scheduled for the Fall time slot of 2021.

The 2021 cycle basics

So how does the Regional Major Ranking system implemented in CS:GO work?

First, teams were split between six regions based on where they physically resided — in contrast to players' nationalities which was used in the Minor era, with some exceptions. In each region there would be 2-3 RMR events, with teams receiving some number of RMR points based on where they placed in the event. Their performance at the last Major (for the 2020 cycle), or in the 2020 RMR cycle (for the 2021 cycle) also awarded a small number of points. The distribution of points at each RMR event was decided by Valve and could be found on the CS:GO blog, with each event awarding more points as time went on and the Major approached. The number of teams in each region that qualified for the Major, and for which stage they qualified, was determined by the results of the StarLadder Berlin Major, which certainly had its downsides considering some key changes in the CS:GO landscape over the two years since that Major.

On the surface, the specific rules relating to roster changes, substitutions, team invites, and the like seem relatively simple, but the more you dig and think, the more questions arise and the more inconsistencies and holes you begin to notice. I followed the RMR cycle closely and was determined to learn all the ins and outs of the system, because a reliable source on the details of the Major rulesets didn't really exist in CS:GO and, as a prominent contributor to Liquipedia at the time I wanted to create a place to which people could refer and reliably know exactly what was going on in any team's qualification for, or status at, the Major. This is basically to say that I looked at a lot of edge cases and wanted to know what would happen.

You can find the ruleset that was (mostly) used on Valve's website but I'll outline the main pieces here. Going into each RMR event, teams can submit 5-7 individuals as part of their lineup: 5 required players, 1 optional coach, and 1 optional substitute. The introduction of the substitute slot was the main change in rules between the 2020 and 2021 cycles, and once again addressed one of the biggest criticisms of Valve's roster rules, which many a time forced teams to use retired coaches who haven't played in years as stand-ins, even if some extenuating circumstance prevented one of the original players from being able to participate.

If a team registered a different lineup of players than the one they had used at the previous RMR event they attended, the team would suffer a deduction in points. This could either be 20% or 40% of their total existing points based on if one or two players were changed. But in order to keep any points at all, a core of three had to be retained from the previous RMR event the team competed in, so if a team replaced three or more players and then registered and played at an RMR event they would have to start from zero points.

In the middle of an event, a team could choose to use their substitute, if they registered one and wished to use them, at the cost of a similar but lesser deduction. When the substitute is swapped into the active lineup, the team suffers a 20% deduction to the points they earn at that event. When the substitute is swapped back out for the original player, there is no further deduction of points, but if they want to swap back in the substitute again later, they would suffer a compounding 20% deduction (which adds up to 36% of points at the event for two substitutions).

Invites to RMR events that use the points were issued after all teams had submitted lineups so that deductions could be calculated and an exact order could be found. These are all the important rules, and this final one leads well into the first of the many issues I encountered covering the 2021 cycle.

Invites by RMR

The first question going into the first RMR event that was not answered by the public CS:GO blog is how invites by RMR standings would work. In the 2021 standings, only a single team carried any points in each of the three small regions. If they were the only teams to receive RMR invites to the Spring series of events, then by the FAQ rules, only a single spot would award RMR points, and that seemed a very unlikely scenario. It turns out that the invites were really based on the final 2020 standings, meaning that teams who didn't make the cutoff could still receive invites by RMR. This directly leads to the second question however, something that was also unknown during the 2020 cycle: How many invites by RMR must each region have? We know that the number of spots that award points is tied to the number of invites by RMR, but how much leeway do the tournament organizers have? Could a TO actually invite just a single team by RMR and then only award RMR points to the tournament winner?

Valve actually gave an answer to this in an email to tournament organizers in January, indicating that each RMR event "features at least as many teams invited by RMR points as there are Major slots available for that region." Without a doubt, this would mean a European RMR event must have at least 11 teams invited by RMR, while an Asian RMR event must have at least 1 team invited by RMR. But not only would this not make sense as the only stipulation, it wasn't at all followed by the TOs or Valve themselves.

In the Spring round of RMR events, cs_summit 8 for NA and EPIC CIS League both immediately broke this stipulation. Both events invited only four teams by RMR to their events despite North America and CIS both having five Major slots available. This couldn't be due to a lack of teams, as the 2020 standings for both regions had more teams available (the next in line for each was TeamOne and Gambit). It also can't be that the invites only considered the 2021 RMR standings, as the South America and Asia regions invited teams by RMR that were not yet present on the 2021 standings for those regions (and went off the 2020 standings).

To add on to this, Valve even contradicted their own stipulation in their communication with CBCS, as the Brazilian tournament organizer was told they had to invite four teams by RMR. Funnily enough, there didn't even exist four teams on the 2020 South America standings to fill those invite slots, even if Valve had gone back on their previous stance. BOOM (who became MIBR), Isurus, and RED Canids (who became Paquetá) all held cores from the 2020 standings and would receive invites to the Spring 2021 event. The final two teams with points on the 2020 standings were Sharks and Imperial. Sharks no longer held a majority of the players that had completed the 2020 RMR cycle, and by Valve's general logic wouldn't be considered the same "team." Imperial also couldn't be invited as they intended to compete in the North American RMR qualifiers, and so CBCS resorted to inviting Sharks and considering them an RMR team.

Technically this type of situation did happen in the 2020 RMR cycle in Asia, where Lucid Dream received an invite to PAL Summer despite changing a majority of the players who competed in the Road to Rio, so this wasn't without precedent.

Let's then entertain the idea that we accept that Valve changed their stipulations for the small regions, and they had to invite at least four teams to allow for more placings at the events to award points. Well EPIC and the RESF clearly didn't receive the same communication that CBCS did, as in their Oceanic RMR event, only Renegades were invited by the RMR. Just like in South America, there were more teams on the 2020 standings that could have been invited.

So right off the bat, we have tournament organizers receiving conflicting information by Valve, resulting in inconsistent rulesets and formats between regions.

RMR points distribution

The second big question was one that was already thought to have an answer. When CBCS announced the details of their Spring tournament and advertised that eight spots at the event would award RMR points, my eyebrows began to raise. They themselves confirmed that only four teams were invited by the RMR, and the FAQ blog clearly states that the number of spots that award points is equal to the number of teams invited by RMR. After confirming that it was indeed eight spots that awarded points, I inquired about how the number was determined. It turns out that Valve had once again devised a new system different to the one they had previously announced, this time contradicting a public document that was supposed to serve as the resource for RMR information.

For the Spring series of RMR events, the number of RMR spots was determined by a formula that started with the number of teams who were "invited based on performance." To Valve, "performance" means teams who received spots at the event through an invite by RMR or through open qualifiers. It's their way of saying they prefer teams to need to "prove it" and that invites based on other means by the tournament organizer don't count towards the number of spots at the event that award RMR points. The sum of both counts (RMR invites and open qualifier teams) represented the maximum number of spots that could award RMR points, with the format of the event being evaluated to narrow down the amount.

"Then, we […] ask how many of those open/RMR teams could remain after all other invited teams (e.g., closed qualifier invitees, direct invitations) could potentially be eliminated. Whichever number is lower is the number of teams that get RMR points."

To use two examples to show the process in action, we can look at both Flashpoint 3, the EU RMR event, and CBCS Elite League S1, the SA RMR event. For EU, there were 11 teams invited by RMR and 8 teams that qualified from open qualifiers making for a maximum of 19 spots that could award RMR points. The closed qualifier was made up of the 8 teams from open qualifiers and 8 invites by other means (of Flashpoint's choosing), with only five teams advancing to the main event stage. While only five teams could advance to the next stage, the format of a double elimination bracket meant that those five teams could have been the only teams that came through open qualifiers. The end result would be 11 teams from RMR invites + 5 from open qualifiers in the main event, which added up to 16 slots getting points at Flashpoint 3.

CBCS Elite League Season 1 featured 4 teams invited by RMR and 4 teams coming from open qualifiers for a maximum of 8 spots getting points. Four teams advanced from the closed qualifier to the main event, and all four open qualifier teams could have been the ones to advance. We can't stop there though, as the main event featured 16 teams, 8 of which were invited by CBCS's own ranking. The initial stage of the main event was a Swiss group, where all 8 of the performance teams could in theory advance to the eight-team playoffs. This meant that the event would indeed award RMR points to eight spots in contrast to the initially thought four.

It couldn't be as simple as just having a different formula for determining the RMR points distribution for the first series of events though, as once again there was a lack of parity between regions. Perfect World, who were running the Asia event, cited the public FAQ and held that only four teams from their event would receive points. Under the current understanding, four teams were invited by RMR and four teams came from open qualifiers, with no other invites existing, clearly pointing toward all eight teams getting RMR points. Of course this could be fixed if our assumption that all four invited teams were by RMR and instead Asia was more akin to Oceania where only TYLOO was invited by RMR, and the other three invites were by other factors. Having a maximum count of five, limited down to four by the format, would actually make sense, but Perfect World themselves confirmed that all four invited teams were based on their RMR points from the previous year ending up with yet another scenario of differing communication and standards between regions.

I won't go over the other issues relating to Flashpoint 3 and the EPIC CIS League that didn't directly related to the RMR system, like the conflicts of interest and lacking measures against integrity, but know that those too showed lack of diligence on the part of Valve in their selection and policing of tournament organizers.

There were only two Summer series RMR events, in CIS and South America, and the only glaring issue was that only these two regions had Summer events. The points distribution had reverted back to the old system from the FAQ, meaning 5 spots award points in CIS, and 4 did in SA, based on the number of teams invited by RMR in each event. Having an additional (third) RMR event helps to not have any individual event have too much performance and better measures the best teams over a period of time. For the other regions, primarily North America and Europe, the final RMR event mattered so much more than the first one that the first almost didn't matter at all.

IEM Fall

IEM Fall was to be the Fall series of RMR events, with ESL running all of the regions. While there were fewer inconsistencies between regions this time around, there were yet more inconsistencies between RMR series. Valve once again had the baffling idea of changing the points distribution rules so that the publicly available ruleset was only actually used in 2 out of 14 RMR events for the Stockholm Major cycle.

Instead of directly tying the number of placings that award RMR points to the number of teams invited by RMR, the number of spots that awarded RMR points would simply be equal to the number of slots that region had at the Major, or four spots for the small regions. So despite 12 (or 15 if you include the closed qualifier) teams being invited by RMR to IEM Fall Europe, only the top 11 teams would receive RMR points in that region. While this isn't the worst case scenario of having different points distribution rules between regions for a single series, having three different formats for each series of RMR events makes the playing field unlevel on a time scale. Teams who played in only the Spring series would have an advantage over teams who only played in the Summer or Fall series events, since it was easier on average to earn points in those events.

Another issue, which wasn't exclusive to IEM Fall but was the most apparent, is the freedom that tournament organizers had with choosing other aspects of the event format. The main way that this directly caused issues is when the number of teams in the main event was the same as the number of spots that awarded RMR points. This isn't particularly impactful on the standings where there are 16 teams in the event, as last place would award only a few hundred points. The issue becomes significantly worse the fewer teams there are in the event, and was first seen at cs_summit 8, at which Evil Geniuses lost all their matches, went out last, and still earned 950 RMR points, simply for being invited based on their placing from the 2020 RMR cycle.

The problem got even worse in the small regions for IEM Fall, where only four teams competed in Asia, Oceania, and South America, despite all the previous events in those regions having eight or even sixteen teams. With a last (fourth) place finish at any of these events still awarding 2031 RMR points, it meant the two teams directly invited to the event by the RMR standings were the only ones who had a shot to qualify for the Major before the events even began. This particular phenomenon was also something that was not present in the other regions as North America, CIS, and Europe all had many placings at the events that did not award RMR points, so any team invited by RMR was not guaranteed to earn any points. So while the third RMR event was essentially meaningless for the small regions, the complete opposite happened in NA and EU, where IEM Fall was essentially the only RMR event that mattered.

When I mentioned before that lacking the Summer RMR event only hurt regions which didn't feature one, it was mainly due to the increase in points awarded at the Fall events. In North America specifically, every single team participating needed to place in the points in order to qualify for the Major, with the exception of EXTREMUM, who could only qualify without placing in the points in an extremely unlikely scenario. To put this into other words, whichever five teams earned points at IEM Fall would be the five teams to attend PGL Stockholm (which was indeed what happened in reality). In that case, what was the purpose of the first RMR event other than deciding which teams got a direct invite and got to skip the qualifiers?

The final problem with IEM Fall is something that never should have been a problem in the first place and was luckily remedied before the event began. When I was discussing the basics of how the RMR cycle worked, I explained that each team could register up to seven people, the five players along with an optional coach and an optional substitute. However, both of these roles couldn't be fully utilized in almost all of the RMR events due to a new rule Valve added for the 2021 cycle. As a direct result of the coach bug scandal that resulted in over 30 individuals banned by ESIC and Valve for abuse of a game bug that allowed coaches to view portions of the map they shouldn't have been able to see, Valve banned coaches from participating in any way during online events. As all of the RMR events except for IEM Fall Europe were held online, coaches couldn't participate, and many tournament organizers only asked for teams to submit a substitute.

Since IEM Fall Europe was held on LAN, then the full extent of the new substitution system should have been able to be fully implemented, with coaches not having to be forced to play in the event that a normal player could not participate. For some reason, however, ESL only asked teams to submit a single coach or substitute, despite this being the only event in the entire RMR cycle that could take advantage of the new system. A team actually was affected by this, as Mohammad "⁠BOROS⁠" Malhas was unable to acquire a visa to travel to Sweden and play with Endpoint. They were then told they would have to play with their new coach Allan "⁠Allan⁠" Hender, who had never been a professional player, completely undermining the entire premise of adding the substitute slot in the first place. Luckily ESL realized this and gave teams a last-minute opportunity to register an additional substitute if they wanted to, and Endpoint were able to play the event with Sabit "⁠mirbit⁠" Coktasar (still suffering an event points deduction, of course).

What should be changed?

So what were the main issues on the whole for the Regional Major Ranking system? A lack of standardization in formats meant that each RMR event was different in how easy or hard it was to earn points, making the requirements for qualification for any given team different based on which region they were in or when they participated.

Does this make the RMR system unsalvageable?

No, definitely not. The current system could be viable if much stricter parity was enforced between regions. In May, Valve announced they were accepting proposals for 2022 and 2023 Majors, saying that they wanted to run a single RMR event in five regions, the same as the 2021 cycle but with Asia and Oceania combined, on LAN before each Major. If only one event is part of the cycle, then the ranking doesn't really matter, and is more of a single big regional Minor that includes all teams from the region. Even if this is what Valve said back in May, it doesn't mean this system is fully locked in for the next few Major cycles and could definitely be changed.

At the end of the day, staying away from legacy invites is probably the best course of action and making teams re-qualify each time keeps the best teams at the Major. Putting all qualifications into a single event makes things very cutthroat, though, and many in the community don't put a lot of stock into single-event performances. Either way, running six RMR events every year just clearly isn't viable, so a single LAN qualifier in each region could still act as a good method of determining the 24 teams at the Major.

The final piece of the system that should be addressed is the slot allocation of regions. The current allocation is based on the results of the Berlin Major, in which Europe has 11 spots, North America and CIS have 5 spots, then South America, Asia, and Oceania each have 1 spot. With each of those numbers is a distribution between Legends, Challengers, and Contenders, with North America famously having three Legends spots as a result of Liquid, NRG, and Renegades all making playoffs in Berlin. Continually updating the distribution based on the last Major's results would be able to change how many of the total number of spots went to Legends, Challengers, or Contenders, but the total number of available spots would not change.

If we look at the results of the Stockholm Major, NA would have 1 Legends spot (FURIA), 2 Challengers spots (Liquid, Evil Geniuses), and 2 Contenders spots (paiN, GODSENT). This better accounts for the strength of the region by having fewer Legends spots, but no matter what the results at the Major were, there would still be five spots for NA. With the rise of CIS and the decline of NA, it would make sense to change the allocation to better fit the current state of the global scene. Therefore there has to exist some kind of mechanism to alter the total number of spots that a region can have allocated based on region strength changing over time. This would likely have to be done manually by Valve on a Major by Major basis.

As the new year approaches, so too does the next Major cycle. As of now, there has not been any formal announcement on who will run the next Major, or where it will take place. However, following the soft reset of the Major cycle due to the pandemic and the return to LAN, now is the perfect time for Valve to overhaul the Major qualification system and make something that will last for years to come.