087 - UK Rankings Proposals

There's a discussion currently in full swing on the Wyrd forums at the moment on the subject of the Rankings system, and whether it should be changed going forwards. Brace yourself - things are about to get ramble-y!

For those not in the know, the current system ranks the tournament performance of each player in the UK by assigning a score for each event they attend. Taking first place at an event is worth 100 points (assuming the event had a turnout of at least 20 players), with second place receiving 95 points, third place receiving 90 points, etc. If there are less than 20 attendees, the maximum score for first place is reduced by 1 point for each player below 20 (so winning a 15 player event would score 95 points), and all other scores are reduced proportionally. A player's best three scores from all the tournaments they enter in a rolling 12 month period are added together to give a total score out of 300.

The potential changes that are being discussed are:

1) Increase the number of events counted in your score from a players best 3 to their best 4.
2) Increase the threshold for scoring "maximum" points from being a 20 player event, to something higher. 24, 26, 28 and 30 have all been suggested for the new threshold.

I have a suspicion that a lot of people are weighing in on this discussion without having consulted any actual data first, so I figure that looking at some numbers will help people frame the debate more clearly:

Increase the number of scoring events from 3 to 4

First, we must consider what this change is intended to achieve. The argument goes that the scores at the top of the table are pretty close together, and that moving to a system of four events would distinguish these people a little more.The top 40 of the rankings table as at time writing (29th January 2014) looks like this:

 

The third column of the table shows the number of events that each player has on record that are not currently contributing to their score - ie. the excess events above three.

Couple of initial observations

• As you progress down the table from high ranks to low, the number of unscored events decreases.
• Outside the Top 10, the vast majority of players have only 3 or 4, or less, events to their name. Even among the Top 40, players attending only 3 events in a year appears to be the most common type of player.
• There is a very small number of people that have attended a large number of events - where I'm classing "large number" as more than 6 events.
• The scores at the top aren't particularly grouped up, with the exception that three players are sitting at two-hundred and seventy something. One of these players has only one other tournament that could be added to their score, where the other two have 6 and 8 events to choose the best result from. Intuitively, the players with more options are at an advantage in the tie break.

Highlighted green are the players who would benefit significantly by being able to add another event to their score - because they have more events to choose their next best from. Highlighted orange are players that would benefit somewhat by adding their 4th (and only other) event to their score - this 4th event might not necessarily be a high scoring one though. Highlighted red are the players that would be disadvantaged by the fact that they have only attended three or less events in the last 12 months, so could not gain any more points than they already have.

In moving to a "Best 4" system, the effect would be that players in green would, generally, move up significantly, players in orange would move up a bit, and players in red would move down.

Below Rank 40 there are significantly more players who have only 2 or 3 tournaments to their name. Most of these players would remain in the same rankings order, however the gap between the "Top" players and the "Rest" would widen significantly, creating what would essentially be a two-tier system.

I'd expect the impact for "The Rest" to be:

a) Rankings become less relevant. If you are in the "lower" tier (having 1, 2 or 3 tournaments to your name) it is harder for you to make a meaningful comparison between your score and the "Top" players.

b) It is harder for newer players to break into the higher ranks. Some of the players that have recently jumped into the Top 20 would not be there under a "Best 4" system, and introducing new blood to the competitive mix is always good to keep things fresh.

These two points are why I'm unsure about this particular change - the benefits (in terms of distinguishing the "Top" players) are marginal at best, and affect only a limited number of people, whereas the *vast* majority of people for whom 4 events in a year is just too much of a stretch are disadvantaged, so my preference would be to stick with a "Best 3" system.


Increase the Threshold for Scoring Max Points from 20 to X

This second change is intended to reward attendance at an event with "many" people, and award less rankings points to an event where "fewer" people attend.

The rationale for this must presumably be: "It is harder to place higher at an event when there are more players".

What makes an event more "difficult" to win? The strength of the competition is probably the most obvious answer. Unfortunately, so many factors contribute to a measure of "strength of competition" that it's not that easy to fully analyse.

Some things we can examine though. I've seen it suggested that smaller tournaments could be easier to win due to attracting a different mix of players - with more local/casual players and less of the experienced tournament crowd. I checked this out by analysing the last 12 months of events, calculating an average score* for all the participants, and plotting that against the number of participants at the event.

As you can see, there isn't really any trend of smaller events being a pushover due to having easier opponents - the smallest events tend to have a similar average ranking score attending them as the larger ones, reflecting the willingness for people to travel a long way for an event, even more so if the event is ranked.

The number of rounds played also has an impact - it is a fact that winning 4 games in a row is necessarily harder than winning 3 games in a row. Regrettably, due to time constraints, most Malifaux events recently have run to only 3 rounds. In an ideal world, with 16 players there would be a 4th round, and with 32 players there would be a 5th round. If this recommendation was implemented, it would provide justification all by itself for larger events scoring more points than smaller.

Limited by playing time constraints (it is difficult to get 4 games, let alone 5 games, into a single day event) and with tournaments typically featuring only 3-games, we should consider the tournament "journey" for a player who goes on to win 1st place in a small tournament and in a large tournament, and we notice that the journey is pretty much identical. Their first game is against a random opponent which we would generally expect them to win, their second game will tend to be against a more difficult opponent than their first, and their third and final game will generally be against another of the top players at the event (which, by the scatter graph above, isn't actually going to be an easier opponent in a smaller event than a larger event). What is it about the three games played at the larger event that we should deem it be worth more points? It doesn't sound any more arduous or accomplished - the only difference seems to be how many people are sitting at adjacent tables.

Many people seem to think the threshold should be increased by a fair margin, and I believe the current 20 is somewhat arbitrary, so I wouldn't be opposed to edging it up slightly to 24, but anything beyond that seems rather unnecessary. Indeed, there have been only *four* events in the last 12 months with 24 or more attendees. (Having 24, 29, 33 and 39 participants), and two of those were M1E events, so going beyond that level seems to be out of step with what we've seen of attendances in 2013. Indeed, increasing the threshold to 24 would cause *seven* events between 20 and 23 attendees to no longer have counted max points. That feels somewhat on the ruthless side, and I admit that it would make it pretty much impossible to get multiple players at the top of the table on the same score. Is that all that's required? Probably.

* Players were assigned weights from 100 to 1 based on their current ranking position, and the score was the mean average of those weights. Players absent from the rankings were assumed to be local/casual players and assigned a weight of 1.