Erroneous league assignment

[Cookie]

(07-27-2012 01:29 AM)GreatGonzales Wrote:  Yeah. Basically, after winning 30 games in a row, I should be playing super-titans, and I (largely) was not. With ELO, I probably would be.

Your "largely" suggests that you were playing some though, which I personally think is not bad after 30 games "only", which were most likely all started over the course of just a few days, or maybe a week. I think you must have been the one beating me in one of the short games on Sharkfood, for example, and I am rated #3 or #4 in one of the Master leagues, and would be guessing that I am at least in Top 10% of users (it is not like the system makes it very easy to know this, right?)

Not quite sure how the leagues are setup, but I am assuming that Super Titan is a very small fraction (1%? 5%?) of total users, which also makes sense as it has to be something to strive for and not something you just enter after 10 or 20 games (generally speaking).

Even though I have something like 80-20 stats, I feel that the system has been relatively good at pairing me, and that I am generally getting good matches now.

One thing that I think needs to be considered is that there are many new players coming into the game. In fact, for any single player, it would already be possible to pinpoint his level fairly accurately by matching him against 10 players who are spread out in the ranks. But, most of the new players would end up near the bottom anyway (i.e. losing 8-10 of these placement games) which would be neither encouraging nor productive for the system as a whole. And, additionally, there is not enough strong players to support doing this, and would end up swamping strong players with games against newbies instead.

Now, with many non-ELO systems, we could instead guess that the system is trying to infer a skill rank based on yours and the skill of your opponents algorithmically. E.g., the system looks at you and says "this player has got 20 wins against noobs, so he is probably better than a noob". But, this does not prove that the user is a Master, and much less a Super Titan (although those words are all relative anyway, and we don't know how many players are in each group). Such systems tend to be cautious, not least to avoid having to demote users (something which just proves others got better at the game, but is mostly interpreted by users as if they got worse).

The reason is that for a difference of something like 300 ELO points, there would already be a probability of 95% for the higher rated player to win. Winning 20 games in a row against noobs only suggests that you are 3+ skill levels better than them, and not that you are a top player. Guessing that you are 5 skill levels better would risk getting you mismatches the other way, but increasing the standard deviation so you get a couple of games with players who are 5+ skill levels better in your next set would make a lot of sense. So with 1000 points (10 "skill levels" if one level is 100 points) from top to bottom, you would perhaps have to prove yourself in 3 or 4 groups of 20 matches before you would reach anywhere near the top, even if you deserve it.

The problem will be naturally mitigated when there are fewer players coming in to the bottom compared to the already established players. And it is probably working rather well already for 95% of users.

One other solution would be to have a reasonably skilled computer opponent that you can play your first placement matches against. That way, the system should be able to conclude pretty quickly that you are not a newbie (or that you learn quickly/have experience with similar games) and give you better match-ups for placement matches against other human players.

As an aside, perhaps the points added and subtracted at the end of the games is just a biased version of the internal rating system? Or perhaps, they are something completely different .