Erroneous league assignment

ArtNJ · 07-27-2012, 01:22 AM

(07-27-2012 12:46 AM)Kamikaze28 Wrote: I understand your point, but once again I'd like to explain that statements like "no one should go 30-0 and be in Gifted." are highly relative, especially in the beginning of a rating system. How good were these 30 opponents? The system doesn't know this from the start. It's bound to be cautious. 30 wins against unranked or freshly ranked players hardly means anything. 30 wins against Masters or Super Titans with high confidence ratings is a very valid reason not to be in Gifted League.

Here we have our first real disagreement. If the system adjusted more dynamically, after winning a bunch of games (say 4-6), you should be being matched with other people dominating unranked, and as you continue dominating, you should be matched with other people showing that level of dominance. If this is done dynamically, ELO or not, it just doesnt take that long to prove yourself.

Constrast this system with both a new and established ELO system:

New: starting rate in Great Big War Game is 1000, and the K value is 20 (meaning you gain or lose 10 points against an equal opponent). Although its very early, you can see from the leaderboard who has been dominating, and those are all very good players. Simply put, you cant get much above 1100 without being quite good, because the points you get for beating noobs goes down and down. It obviously doesnt track skill perfectly yet, many noobs are free points, and the system is too new, but you *can* match yourself against quality opponents just by looking at the leaderboard (GBWG doesnt do that automatically, but that doesnt change the point).

Established: The #4 player in the Hero Academy League is "only" 22-2. After starting like 12-0, he started exclusively challenging the top rated folks, and dominating them. Under the Outwitters system, he would be 24-0 and bored stiff battling noobs. He probably would have just quit -- he doesnt play all that much, and only takes high challenge matches.

Outwitters doesnt work as quickly as either system, and its a problem for the skilled -- they get bored.

GreatGonzales · 07-27-2012, 01:29 AM

Yeah. Basically, after winning 30 games in a row, I should be playing super-titans, and I (largely) was not. With ELO, I probably would be.

Kamikaze28 · 07-27-2012, 02:25 AM

(07-27-2012 01:29 AM)GreatGonzales Wrote: Yeah. Basically, after winning 30 games in a row, I should be playing super-titans, and I (largely) was not. With ELO, I probably would be.

Okay, now let's come back full circle.

You feel that it took too long for you to be recognized by the system as a skilled player. That's a valid concern in my opinion and as far as my knowledge goes Alex is monitoring these things actively and he is not opposed to modifications as we have seen.
You argue, that ELO would have prevented this from happening. Probably, based on ArtNJs experience with the Hero Academy league and my rudimentary understanding of ELO.
You (or ArtNJ) conclude that ELO would have been the better choice for Outwitters. Given that we don't know how the system works exactly, we cannot say for sure that it is not something based on ELO with a few quirks and tweaks.

I find that the thread title 'Erroneous league assignment' is misleading after this discussion. I'd rather call it 'too slow league advancement'.

Cookie · 07-27-2012, 02:53 AM

(07-27-2012 01:29 AM)GreatGonzales Wrote: Yeah. Basically, after winning 30 games in a row, I should be playing super-titans, and I (largely) was not. With ELO, I probably would be.

Your "largely" suggests that you were playing some though, which I personally think is not bad after 30 games "only", which were most likely all started over the course of just a few days, or maybe a week. I think you must have been the one beating me in one of the short games on Sharkfood, for example, and I am rated #3 or #4 in one of the Master leagues, and would be guessing that I am at least in Top 10% of users (it is not like the system makes it very easy to know this, right?)

Not quite sure how the leagues are setup, but I am assuming that Super Titan is a very small fraction (1%? 5%?) of total users, which also makes sense as it has to be something to strive for and not something you just enter after 10 or 20 games (generally speaking).

Even though I have something like 80-20 stats, I feel that the system has been relatively good at pairing me, and that I am generally getting good matches now.

One thing that I think needs to be considered is that there are many new players coming into the game. In fact, for any single player, it would already be possible to pinpoint his level fairly accurately by matching him against 10 players who are spread out in the ranks. But, most of the new players would end up near the bottom anyway (i.e. losing 8-10 of these placement games) which would be neither encouraging nor productive for the system as a whole. And, additionally, there is not enough strong players to support doing this, and would end up swamping strong players with games against newbies instead.

Now, with many non-ELO systems, we could instead guess that the system is trying to infer a skill rank based on yours and the skill of your opponents algorithmically. E.g., the system looks at you and says "this player has got 20 wins against noobs, so he is probably better than a noob". But, this does not prove that the user is a Master, and much less a Super Titan (although those words are all relative anyway, and we don't know how many players are in each group). Such systems tend to be cautious, not least to avoid having to demote users (something which just proves others got better at the game, but is mostly interpreted by users as if they got worse).

The reason is that for a difference of something like 300 ELO points, there would already be a probability of 95% for the higher rated player to win. Winning 20 games in a row against noobs only suggests that you are 3+ skill levels better than them, and not that you are a top player. Guessing that you are 5 skill levels better would risk getting you mismatches the other way, but increasing the standard deviation so you get a couple of games with players who are 5+ skill levels better in your next set would make a lot of sense. So with 1000 points (10 "skill levels" if one level is 100 points) from top to bottom, you would perhaps have to prove yourself in 3 or 4 groups of 20 matches before you would reach anywhere near the top, even if you deserve it.

The problem will be naturally mitigated when there are fewer players coming in to the bottom compared to the already established players. And it is probably working rather well already for 95% of users.

One other solution would be to have a reasonably skilled computer opponent that you can play your first placement matches against. That way, the system should be able to conclude pretty quickly that you are not a newbie (or that you learn quickly/have experience with similar games) and give you better match-ups for placement matches against other human players.

As an aside, perhaps the points added and subtracted at the end of the games is just a biased version of the internal rating system? Or perhaps, they are something completely different Smile

.

ishu76 · 07-27-2012, 02:57 AM

I have 86 wins. I was in the Clever League and got demoted to the Fluffy league. No idea why. I am ranked 1 there, obviously. I have 432 points and the person below me, i.e., rank 2 has 52 point. Obviously I am in the wrong league. I am not a master, but I am not fluffy either.

Where can I find a list of leagues and the heirachy?

ArtNJ · 07-27-2012, 03:01 AM

(07-27-2012 02:25 AM)Kamikaze28 Wrote:
(07-27-2012 01:29 AM)GreatGonzales Wrote: Yeah. Basically, after winning 30 games in a row, I should be playing super-titans, and I (largely) was not. With ELO, I probably would be.

Okay, now let's come back full circle.

You feel that it took too long for you to be recognized by the system as a skilled player. That's a valid concern in my opinion and as far as my knowledge goes Alex is monitoring these things actively and he is not opposed to modifications as we have seen.

You argue, that ELO would have prevented this from happening. Probably, based on ArtNJs experience with the Hero Academy league and my rudimentary understanding of ELO.

You (or ArtNJ) conclude that ELO would have been the better choice for Outwitters. Given that we don't know how the system works exactly, we cannot say for sure that it is not something based on ELO with a few quirks and tweaks.

I find that the thread title 'Erroneous league assignment' is misleading after this discussion. I'd rather call it 'too slow league advancement'.

Well said. The ELO discussion is interesting, but really just a side issue. The system we have is the one that will go forward, and the issue is tweaking it to permit faster movement.

Kamikaze28 · 07-27-2012, 03:02 AM

(07-27-2012 02:57 AM)ishu76 Wrote: I have 86 wins. I was in the Clever League and got demoted to the Fluffy league. No idea why. I am ranked 1 there, obviously. I have 432 points and the person below me, i.e., rank 2 has 52 point. Obviously I am in the wrong league. I am not a master, but I am not fluffy either.

That sounds more like an erroneous league assignment. Something fishy is going on there if your numbers are correct.

(07-27-2012 02:57 AM)ishu76 Wrote: Where can I find a list of leagues and the heirachy?

Either in the help section or after this break:

Super-Titan League
Master League
Gifted League
Clever League
Fluffy League

ishu76 · 07-27-2012, 03:18 AM

Thank you for your quick response. How do I get my league corrected? Also how do divisions play into the leagues. Been looking on this website to understand the relationship between Leagues and Divisions, and can't seem to find it anywhere.

Bluegodzill · (This post was last modified: 07-27-2012 03:30 AM by Bluegodzill.)

(07-27-2012 03:18 AM)ishu76 Wrote: Thank you for your quick response. How do I get my league corrected? Also how do divisions play into the leagues. Been looking on this website to understand the relationship between Leagues and Divisions, and can't seem to find it anywhere.

The leagues are like very general divisions in a way, in each league there are a lot of divisions of people with about 100 people in each division. Be patient and keep winning and you'll be promoted sometime soon.

Kamikaze28 · (This post was last modified: 07-27-2012 03:30 AM by Kamikaze28.)

(07-27-2012 03:18 AM)ishu76 Wrote: Thank you for your quick response.

You're welcome.

(07-27-2012 03:18 AM)ishu76 Wrote: How do I get my league corrected?

Continue winning. To my knowledge, there is no way to manually have the devs alter any individual league assignment.

(07-27-2012 03:18 AM)ishu76 Wrote: Also how do divisions play into the leagues. Been looking on this website to understand the relationship between Leagues and Divisions, and can't seem to find it anywhere.

(07-15-2012 10:42 PM)Kamikaze28 Wrote: Divisions are created on-demand. Each division can hold up to 100 players. They exist mainly to give ranks a meaning - being #28 of 100 conveys more sense than being #26234 of 54981.
Divisions are not meant to separate by skill any further than leagues do, so there really is no way to compare two divisions.

Also, this is going a little bit off-topic.