Just a little praise I think OML needs

CombatEX · 09-26-2012, 04:48 AM

(09-26-2012 03:38 AM)Harti Wrote:
(09-26-2012 02:38 AM)Ravernoth Wrote: The World Cup tournament alone has enough games to be statistically valid. Whether it's 2:1 or 3:1, it's a big issue.
Haha, sorry. While I'm your side it's false to claim that a sample size smaller than 10- or 100,000 shows a statistically valid empirical result.

Good news is that Alex likes the 8 wit idea.
Sounds like we're testing it next week. =)

By we do you mean the current beta testers in a test build or people in friendly matches artificially limiting wits that would emulate this change as someone suggested in another thread? I'd like to help test this change if it is the latter (or the former too if I manage to get into the beta).

**Harti** · 09-26-2012, 04:54 AM

I'm not sure about it but chances are good that it's coming to the beta build.

@vivafringe: Seems you owned me! I didn't like these classes but we got told very often that you usually need a huge sample for the significance level to be good.

GreatGonzales · 09-26-2012, 05:08 AM

(09-26-2012 04:42 AM)vivafringe Wrote: This isn't true! It's a common fallacy that you always need a huge sample size to prove things. It turns out our data is so incredible that there is a huge probability that P1 is advantaged even with our tiny sample size. The standard statistical test for this type of data is, as mentioned in the other thread, to look at the ties. In the other thread, there were 17 tied games where the players won as 1p, and 2 tied games where the players won as 2p. Let's say we want to test whether there is an advantage to playing first. A naive view would be that player 1 has a 50% winrate vs. player 2. Let p be the winrate of player 1. We'll do a 2-sided test, even though a 1-sided could be argued to be more appropriate.

Ho (the null hypothesis): p = .5
Ha (the alternative hypothesis): p does not equal 0.5

The number of tied games where p1 follows a binomial distribution with n = 19, p = 0.5. Our p-value is 0.000729. In other words, if the actual winrate was 0.5, we would only expect a result this unusual 1 out of 1372 times.

Hope OML sees this.

**TheQwertiest** · 09-26-2012, 05:10 AM

i thought this thread was suppose to be for the praise OML needs xD

Necrocat219 · 09-26-2012, 05:14 AM

(09-26-2012 05:10 AM)TheQwertiest Wrote: i thought this thread was suppose to be for the praise OML needs xD

umm... it was, but I kinda made a wrong assumption in my post so the topic got changed ^^;

awpertunity · (This post was last modified: 09-26-2012 07:16 AM by awpertunity.)

(09-26-2012 04:42 AM)vivafringe Wrote:
(09-26-2012 03:38 AM)Harti Wrote: Haha, sorry. While I'm your side it's false to claim that a sample size smaller than 10- or 100,000 shows a statistically valid empirical result.

This isn't true! It's a common fallacy that you always need a huge sample size to prove things. It turns out our data is so incredible that there is a huge probability that P1 is advantaged even with our tiny sample size. The standard statistical test for this type of data is, as mentioned in the other thread, to look at the ties. In the other thread, there were 17 tied games where the players won as 1p, and 2 tied games where the players won as 2p. Let's say we want to test whether there is an advantage to playing first. A naive view would be that player 1 has a 50% winrate vs. player 2. Let p be the winrate of player 1. We'll do a 2-sided test, even though a 1-sided could be argued to be more appropriate.

Ho (the null hypothesis): p = .5
Ha (the alternative hypothesis): p does not equal 0.5

The number of tied games where p1 follows a binomial distribution with n = 19, p = 0.5. Our p-value is 0.000729. In other words, if the actual winrate was 0.5, we would only expect a result this unusual 1 out of 1372 times.

How did you get that? I get an even smaller p-value of
0.000364303588866965 for the chances of P1 winning 17/19 or more of the games, or 1 in 2744.

EDIT: Oh nevermind you used a 2-sided test for some reason haha. So mine is exactly half of yours Tongue

vivafringe · (This post was last modified: 09-26-2012 12:15 PM by vivafringe.)

yeah 1-sided is the correct choice. I just used 2-sided because the Ho/Ha is slightly easier to grok, and because the p-values are so small that they're essentially 0 anyway.

worldfamous · (This post was last modified: 09-26-2012 12:21 PM by worldfamous.)

(09-26-2012 07:14 AM)awpertunity Wrote:
(09-26-2012 04:42 AM)vivafringe Wrote:
(09-26-2012 03:38 AM)Harti Wrote: Haha, sorry. While I'm your side it's false to claim that a sample size smaller than 10- or 100,000 shows a statistically valid empirical result.

This isn't true! It's a common fallacy that you always need a huge sample size to prove things. It turns out our data is so incredible that there is a huge probability that P1 is advantaged even with our tiny sample size. The standard statistical test for this type of data is, as mentioned in the other thread, to look at the ties. In the other thread, there were 17 tied games where the players won as 1p, and 2 tied games where the players won as 2p. Let's say we want to test whether there is an advantage to playing first. A naive view would be that player 1 has a 50% winrate vs. player 2. Let p be the winrate of player 1. We'll do a 2-sided test, even though a 1-sided could be argued to be more appropriate.

Ho (the null hypothesis): p = .5
Ha (the alternative hypothesis): p does not equal 0.5

The number of tied games where p1 follows a binomial distribution with n = 19, p = 0.5. Our p-value is 0.000729. In other words, if the actual winrate was 0.5, we would only expect a result this unusual 1 out of 1372 times.

How did you get that? I get an even smaller p-value of
0.000364303588866965 for the chances of P1 winning 17/19 or more of the games, or 1 in 2744.

EDIT: Oh nevermind you used a 2-sided test for some reason haha. So mine is exactly half of yours

So, for us laymen, what you're saying is something like, if you flip a coin 19 times there's a 1/1372 chance that it will land on heads 17/19 times? Henceforth, it's unlikely that its a coincidence that P1 won 17/19 times.

garcia1000 · (This post was last modified: 09-26-2012 12:45 PM by garcia1000.)

Here's a real thing that happened. For those of you who don't know, I'm a full-time game designer. A week or so ago I met up with some of my game designer friends, most of them are working on puzzle games like Bejeweled clones, but some of them are FPS technical designers and one of them works for an evil Skinner box type game. lol we make fun of him for that all the time.

Anyway I told these guys about Outwitters, and I praised the elegance of the design. Like everyone knows that hex-based games are usually poison because of the association with grognard wargames, but I said this one was great! And I described the system to them and they agreed it was interesting. Some of them thought that the wit resource was overloaded since it's used for everything, and makes the system too interconnected and hard to 'turn the dials' but I told them that in practice it works really well, the small maps make things interesting and so that movement doesn't become too much of a wit hog, etc.

Then they asked me, this is basically the verbatim conversation as it happened, as best as I can recall:

"Sounds good, did they balance second player through map design or through wit levels?"

"Well... at the moment nothing. I'm not sure if they are going to do anything about that."

"Wait, you mean the second player gets nothing?"

"Yeah."

"What? Are you serious? The first player's starting situation strictly dominates?"

"Yeah..."

"It doesn't take rocket science to see there's an easily fixable problem here man. Are you sure their game design is great?"

Then they sort of looked at me in disbelief and laughed a bit. After that they quickly lost interest and we discussed other boring stuff lol. Anyway I was sort of embarrassed. Next time I see them I will plan out what to say to convince them, I'm sure they would like the elegance of this game system

Edit: I am glad I didn't have to mention 1P queue stacking, lol

vivafringe · (This post was last modified: 09-26-2012 01:22 PM by vivafringe.)

(09-26-2012 12:20 PM)worldfamous Wrote:
(09-26-2012 07:14 AM)awpertunity Wrote:
(09-26-2012 04:42 AM)vivafringe Wrote: The number of tied games where p1 follows a binomial distribution with n = 19, p = 0.5. Our p-value is 0.000729. In other words, if the actual winrate was 0.5, we would only expect a result this unusual 1 out of 1372 times.

How did you get that? I get an even smaller p-value of
0.000364303588866965 for the chances of P1 winning 17/19 or more of the games, or 1 in 2744.

EDIT: Oh nevermind you used a 2-sided test for some reason haha. So mine is exactly half of yours
So, for us laymen, what you're saying is something like, if you flip a coin 19 times there's a 1/1372 chance that it will land on heads 17/19 times? Henceforth, it's unlikely that its a coincidence that P1 won 17/19 times.

There's a 1/1372 chance that if you flip a coin 19 times, you'll get 0,1,2,17,18 or 19 heads. Predictably it's half that chance if you only care about 17, 18 or 19. "This unusual" actually means "this unusual or even more unusual," but people get glossy eyed when I say the second thing.