Dominion Strategy Forum

Please login or register.

Login with username, password and session length
Pages: 1 [2] 3  All

Author Topic: Bot "Pro" Games Effect Leaderboard  (Read 27115 times)

0 Members and 1 Guest are viewing this topic.

Polk5440

  • Torturer
  • *****
  • Offline Offline
  • Posts: 1708
  • Respect: +1788
    • View Profile
Re: Bot "Pro" Games Effect Leaderboard
« Reply #25 on: September 24, 2014, 09:07:15 am »
0

This is just selection bias: you won't notice the hundreds of guys who play a bunch against bots and achieve nothing spectacular, but you will notice the one guy who runs extremely hot. I predict he'll be nowhere near his current level after 110 more games.

Trusting the description of his play in this thread is accurate and taking into account the very low prior, it's extremely unlikely he's actually very good. Moreover, I've actually proofread the 3rd Rating Analysis article many months ago, and one of its conclusions was that in the long run it's impossible to maintain your rating when playing much lower ranked opposition. This happens because both Goko and to a lesser extent Iso underappreciate the amount of randomness in Dominion and thus give disproportional weights to upsets.

I see there's still interest in this matter; AI is extremely busy right now, but I'll ask him if he's willing to post it when he has a little more time (I don't think he's reading this thread).

Also, Trueskill is designed to be very good when players of similar rankings play eachother (which is why microsoft paired it with an automatch system for its games). Trueskill will be bad if players don't sufficiently mix. There aren't "enough" comparisons made and comparisons between sets of players who don't play each other become much less reliable.
Logged

Holger

  • Minion
  • *****
  • Offline Offline
  • Posts: 736
  • Respect: +458
    • View Profile
Re: Bot "Pro" Games Effect Leaderboard
« Reply #26 on: September 24, 2014, 09:37:53 am »
0

This is just selection bias: you won't notice the hundreds of guys who play a bunch against bots and achieve nothing spectacular, but you will notice the one guy who runs extremely hot. I predict he'll be nowhere near his current level after 110 more games.

That's possible, but substracting 3σ as in Isotropish's level should mean that any player is above his nominal level at 95% 99,7% certainty. And CPA has been improving his level further since this thread was started.
If Iso underestimates upsets, the skill of a strong player mainly playing bots should actually be underestimated, shouldn't it?

Many people have been asking AI to post his article in his "Part 2" thread; but maybe he'll listen to you since you've proofread it  ;) 
Does the article give a hint why CPA is ranked so much lower at Goko in spite of them substracting only 2σ, and CPA playing dozens of games per day (so Goko's daily uncertainty increase can only help him)?

Edit: fixed percentage of 3σ level.
« Last Edit: September 29, 2014, 12:58:42 pm by Holger »
Logged

Mic Qsenoch

  • 2015 DS Champion
  • *
  • Offline Offline
  • Posts: 1709
  • Respect: +4329
    • View Profile
Re: Bot "Pro" Games Effect Leaderboard
« Reply #27 on: September 24, 2014, 09:45:05 am »
+5

Not against real players, but against bots, you can misplay and win inspite of it (or even because of it). Take a look at the game I linked, it's one of the worst but not the only one. Seriously, an opponent buying nothing but Silvers, Golds and green cards could beat him in that game with some decent luck, and it is not a small oversight even a top player could do sometimes, it's going for Philosopher's Stone when there's a fast big money strategy with support in the kingdom.
This is totally wrong. This same thing happens all the time with human players. I win games despite major strategic misplays, and I try silly strategies sometimes that look like something stupid a bot would do. I play against humans, even top players, who choose nonsensical strategies. You can watch streams of top players playing really stupid mirrors against each other, it's not that uncommon.

Certainly if you were playing only humans of iso level 5-15 you would be playing against a lot of stupid strategies. It would still take a decent amount of skill to maintain a high ranking in that pool of players.

You are ignoring the fact that the goal isn't to play "well" by some measure where Awaclus looks at your games and decides whether you chose a good strategy, it's just to win the game. So you are allowed wiggle room to try stupid stuff if your opponent is playing something dumb, you just need to show enough skill to beat whatever they are doing. And this is how it works at every level of play! High or low.
Logged

ThaddeusB

  • Young Witch
  • ****
  • Offline Offline
  • Posts: 133
  • Respect: +140
    • View Profile
Re: Bot "Pro" Games Effect Leaderboard
« Reply #28 on: September 24, 2014, 09:51:24 am »
0

This is just selection bias: you won't notice the hundreds of guys who play a bunch against bots and achieve nothing spectacular, but you will notice the one guy who runs extremely hot. I predict he'll be nowhere near his current level after 110 more games.

Trusting the description of his play in this thread is accurate and taking into account the very low prior, it's extremely unlikely he's actually very good. Moreover, I've actually proofread the 3rd Rating Analysis article many months ago, and one of its conclusions was that in the long run it's impossible to maintain your rating when playing much lower ranked opposition. This happens because both Goko and to a lesser extent Iso underappreciate the amount of randomness in Dominion and thus give disproportional weights to upsets.

I see there's still interest in this matter; AI is extremely busy right now, but I'll ask him if he's willing to post it when he has a little more time (I don't think he's reading this thread).

This. It's not at all unlikely that someone with a true expected win rate of 75-80% over ISO 10 players could hit 90% over a 125 sample. I also think the second part is true - almost always when someone seems too  highly rayed to me it turns out they play almost exclusively against much higher ranked opponents, getting too high of a boost from the occasional upset.
Logged

Awaclus

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 11809
  • Shuffle iT Username: Awaclus
  • (´。• ω •。`)
  • Respect: +12848
    • View Profile
    • Birds of Necama
Re: Bot "Pro" Games Effect Leaderboard
« Reply #29 on: September 24, 2014, 10:10:26 am »
0

Not against real players, but against bots, you can misplay and win inspite of it (or even because of it). Take a look at the game I linked, it's one of the worst but not the only one. Seriously, an opponent buying nothing but Silvers, Golds and green cards could beat him in that game with some decent luck, and it is not a small oversight even a top player could do sometimes, it's going for Philosopher's Stone when there's a fast big money strategy with support in the kingdom.
This is totally wrong. This same thing happens all the time with human players. I win games despite major strategic misplays, and I try silly strategies sometimes that look like something stupid a bot would do. I play against humans, even top players, who choose nonsensical strategies. You can watch streams of top players playing really stupid mirrors against each other, it's not that uncommon.

Certainly if you were playing only humans of iso level 5-15 you would be playing against a lot of stupid strategies. It would still take a decent amount of skill to maintain a high ranking in that pool of players.

You are ignoring the fact that the goal isn't to play "well" by some measure where Awaclus looks at your games and decides whether you chose a good strategy, it's just to win the game. So you are allowed wiggle room to try stupid stuff if your opponent is playing something dumb, you just need to show enough skill to beat whatever they are doing. And this is how it works at every level of play! High or low.
It's not the same thing. People play bad strategies too, but you can't predict what low level players do, while you can oftentimes predict what a bot is going to do. In order to beat random L5-15 strangers 90% of the time, you need to be very good at playing Dominion. In order to beat bots, you need to be very good at playing against the bots.
Logged
Bomb, Cannon, and many of the Gunpowder cards can strongly effect gameplay, particularly in a destructive way

The YouTube channel where I make musicDownload my band's Creative Commons albums for free

Mic Qsenoch

  • 2015 DS Champion
  • *
  • Offline Offline
  • Posts: 1709
  • Respect: +4329
    • View Profile
Re: Bot "Pro" Games Effect Leaderboard
« Reply #30 on: September 24, 2014, 10:27:17 am »
+2

Not against real players, but against bots, you can misplay and win inspite of it (or even because of it). Take a look at the game I linked, it's one of the worst but not the only one. Seriously, an opponent buying nothing but Silvers, Golds and green cards could beat him in that game with some decent luck, and it is not a small oversight even a top player could do sometimes, it's going for Philosopher's Stone when there's a fast big money strategy with support in the kingdom.
This is totally wrong. This same thing happens all the time with human players. I win games despite major strategic misplays, and I try silly strategies sometimes that look like something stupid a bot would do. I play against humans, even top players, who choose nonsensical strategies. You can watch streams of top players playing really stupid mirrors against each other, it's not that uncommon.

Certainly if you were playing only humans of iso level 5-15 you would be playing against a lot of stupid strategies. It would still take a decent amount of skill to maintain a high ranking in that pool of players.

You are ignoring the fact that the goal isn't to play "well" by some measure where Awaclus looks at your games and decides whether you chose a good strategy, it's just to win the game. So you are allowed wiggle room to try stupid stuff if your opponent is playing something dumb, you just need to show enough skill to beat whatever they are doing. And this is how it works at every level of play! High or low.
It's not the same thing. People play bad strategies too, but you can't predict what low level players do, while you can oftentimes predict what a bot is going to do. In order to beat random L5-15 strangers 90% of the time, you need to be very good at playing Dominion. In order to beat bots, you need to be very good at playing against the bots.

I don't think predictability is all that valuable in upping your winrate. I mean probably the most glaring issue with the bots is easier wins on boards with Potion cards. And presumably, any harm to the bots from being predictable is reflected in them having lower rankings and thus wins against them are less valuable (I understand that there's asymmetry here if a player is only playing bots, I just don't believe it's a huge deal, I could be wrong). I have played hundreds (thousands?) of bots games and I see a lot of variety in the strategies they pursue, even if they lack finesse. I have been downright shocked by some of the stuff they've cobbled together.
« Last Edit: September 24, 2014, 10:30:24 am by Mic Qsenoch »
Logged

Holger

  • Minion
  • *****
  • Offline Offline
  • Posts: 736
  • Respect: +458
    • View Profile
Re: Bot "Pro" Games Effect Leaderboard
« Reply #31 on: September 26, 2014, 12:42:20 pm »
0

Does the article give a hint why CPA is ranked so much lower at Goko in spite of them substracting only 2σ, and CPA playing dozens of games per day (so Goko's daily uncertainty increase can only help him)?

I suppose I can answer this myself: Goko's rating still contains lots of overrated people due to the recent "Adventures count as Pro games" bug.

This. It's not at all unlikely that someone with a true expected win rate of 75-80% over ISO 10 players could hit 90% over a 125 sample. I also think the second part is true - almost always when someone seems too  highly rayed to me it turns out they play almost exclusively against much higher ranked opponents, getting too high of a boost from the occasional upset.

I've done the math with a true expected win rate of 80% and the current 145 bot games CPA has now played: The probability for a coincidental 90% win rate in this case is about 0.3% - it's very close to the upper 3 sigma level. That's not at all likely. 100+ games are quite a representative sample.

(And even an 80% win rate against bots would still be quite good IMO - I'm not sure I've reached that rate against bots without choosing the kingdom.)
Logged

Ratsia

  • Moneylender
  • ****
  • Offline Offline
  • Posts: 168
  • Respect: +113
    • View Profile
Re: Bot "Pro" Games Effect Leaderboard
« Reply #32 on: September 27, 2014, 12:07:34 pm »
+2

This is just selection bias: you won't notice the hundreds of guys who play a bunch against bots and achieve nothing spectacular, but you will notice the one guy who runs extremely hot.
Indeed.

I play almost exclusively against bots on Goko, for the simple reason that I mostly play in situations where I cannot play a full game in a row but can guarantee that I (usually) manage to do my next turn before the time out. In other words, I could not play against humans but bots don't mind my slow-playing.

After quite a bit of games, my isotropish rating is at 28 and has been fluctuating between roughly 25 and 30 all the time. When Isotropic was still around, my rating hovered between 27 and 31. The final rating was 29, after roughly as many games that I have now played on Goko.

Even though I'm just another isolated example, for me the rating system seems to work very accurately despite playing only against the bots, giving me a rating that is practically indistinguishable from what I had on Isotropic playing only against humans (and in general the two ratings seem to be quite comparable). I guess I could improve my rating on Goko a bit by trying to game the bots, but I have zero interested in doing that.


Edit: ...and I would hate to have games against bots removed from the ranking. If I have an opportunity to play against humans it's way better that my rating is roughly at the right level so that I can get right opponents via automatch.
« Last Edit: September 27, 2014, 12:10:52 pm by Ratsia »
Logged

flies

  • Minion
  • *****
  • Offline Offline
  • Posts: 629
  • Shuffle iT Username: flies
  • Statistical mechanics of hard rods on a 1D lattice
  • Respect: +348
    • View Profile
    • ask the atheists
Re: Bot "Pro" Games Effect Leaderboard
« Reply #33 on: September 30, 2014, 04:55:45 pm »
0

i play against bots when I don't really want to pay attention to what I'm doing or when I want to experiment with long-shot strategies.  I do not want "play bots" games included in the ranking.  I do not want "play bots" counted on the Pro leaderboard at all.  I suppose it makes some sense to have the option to play bots on the pro leaderboard, but for now, if I want to play casual games against bots it is veryrelatively laborious (must do it thru the multiplayer interface).
« Last Edit: September 30, 2014, 08:33:07 pm by flies »
Logged
Gotta be efficient when most of your hand coordination is spent trying to apply mascara to your beard.
flies Dominionates on youtube

Ratsia

  • Moneylender
  • ****
  • Offline Offline
  • Posts: 168
  • Respect: +113
    • View Profile
Re: Bot "Pro" Games Effect Leaderboard
« Reply #34 on: October 01, 2014, 02:52:12 am »
0

i I suppose it makes some sense to have the option to play bots on the pro leaderboard, but for now, if I want to play casual games against bots it is veryrelatively laborious (must do it thru the multiplayer interface).
It indeed should be possible to do both, with equal easy.

For a long time it was laborious to play with the pro ranking, using the same multiplayer interface trick. in fact, I had not even realized I don't anymore need to do this but can use the Play bots -link instead.

Edit: Apparently one still needs to use the multiplayer interface. I just tried with Play bots and it put me against Serf bot, which is not really a bot at all. The other bots are not great either, but at least they attempt to play and hence a match against them can be justified as reasonable use of time. Playing against Serf bot does not qualify for that.
« Last Edit: October 01, 2014, 03:00:47 am by Ratsia »
Logged

Holger

  • Minion
  • *****
  • Offline Offline
  • Posts: 736
  • Respect: +458
    • View Profile
Re: Bot "Pro" Games Effect Leaderboard
« Reply #35 on: October 01, 2014, 06:25:11 am »
0

i I suppose it makes some sense to have the option to play bots on the pro leaderboard, but for now, if I want to play casual games against bots it is veryrelatively laborious (must do it thru the multiplayer interface).
It indeed should be possible to do both, with equal easy.

Yes, it should be (just ask for the rating in addition to the player number in the pop-up window). But for some reason MF seem to prefer bot games only in Pro, because they consider it "fairer". It was (is) indeed easy to rise to the top of the Casual leaderboard by choosing a kingdom which bots play badly (e.g. a Rebuild board); but at the very least they should give a simple way to play Unrated bot games.

Quote
For a long time it was laborious to play with the pro ranking, using the same multiplayer interface trick. in fact, I had not even realized I don't anymore need to do this but can use the Play bots -link instead.

Edit: Apparently one still needs to use the multiplayer interface. I just tried with Play bots and it put me against Serf bot, which is not really a bot at all. The other bots are not great either, but at least they attempt to play and hence a match against them can be justified as reasonable use of time. Playing against Serf bot does not qualify for that.

I've also been using the Play Bots button lately and have never been matched against Serf Bot. I think they automatically match you against the bot who's closest to your rating, so you would only play Serf Bot if you have a Goko rating <2000 or something like that. Otherwise I'd consider this a bug...
Logged

Ratsia

  • Moneylender
  • ****
  • Offline Offline
  • Posts: 168
  • Respect: +113
    • View Profile
Re: Bot "Pro" Games Effect Leaderboard
« Reply #36 on: October 01, 2014, 08:48:48 am »
0

I've also been using the Play Bots button lately and have never been matched against Serf Bot. I think they automatically match you against the bot who's closest to your rating, so you would only play Serf Bot if you have a Goko rating <2000 or something like that. Otherwise I'd consider this a bug...
I tried twice, both time getting Serf bot. My Pro rating is around 4700, clearly above all bots, so apparently it is not choosing the best match at least according to that rating. It could use the Casual rating for making the choice, despite the result influencing the Pro rating; it wouldn't make sense, but this sounds like a rather easy mistake to make. My Casual rating is as low as possible, since I haven't played any games in that mode in ages.
Logged

Holger

  • Minion
  • *****
  • Offline Offline
  • Posts: 736
  • Respect: +458
    • View Profile
Re: Bot "Pro" Games Effect Leaderboard
« Reply #37 on: October 01, 2014, 11:02:45 am »
0

I've also been using the Play Bots button lately and have never been matched against Serf Bot. I think they automatically match you against the bot who's closest to your rating, so you would only play Serf Bot if you have a Goko rating <2000 or something like that. Otherwise I'd consider this a bug...
I tried twice, both time getting Serf bot. My Pro rating is around 4700, clearly above all bots, so apparently it is not choosing the best match at least according to that rating. It could use the Casual rating for making the choice, despite the result influencing the Pro rating; it wouldn't make sense, but this sounds like a rather easy mistake to make. My Casual rating is as low as possible, since I haven't played any games in that mode in ages.

That's strange. Your explanation might be true; I have a Casual rating of about 2000, which would be enough to not get Serf Bot, at least if the bots are also judged by their Casual ratings. Since MF already mistakenly allows the Kingdom Generator to be used with that button, maybe they also forgot to switch the "bot matchmaking" from Casual to Pro rating.

Another stupid thing that they should change ASAP is timing out in bot games - it's very annoying to lose a bot game just because you're doing something else for 5 or 10 minutes.
Logged

flies

  • Minion
  • *****
  • Offline Offline
  • Posts: 629
  • Shuffle iT Username: flies
  • Statistical mechanics of hard rods on a 1D lattice
  • Respect: +348
    • View Profile
    • ask the atheists
Re: Bot "Pro" Games Effect Leaderboard
« Reply #38 on: October 01, 2014, 03:26:32 pm »
0

plus having your quit % affected when playing against bots is wrong.

the simplest solution, suggested above, is to have a rating-mode switch when hitting 'play bots'.
Logged
Gotta be efficient when most of your hand coordination is spent trying to apply mascara to your beard.
flies Dominionates on youtube

jaybeez

  • Explorer
  • *****
  • Offline Offline
  • Posts: 335
  • Shuffle iT Username: jaybeez
  • Respect: +395
    • View Profile
Re: Bot "Pro" Games Effect Leaderboard
« Reply #39 on: February 27, 2015, 01:01:13 pm »
0

Sorry for the thread necro, but I was unaware of this until I used the Play Bots option last night for the first time probably a year or more.  I just wanted to try out a couple of things, actually lost one of the games, and quit another, and noticed my Pro rating drop by a bunch.

Can I suggest some kind of pop-up that warns people of this when clicking the Play Bots button?  Just a little dialog window like, "Warning: games played using Play Bots are counted as Pro games!"  Is that even possible with the extension?
« Last Edit: February 27, 2015, 01:53:05 pm by jaybeez »
Logged

SCSN

  • Mountebank
  • *****
  • Offline Offline
  • Posts: 2227
  • Respect: +7140
    • View Profile
Re: Bot "Pro" Games Effect Leaderboard
« Reply #40 on: February 27, 2015, 01:11:16 pm »
+12

Sorry for the thread necro

One of the many nice things about Goko is that you don't have to worry about things like dates and stuff, for if something was an issue two years ago it's almost certainly still an issue today!
Logged

WanderingWinder

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5275
  • ...doesn't really matter to me
  • Respect: +4381
    • View Profile
    • WanderingWinder YouTube Page
Re: Bot "Pro" Games Effect Leaderboard
« Reply #41 on: February 27, 2015, 01:42:21 pm »
0

To be fair, this is also a thing on isotropish.

-Stef-

  • 2012 & 2016 DS Champion
  • *
  • Offline Offline
  • Posts: 1574
  • Respect: +4419
    • View Profile
Re: Bot "Pro" Games Effect Leaderboard
« Reply #42 on: February 27, 2015, 01:45:34 pm »
+1

To be fair, this is also a thing on isotropish.

Yes but it's only an Isotopish thing because Andrew is under the impression that we (the f.ds community) think it's better this way.
I certainly don't.
Logged
Join the Dominion League!

jaybeez

  • Explorer
  • *****
  • Offline Offline
  • Posts: 335
  • Shuffle iT Username: jaybeez
  • Respect: +395
    • View Profile
Re: Bot "Pro" Games Effect Leaderboard
« Reply #43 on: February 27, 2015, 01:52:44 pm »
0

To be fair, this is also a thing on isotropish.
Yes, that's precisely what the problem is.
Logged

WanderingWinder

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5275
  • ...doesn't really matter to me
  • Respect: +4381
    • View Profile
    • WanderingWinder YouTube Page
Re: Bot "Pro" Games Effect Leaderboard
« Reply #44 on: February 27, 2015, 02:22:08 pm »
+1

To be fair, this is also a thing on isotropish.

Yes but it's only an Isotopish thing because Andrew is under the impression that we (the f.ds community) think it's better this way.
I certainly don't.

Yes, and it's only a Goko thing because they were under the impression that their customer base would like it better that way. They may or may not be wrong (I, like you, think they shouldn't be included), but on this issue at least, there's not much difference. The real difference is that this is one of few issues for Salvager, and many for Goko.

qmech

  • Torturer
  • *****
  • Offline Offline
  • Posts: 1918
  • Shuffle iT Username: qmech
  • What year is it?
  • Respect: +2320
    • View Profile
Re: Bot "Pro" Games Effect Leaderboard
« Reply #45 on: March 01, 2015, 01:30:12 pm »
0

I've played quite a lot of bots recently to see what effect it has on my rating, and the answer is that it has certainly increased. 

Unless you think the Bots isotropish ratings are majorly out of whack, this shouldn't matter.

If you believe in the rating system, then I think you have to take this line.  But it's easy to imagine reasons why the bots' ratings might be inflated: for example, they're likely to win a reasonable number of games against opponents who hit the Play Bots button then quit when they run out of time to play.

Whatever the truth there, I've found playing the bots a much easier way to gain rating than playing human opposition, for two related reasons.  The games are usually easier, so if you're in a reasonable position then you can play much more quickly.  And that means it's easy to play a lot of games in a short time span: I might have gone up by the same amount if I'd played the same number of games against high level humans, but it would have taken a lot longer and required a lot more mental effort.

I can also back up MicQ's observation in whichever thread that the Goko ratings do not respond as positively to repeated crushing of the bots.
Logged

polot38

  • Baron
  • ****
  • Offline Offline
  • Posts: 52
  • Respect: +35
    • View Profile
Re: Bot "Pro" Games Effect Leaderboard
« Reply #46 on: March 01, 2015, 06:07:55 pm »
0

I don't really understand the opposition to having bots removed from the rankings. I mean, either:
a) bots are easy to beat if you know how to play against specifically bots (or they are somehow systematically underrated), in which case bots should clearly be removed, or
b) bots are roughly equal in skill to players around their level, in which case you don't really lose much by kicking them out anyways.

In either case, you still have the issue that some people would like to play bot-games when they don't really like playing serious dominion, and you could also make the case that, even with a 100% win-rate, you shouldn't be able to get to one of the highest ranking in dominion by only playing lower level players.
Logged

GeoLib

  • Jester
  • *****
  • Offline Offline
  • Posts: 965
  • Respect: +1265
    • View Profile
Re: Bot "Pro" Games Effect Leaderboard
« Reply #47 on: March 01, 2015, 06:42:50 pm »
+1

The problem with removing bots is that they give people a fast way to practice and prove their skill so they don't have to jump into multiplayer at the bottom. When so many games have rating caps it's kind of nice to be able to quickly climb the ladder (because bot games are quick).
Logged
"All advice is awful"
 —Count Grishnakh

polot38

  • Baron
  • ****
  • Offline Offline
  • Posts: 52
  • Respect: +35
    • View Profile
Re: Bot "Pro" Games Effect Leaderboard
« Reply #48 on: March 01, 2015, 10:34:42 pm »
0

The problem with removing bots is that they give people a fast way to practice and prove their skill so they don't have to jump into multiplayer at the bottom. When so many games have rating caps it's kind of nice to be able to quickly climb the ladder (because bot games are quick).

Is that really a large issue though? I mean, if your skill really is above most rating caps, you should be able to quickly jump up in ratings by playing normal players anyways. And it is only likely to affect very new players, a lot of whom aren't skilled enough to start with to jump into games with rating caps.
Logged

Throwaway_bicycling

  • Moneylender
  • ****
  • Offline Offline
  • Posts: 151
  • Respect: +140
    • View Profile
Re: Bot "Pro" Games Effect Leaderboard
« Reply #49 on: March 01, 2015, 10:51:48 pm »
0

I've played quite a lot of bots recently to see what effect it has on my rating, and the answer is that it has certainly increased. 
Skipping the boring background, I can now confess that I never actually made a conscious decision to play Pro games on Dominion Online, but started to do so when I found out that I was actually on the Isotropish leaderboard because bot games had started to count as Pro. In any case, I had a ranking of about 200 and Isotropish thought I was about Level 30 at the end of October, 2014. That gave me (over)confidence, and I have played about 300 real games after that...and at first watched my rank and rating sink like a stone. In my case, I would estimate the "Bot Effect" as being 10-12 Isotropish levels.

So that *sounds* like a serious distortion, but I am actually not sure the problem is completely with the Bots' ratings in the broad sense (they have all played about 250K games apiece in two-player, against all levels of opponents) as it is that when you are playing Bots, you are playing only one of 6 opponents, and opponents who do not learn or change, rather than the dozens or hundreds of different opponents you could in real life. I don't know for sure (it's an empirical question, though), but my guess is that stronger players would have some "ratings inflation" generally against hapless weaker opponents they played relatively often. In general, there may be meaningful effects of the diversity of opponents you play. My intuition is that if you had two players equal in mu and sigma and games played, the one who had played a significantly larger number of opponents would be the favorite in a match played between them.

Logged
Pages: 1 [2] 3  All
 

Page created in 0.062 seconds with 21 queries.