Dominion Strategy Forum

Please login or register.

Login with username, password and session length
Pages: [1]

Author Topic: bug report about matchmaking  (Read 7031 times)

0 Members and 1 Guest are viewing this topic.

-Stef-

  • 2012 & 2016 DS Champion
  • *
  • Offline Offline
  • Posts: 1574
  • Respect: +4419
    • View Profile
bug report about matchmaking
« on: October 28, 2015, 10:54:03 am »
+7

I just created a bug report about the matchmaking on the MF forums.

Please help me put this bug on their radar. Maybe you can like it, thank it, or even better: reply to it and describe in your own words this is important indeed.
Logged
Join the Dominion League!

-Stef-

  • 2012 & 2016 DS Champion
  • *
  • Offline Offline
  • Posts: 1574
  • Respect: +4419
    • View Profile
Re: bug report about matchmaking
« Reply #1 on: October 28, 2015, 12:16:11 pm »
+8

Apparently Adam posted the very same thing in their feedback and suggestions thread.
Had I known about that, I would probably not have made this post but simply replied to that one.

But let me be clear on this. It is not a suggestion for an improvement. Their current system is simply bugged. It does not work as they intended it to work. I don't want an improvement, I want a bugfix. The only reason nobody dared to 100% claim it's a bug is because you can't be sure after playing only a couple of games.

Also, Adam is going out of his way to collect data, and asks people not to respond in that thread. I don't agree with that but I will at least respect his wish not to reply over there so I guess I'll respond here:

Collecting data about this bug is a bad plan and a total waste of time. If a developer at MF is actually interested in this bug, it's super easy to trace it in their debugger and find the bug in a very short amount of time. You only need 1 example of a person getting matched with someone >1000 points difference to know it's bugged, and that happens more often than not. I'm not claiming collecting data is always a bad plan. Some bugs - "heisenbugs" - are really hard to track, replicate and/or debug, and than feedback from a user describing the bug may be very helpful. Especially if multi-threading and race conditions are involved, scenarios might pop up the developer never considered. This however is a simple bug contained in a single function, namely the one that matches players with other players.

The only thing we have to do is make them actually look at it.
Logged
Join the Dominion League!

Donald X.

  • Dominion Designer
  • *****
  • Offline Offline
  • Posts: 6357
  • Respect: +25672
    • View Profile
Re: bug report about matchmaking
« Reply #2 on: October 28, 2015, 01:01:22 pm »
+2

Collecting data about this bug is a bad plan and a total waste of time. If a developer at MF is actually interested in this bug, it's super easy to trace it in their debugger and find the bug in a very short amount of time. You only need 1 example of a person getting matched with someone >1000 points difference to know it's bugged, and that happens more often than not. I'm not claiming collecting data is always a bad plan. Some bugs - "heisenbugs" - are really hard to track, replicate and/or debug, and than feedback from a user describing the bug may be very helpful. Especially if multi-threading and race conditions are involved, scenarios might pop up the developer never considered. This however is a simple bug contained in a single function, namely the one that matches players with other players.
Well we don't even have a single example. Yes, I was so eager to be friendly while defending off-topic posts everywhere that I didn't think about it, but really, one log and you've got something to point to.
Logged

Kirian

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 7096
  • Shuffle iT Username: Kirian
  • An Unbalanced Equation
  • Respect: +9411
    • View Profile
Re: bug report about matchmaking
« Reply #3 on: October 28, 2015, 01:29:32 pm »
+1

Collecting data about this bug is a bad plan and a total waste of time. If a developer at MF is actually interested in this bug, it's super easy to trace it in their debugger and find the bug in a very short amount of time. You only need 1 example of a person getting matched with someone >1000 points difference to know it's bugged, and that happens more often than not. I'm not claiming collecting data is always a bad plan. Some bugs - "heisenbugs" - are really hard to track, replicate and/or debug, and than feedback from a user describing the bug may be very helpful. Especially if multi-threading and race conditions are involved, scenarios might pop up the developer never considered. This however is a simple bug contained in a single function, namely the one that matches players with other players.
Well we don't even have a single example. Yes, I was so eager to be friendly while defending off-topic posts everywhere that I didn't think about it, but really, one log and you've got something to point to.


I can agree with this.  I mean, it's not exactly an opinion to agree with or anything, but it would be great.

OK, so here:  http://dominion-game-logs.s3.amazonaws.com/game_logs/20151028/log.0.1445999669685.txt

I can't force SCSN into the thread, so I merely hope he looks here.  SCSN currently has a MF rating of about 5400; Wrathfull has a low Isotropish rank and I can't find him anywhere in the MF ratings list above 4000.  SCSN, can I assume you were play on what I expect were your normal settings (within 1000, pro game)?  If so, here's a single example for a bug.
Logged
Kirian's Law of f.DS jokes:  Any sufficiently unexplained joke is indistinguishable from serious conversation.

-Stef-

  • 2012 & 2016 DS Champion
  • *
  • Offline Offline
  • Posts: 1574
  • Respect: +4419
    • View Profile
Re: bug report about matchmaking
« Reply #4 on: October 28, 2015, 01:31:49 pm »
+1

Well we don't even have a single example. Yes, I was so eager to be friendly while defending off-topic posts everywhere that I didn't think about it, but really, one log and you've got something to point to.
There is an excellent log search functionality that allows you to retrieve logs. I think of the games I played >90% are an example of when the matchmaking failed.

But in case you really want a specific one: here is a log.
This turn makes kind of clear this person is not within 1000 rating points...

---------- Stef: turn 33 ----------
Stef - plays Swindler
Feu - reveals Gold
Feu - trashes Gold
Feu - reveals reaction Trader
Feu - gains Silver

(it's not in the log what I tried to replace the gold with, but it was another gold.)

... but to be absolutely sure I also just scrolled down the MF leaderboard to my rating minus 1000, and did not find this username.
Logged
Join the Dominion League!

GendoIkari

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 9701
  • Respect: +10741
    • View Profile
Re: bug report about matchmaking
« Reply #5 on: October 28, 2015, 01:38:31 pm »
0

Well we don't even have a single example. Yes, I was so eager to be friendly while defending off-topic posts everywhere that I didn't think about it, but really, one log and you've got something to point to.
There is an excellent log search functionality that allows you to retrieve logs. I think of the games I played >90% are an example of when the matchmaking failed.

But in case you really want a specific one: here is a log.
This turn makes kind of clear this person is not within 1000 rating points...

---------- Stef: turn 33 ----------
Stef - plays Swindler
Feu - reveals Gold
Feu - trashes Gold
Feu - reveals reaction Trader
Feu - gains Silver

(it's not in the log what I tried to replace the gold with, but it was another gold.)

... but to be absolutely sure I also just scrolled down the MF leaderboard to my rating minus 1000, and did not find this username.

Couldn't this sort of thing be a simple misclick?
Logged
Check out my F.DS extension for Chrome! Card links; Dominion icons, and maybe more! http://forum.dominionstrategy.com/index.php?topic=13363.0

Thread for Firefox version:
http://forum.dominionstrategy.com/index.php?topic=16305.0

-Stef-

  • 2012 & 2016 DS Champion
  • *
  • Offline Offline
  • Posts: 1574
  • Respect: +4419
    • View Profile
Re: bug report about matchmaking
« Reply #6 on: October 28, 2015, 01:53:42 pm »
+4

Couldn't this sort of thing be a simple misclick?
Of course it could. In this particular case that isn't even unreasonable to think. But that still doesn't put him in range of 1000 below my rating.

Today I played a game on a board with Cultist, City, Black Market and Altar. I was a bit in doubt whether to keep it simple with money+cultist or build the engine.
My opponent certainly decided to keep it simple, as he skipped all kingdom cards! Now that first Gold may have been a misclick, but those early Duchies over Cultists...

Occam's Razor is a nice theory that tells you to believe in the most simple explanation using the least assumptions. And here by far the most simple explanation is a bug in the MF matchmaking code. It's not that 90% of my opponents make misclicks.
Logged
Join the Dominion League!

JW

  • Jester
  • *****
  • Offline Offline
  • Posts: 969
  • Shuffle iT Username: JW
  • Respect: +1781
    • View Profile
Re: bug report about matchmaking
« Reply #7 on: October 28, 2015, 01:56:06 pm »
+2

For most people it would be very difficult to confirm that someone isn't within 1000 points of your rating on the leaderboard because you would need to scroll through hundreds of positions on the leaderboard. Since Stef is nearly at the top of the leaderboard, last I checked there were fewer than 100 people with ratings within 1000 points of his.

The chance that someone could make it to the top 100 of the leaderboard while playing like this also seems remote:

Today I played a game on a board with Cultist, City, Black Market and Altar. I was a bit in doubt whether to keep it simple with money+cultist or build the engine.
My opponent certainly decided to keep it simple, as he skipped all kingdom cards! Now that first Gold may have been a misclick, but those early Duchies over Cultists...
Logged

mumposaur

  • Steward
  • ***
  • Offline Offline
  • Posts: 29
  • Respect: +19
    • View Profile
Re: bug report about matchmaking
« Reply #8 on: October 29, 2015, 01:15:13 pm »
+6

Well we don't even have a single example. Yes, I was so eager to be friendly while defending off-topic posts everywhere that I didn't think about it, but really, one log and you've got something to point to.
There is an excellent log search functionality that allows you to retrieve logs. I think of the games I played >90% are an example of when the matchmaking failed.

But in case you really want a specific one: here is a log.
This turn makes kind of clear this person is not within 1000 rating points...

---------- Stef: turn 33 ----------
Stef - plays Swindler
Feu - reveals Gold
Feu - trashes Gold
Feu - reveals reaction Trader
Feu - gains Silver

(it's not in the log what I tried to replace the gold with, but it was another gold.)

... but to be absolutely sure I also just scrolled down the MF leaderboard to my rating minus 1000, and did not find this username.

I found this player on the MF leaderboard, they are currently 3299. I kind of doubt they were within 1000 of Stef's rating (so over ~5000) a day ago or whenever this game took place. And yes, I should find better ways to spend my time than clicking a down arrow 100 times.

Also this demonstrates how far off the matchmaking is from working as advertised.
« Last Edit: October 29, 2015, 01:20:08 pm by mumposaur »
Logged

Kirian

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 7096
  • Shuffle iT Username: Kirian
  • An Unbalanced Equation
  • Respect: +9411
    • View Profile
Re: bug report about matchmaking
« Reply #9 on: October 29, 2015, 05:21:26 pm »
+3

And yes, I should find better ways to spend my time than clicking a down arrow 100 times.

Also this demonstrates how far off the matchmaking is from working as advertised.

It also demonstrates how bad their leaderboard is.
Logged
Kirian's Law of f.DS jokes:  Any sufficiently unexplained joke is indistinguishable from serious conversation.

pst

  • Minion
  • *****
  • Offline Offline
  • Posts: 584
  • Respect: +906
    • View Profile
Re: bug report about matchmaking
« Reply #10 on: October 30, 2015, 09:05:29 am »
+2

I find it curious that Stef so often is matched against players who seem to be so much worse than players I normally get matched up with there. It wouldn't surprise me that much if it turns out there is some "going around the corner" bug where the very best are seen to be close to the very worst.
Logged

Voltaire

  • Jester
  • *****
  • Offline Offline
  • Posts: 957
  • flavor text
  • Respect: +1097
    • View Profile
Re: bug report about matchmaking
« Reply #11 on: October 30, 2015, 10:34:20 am »
+4

I was all ready to post another bugged log after an opponent opened Scout against me yesterday, but then I lost so...
Logged

Deadlock39

  • Torturer
  • *****
  • Offline Offline
  • Posts: 1722
  • Respect: +1757
    • View Profile
Re: bug report about matchmaking
« Reply #12 on: October 30, 2015, 10:46:58 am »
0

I find it curious that Stef so often is matched against players who seem to be so much worse than players I normally get matched up with there. It wouldn't surprise me that much if it turns out there is some "going around the corner" bug where the very best are seen to be close to the very worst.

It feels a bit like that to me as well, but it is hard to say. If this is really something that is happening, my first guess would be that if there isn't anyone in the automatch pool within 1000, it is just giving up and matching you against the first person it finds.

pst

  • Minion
  • *****
  • Offline Offline
  • Posts: 584
  • Respect: +906
    • View Profile
Re: bug report about matchmaking
« Reply #13 on: October 30, 2015, 10:50:07 am »
0

It feels a bit like that to me as well, but it is hard to say. If this is really something that is happening, my first guess would be that if there isn't anyone in the automatch pool within 1000, it is just giving up and matching you against the first person it finds.

Perhaps someone else who it also was hard to find a suitable match for! (though for opposite reasons)
Logged

sc0UT

  • Young Witch
  • ****
  • Offline Offline
  • Posts: 133
  • Shuffle iT Username: sc0UT
  • Respect: +106
    • View Profile
Re: bug report about matchmaking
« Reply #14 on: October 30, 2015, 11:08:20 am »
+1

I find it curious that Stef so often is matched against players who seem to be so much worse than players I normally get matched up with there. It wouldn't surprise me that much if it turns out there is some "going around the corner" bug where the very best are seen to be close to the very worst.

It feels a bit like that to me as well, but it is hard to say. If this is really something that is happening, my first guess would be that if there isn't anyone in the automatch pool within 1000, it is just giving up and matching you against the first person it finds.

MF is obviously fan of bias options, they just forgot the word "prefer" in their text.

  • Prefer compatible rating if available
  • Prefer compatible rating. Never more than 1000 below my rating
  • Prefer any ranking

Problem solved. There's no bug in the matchmaking code.
Logged

-Stef-

  • 2012 & 2016 DS Champion
  • *
  • Offline Offline
  • Posts: 1574
  • Respect: +4419
    • View Profile
Re: bug report about matchmaking
« Reply #15 on: October 30, 2015, 06:18:00 pm »
+5

I just created a bug report about the matchmaking on the MF forums.

Please help me put this bug on their radar. Maybe you can like it, thank it, or even better: reply to it and describe in your own words this is important indeed.

Jeff at Making Fun says that they've fixed this bug and it will be in the next release. http://forum.makingfun.com/showthread.php?8051-seek-a-game-compatible-rating-does-not-work&p=41557&viewfull=1#post41557

Quote from: Jeff (Developer);41557
A lot of posts for a day. Thanks for the input. This bug (rating not less than 1000 isn't working correctly)  is already fixed internally and the fix will be in the next release.

I just downloaded that new release, version 2.0.44.

It displays the number of active players / seeking players / games played while seeking a game.
It also has the horrible CAPS ONLY FONT in the log replaced by a more readable one.
And when I said "not 1000 below my rating" it actually didn't match me, so that makes it quite likely they fixed that too.

Hurray! :)

Unfortunately after the first game my screen went completely black and in the next game I was stuck on turn 1, unable to buy anything.
But I guess that's the price of progress.

In all seriousness it is hopeful, assuming the black-screen-thing is a bug they will fix soon.
Logged
Join the Dominion League!

mpsprs

  • Explorer
  • *****
  • Offline Offline
  • Posts: 332
  • Respect: +169
    • View Profile
Re: bug report about matchmaking
« Reply #16 on: October 30, 2015, 06:29:23 pm »
+3

Unfortunately after the first game my screen went completely black and in the next game I was stuck on turn 1, unable to buy anything.
But I guess that's the price of progress.


That sounds like progress to me.  With this new "Black Screen for Stef" feature, I might be able to beat him!  And I like the actual improvements too.

-Stef-

  • 2012 & 2016 DS Champion
  • *
  • Offline Offline
  • Posts: 1574
  • Respect: +4419
    • View Profile
Re: bug report about matchmaking
« Reply #17 on: October 30, 2015, 06:39:23 pm »
+1

Unfortunately after the first game my screen went completely black and in the next game I was stuck on turn 1, unable to buy anything.
But I guess that's the price of progress.


That sounds like progress to me.  With this new "Black Screen for Stef" feature, I might be able to beat him!  And I like the actual improvements too.

I really like the improvements, but something is also very wrong here. I've successfully played 2 games now. My client crashed 5 times. And 3 more times I was in a game but couldn't buy/play/chat anything.
Logged
Join the Dominion League!

mpsprs

  • Explorer
  • *****
  • Offline Offline
  • Posts: 332
  • Respect: +169
    • View Profile
Re: bug report about matchmaking
« Reply #18 on: October 30, 2015, 06:43:09 pm »
0

I just tried one game now (successfully matched against someone ranked more that 1000 points above me), and had no performance issues.  But I haven't had any of the performance issues that others have had before either, so ... who knows.

blueblimp

  • Margrave
  • *****
  • Offline Offline
  • Posts: 2849
  • Respect: +1559
    • View Profile
Re: bug report about matchmaking
« Reply #19 on: October 30, 2015, 07:03:08 pm »
0

Literally my first game, I play 4 coppers on turn 1 and it's only letting me buy cards up to $2. Also chat is not working. So I think it has disc'd somehow.

Edit: Second game, same idea, except only allowing buys up to $3. :( This release seems not ready for prime time on my system (OS X).
« Last Edit: October 30, 2015, 07:05:35 pm by blueblimp »
Logged

mpsprs

  • Explorer
  • *****
  • Offline Offline
  • Posts: 332
  • Respect: +169
    • View Profile
Re: bug report about matchmaking
« Reply #20 on: October 30, 2015, 07:04:06 pm »
0

I just tried one game now (successfully matched against someone ranked more that 1000 points above me), and had no performance issues.  But I haven't had any of the performance issues that others have had before either, so ... who knows.

I spoke too soon.  Just started a game.  Played 4 copper, no button showed up for me to buy my Sea Hag (though I could click on a few things).  Tried to buy a silver.  Silver went to my deck, and then the game hung.

PPE:  Very similar to blueblimp's experience.

pst

  • Minion
  • *****
  • Offline Offline
  • Posts: 584
  • Respect: +906
    • View Profile
Re: bug report about matchmaking
« Reply #21 on: October 30, 2015, 07:08:53 pm »
0

I haven't tried myself yet, but it sounds like there are grave problems with this version.

That is really great! My positive interpretation of that is that they really care about what we "forum people" think, and that is the reason they eagerly rushed out this update that otherwise perhaps would have come next week when things were more checked. But since it fixed our main concern it seemed so important to get it out early. Thanks for seeing it that way!
Logged

Beyond Awesome

  • Global Moderator
  • *****
  • Offline Offline
  • Posts: 2941
  • Shuffle iT Username: Beyond Awesome
  • Respect: +2466
    • View Profile
Re: bug report about matchmaking
« Reply #22 on: October 30, 2015, 07:43:44 pm »
0

Just awesome.

Hey, at least they are trying
Logged

blueblimp

  • Margrave
  • *****
  • Offline Offline
  • Posts: 2849
  • Respect: +1559
    • View Profile
Re: bug report about matchmaking
« Reply #23 on: October 30, 2015, 07:48:27 pm »
0

It's a little concerning that their pre-release testing does not appear to include checking whether the game works at all. That's okay if the other client is still up and you're just offering early adopters a taste of the future, but it's less okay if it's the only available client and people have already paid money to play on it.

I'm not angry or anything, it's just something they should improve on.
Logged

popsofctown

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5477
  • Respect: +2860
    • View Profile
Re: bug report about matchmaking
« Reply #24 on: October 30, 2015, 08:58:55 pm »
0

It's a little concerning that their pre-release testing does not appear to include checking whether the game works at all. That's okay if the other client is still up and you're just offering early adopters a taste of the future, but it's less okay if it's the only available client and people have already paid money to play on it.

I'm not angry or anything, it's just something they should improve on.
One player randomly had their game worked, so maybe they just only tested it once or twice?  Bug testing is hard sometimes.
Logged
Pages: [1]
 

Page created in 0.771 seconds with 20 queries.