Dominion Strategy Forum

Please login or register.

Login with username, password and session length
Pages: 1 2 3 [4] 5 6 ... 8  All

Author Topic: Log Search Engine  (Read 37378 times)

0 Members and 1 Guest are viewing this topic.

StrongRhino

  • Witch
  • *****
  • Offline Offline
  • Posts: 468
  • Shuffle iT Username: StrongRhino
  • Respect: +246
    • View Profile
Re: Log Search Engine
« Reply #75 on: July 01, 2013, 06:13:32 pm »
+1

It would be awesome if key cards in the kingdom could be highlighted and searched for.
I just tried Searching for games where Rats were awesome, but had to open all the logs of games where they didn't get bought.

Maybe do something like putting the number of times a card was gained in brackets after it (or the number of times it was in player's decks at the end, which would be easier but less useful), and allow searching for that.
So this game would show
Fishing Village (0), Market Square (10), Oasis (2), Bridge (1), Caravan (5), Rats (10), Remodel (1), Pillage (0), Harem (4), Peddler (10)

and could be found by entering stuff like the following in the kingdom field:
"Rats 1+", "Peddler 10", or "Fishing Village 0"

Also, I'd prefer if clicking the link to the log of a game would open it in a new tab instead of the one the search is in.
Similar to this, maybe you could be able to search for what cards aren't in a kingdom (like golem with no- *name all villages* to narrow down games to find possible golem engines). Thanks.
Logged

GwinnR

  • Tactician
  • *****
  • Offline Offline
  • Posts: 417
  • Respect: +782
    • View Profile
    • German Youtube-Videos
Re: Log Search Engine
« Reply #76 on: July 04, 2013, 10:05:36 am »
+1

Hey, thanks for that!

I have a (little) wishe. It would be cool to find games without naming a player. So I could find for example all games with Rebuild to learn from it.
Logged
Nobody's perfect, but I'm only a nobody o.O

My german Youtube-Channel: http://www.youtube.com/gwinnrdominion

ragingduckd

  • Board Moderator
  • *
  • Offline Offline
  • Posts: 1059
  • Respect: +3525
    • View Profile
Re: Log Search Engine
« Reply #77 on: July 04, 2013, 10:37:25 am »
+3

Hey, thanks for that!

I have a (little) wishe. It would be cool to find games without naming a player. So I could find for example all games with Rebuild to learn from it.

Unfortunately, this is prohibitively slow, at least with how my database query code is written now. I may be able to improve it.

Do you really want all Rebuild games? How about all Rebuild games with top-25 players? Or all games between two top-100 players?
Logged
Salvager Extension | Isotropish Leaderboard | Game Data | Log Search & other toys | Salvager Bug Reports

Salvager not working for me at all today. ... Please help! I can't go back to playing without it like an animal!

serakfalcon

  • Apprentice
  • *****
  • Offline Offline
  • Posts: 278
  • Shuffle iT Username: serakfalcon
  • Respect: +229
    • View Profile
Re: Log Search Engine
« Reply #78 on: July 06, 2013, 04:45:59 pm »
0

Quote
Unfortunately, this is prohibitively slow, at least with how my database query code is written now. I may be able to improve it.


with MySQL u could use LIMIT ($pagenum * $resultsperpage,($pagenum +1)*$resultsperpage) and query per page, instead of running the entire query all at once
Logged

ragingduckd

  • Board Moderator
  • *
  • Offline Offline
  • Posts: 1059
  • Respect: +3525
    • View Profile
Re: Log Search Engine
« Reply #79 on: July 06, 2013, 04:58:52 pm »
0

Quote
Unfortunately, this is prohibitively slow, at least with how my database query code is written now. I may be able to improve it.


with MySQL u could use LIMIT ($pagenum * $resultsperpage,($pagenum +1)*$resultsperpage) and query per page, instead of running the entire query all at once

Okay, okay. Yeah. What I really meant was "I will definitely be able to improve it, but I'm out of town and can't get to it right away." ;)
Logged
Salvager Extension | Isotropish Leaderboard | Game Data | Log Search & other toys | Salvager Bug Reports

Salvager not working for me at all today. ... Please help! I can't go back to playing without it like an animal!

fprefect

  • Chancellor
  • ***
  • Offline Offline
  • Posts: 21
  • Respect: +18
    • View Profile
Re: Log Search Engine
« Reply #80 on: July 08, 2013, 08:23:44 am »
+1

Dear RagingDuckd,

Would it be possible to extract a history of all game results (no logs, but information about gamelength and maybe kingdom) into a single text file in any way (if it is not too much work). I want to have a look at the issues concerning rating systems.

regards,

fprefect
Logged

ragingduckd

  • Board Moderator
  • *
  • Offline Offline
  • Posts: 1059
  • Respect: +3525
    • View Profile
Re: Log Search Engine
« Reply #81 on: July 10, 2013, 03:15:26 am »
+1

Dear RagingDuckd,

Would it be possible to extract a history of all game results (no logs, but information about gamelength and maybe kingdom) into a single text file in any way (if it is not too much work). I want to have a look at the issues concerning rating systems.

regards,

fprefect

Yes. I can do that.

What data format would you like? Either csv or postgresql dump is easiest for me.

Which variables do you want? I have game date/time; kingdom; whether the game has colonies/shelters; player names; whether players are bots/guests; rating system; each player's vps, # of turns, and whether he quit/resigned. I don't have Goko ratings for many of the games.
Logged
Salvager Extension | Isotropish Leaderboard | Game Data | Log Search & other toys | Salvager Bug Reports

Salvager not working for me at all today. ... Please help! I can't go back to playing without it like an animal!

fprefect

  • Chancellor
  • ***
  • Offline Offline
  • Posts: 21
  • Respect: +18
    • View Profile
Re: Log Search Engine
« Reply #82 on: July 10, 2013, 07:41:33 am »
+1

Hello,

thank you for the offer. I would be most pleased with csv (one result : one line). I wouldnt mind to have all the information in you are suggesting but the most important for me would be:

date/time; player names; result; bots/guests; #turns; quit/resign; vps;

I am not interested in goko ratings much. You could upload it to somewhere and leave the link here, because I might not be the only one interested.

fprefect
Logged

WanderingWinder

  • Adventurer
  • ******
  • Offline Offline
  • Posts: 5275
  • ...doesn't really matter to me
  • Respect: +4359
    • View Profile
    • WanderingWinder YouTube Page
Re: Log Search Engine
« Reply #83 on: July 10, 2013, 07:48:23 am »
0

Hello,

thank you for the offer. I would be most pleased with csv (one result : one line). I wouldnt mind to have all the information in you are suggesting but the most important for me would be:

date/time; player names; result; bots/guests; #turns; quit/resign; vps;

I am not interested in goko ratings much. You could upload it to somewhere and leave the link here, because I might not be the only one interested.

fprefect
You aren't.

Lightning Edit: You aren't the only one interested, that is.

ragingduckd

  • Board Moderator
  • *
  • Offline Offline
  • Posts: 1059
  • Respect: +3525
    • View Profile
Re: Log Search Engine
« Reply #84 on: July 10, 2013, 10:45:27 am »
+6

Here's all the 2-player pro games since goko started distinguishing pro from casual and unrated. Let me know if you want 3+ player games or the other rating modes too.

http://www.sendspace.com/file/21v5ym

Note that "bot" is true if either player is a bot. Same for "guest."
Logged
Salvager Extension | Isotropish Leaderboard | Game Data | Log Search & other toys | Salvager Bug Reports

Salvager not working for me at all today. ... Please help! I can't go back to playing without it like an animal!

greatexpectations

  • Saboteur
  • *****
  • Offline Offline
  • Posts: 1095
  • Respect: +1060
    • View Profile
Re: Log Search Engine
« Reply #85 on: July 10, 2013, 11:19:18 am »
+3

this probably isn't the right thread for this, but i wasn't sure it was worth starting a new one. what would the difficulty be of creating an archive of the goko leaderboards? or does such a thing exist already? i'm thinking of something along the lines of this for the old iso leaderboards. it wouldn't necessarily have to include game totals or even be pulled once a day. but i think it would be interesting to track progress/ratings over time as well as having a simple way of scanning the leaderboard for names of players.
Logged
momomoto: ...I looked at the tableau and went "Mountebank? That's for jerks."
rrenaud: Jerks win.

fprefect

  • Chancellor
  • ***
  • Offline Offline
  • Posts: 21
  • Respect: +18
    • View Profile
Re: Log Search Engine
« Reply #86 on: July 11, 2013, 10:45:51 am »
0

Thanks a million times,

I will tell you when I had a close look.

fprefect
Logged

Watno

  • Margrave
  • *****
  • Offline Offline
  • Posts: 2720
  • Shuffle iT Username: Watno
  • Respect: +2932
    • View Profile
Re: Log Search Engine
« Reply #87 on: July 11, 2013, 05:37:08 pm »
+3

Not a huge deal, but I would prefer if the default timerange included all games and not just those of the last month.
Logged

ragingduckd

  • Board Moderator
  • *
  • Offline Offline
  • Posts: 1059
  • Respect: +3525
    • View Profile
Re: Log Search Engine
« Reply #88 on: July 12, 2013, 07:51:48 pm »
+5

For those of you who are interested in large piles of data, you can now access my game database directly. Install a PostgreSQL client and connect to:

host: gokosalvager.com
port: 5432
user: forum
pass: fds

This is the same server that's supporting the search engine and kingdom visualizer, so please be considerate.
« Last Edit: April 11, 2014, 11:50:30 am by ragingduckd »
Logged
Salvager Extension | Isotropish Leaderboard | Game Data | Log Search & other toys | Salvager Bug Reports

Salvager not working for me at all today. ... Please help! I can't go back to playing without it like an animal!

Watno

  • Margrave
  • *****
  • Offline Offline
  • Posts: 2720
  • Shuffle iT Username: Watno
  • Respect: +2932
    • View Profile
Re: Log Search Engine
« Reply #89 on: July 13, 2013, 07:48:12 pm »
+1

I can't fid games with both players specified anymore. The games still show up if you specify just one player.
Logged

yed

  • Minion
  • *****
  • Offline Offline
  • Posts: 618
  • Shuffle iT Username: yed
  • Respect: +570
    • View Profile
Re: Log Search Engine
« Reply #90 on: July 13, 2013, 08:02:14 pm »
+1

I keep getting this error:
500: Internal Server Error
Logged

ragingduckd

  • Board Moderator
  • *
  • Offline Offline
  • Posts: 1059
  • Respect: +3525
    • View Profile
Re: Log Search Engine
« Reply #91 on: July 13, 2013, 08:48:19 pm »
+4

Both of those errors are fixed now worse than when I started. I'm puzzled.

Fixed for real this time.
« Last Edit: July 14, 2013, 02:17:33 am by ragingduckd »
Logged
Salvager Extension | Isotropish Leaderboard | Game Data | Log Search & other toys | Salvager Bug Reports

Salvager not working for me at all today. ... Please help! I can't go back to playing without it like an animal!

ragingduckd

  • Board Moderator
  • *
  • Offline Offline
  • Posts: 1059
  • Respect: +3525
    • View Profile
Re: Log Search Engine
« Reply #92 on: July 16, 2013, 09:28:59 pm »
+10

Sorry if the log search has been down for you recently. My internet was down for most of the day. Please bear in mind that home internet access is a new and challenging technology and we all need to be patient for the next 10-20 years it'll take for the United States' government-enforced telecom monoplies to get a handle on it.

Seriously, it should be okay now. I'll switch to an offsite server if the problem keeps up and if I can find hosting that will give me fast access to the ~100GB of game data.
Logged
Salvager Extension | Isotropish Leaderboard | Game Data | Log Search & other toys | Salvager Bug Reports

Salvager not working for me at all today. ... Please help! I can't go back to playing without it like an animal!

ashersky

  • Mountebank
  • *****
  • Offline Offline
  • Posts: 2336
  • 2013/2014/2015 Mafia Mod of the Year
  • Respect: +1510
    • View Profile
Re: Log Search Engine
« Reply #93 on: July 16, 2013, 11:27:10 pm »
+4

AI, can you and DougZ have a baby that will grow up to create the Virtual Reality Live Action Dominion-torium for us all to battle in Yu-Gi-Oh style?



What I mean is, thank you for all your incredible work.  It's awesome.
Logged
f.ds Mafia Board Moderator

2013, 2014, 2015 Mafia Mod of the Year

Games (30): Mafia: 17, 21, 24, 31, 39, 43, 67, 72, 86 || Newbie: 7 || RMM: 6, 7, 10, 20, 22, 26, 32, 45 || Blitz: 4-7, X, 17-19, 22, 24 || BM: 14, 19

2015 f.ds Representative, World Forum Mafia Championships
2013, 2014 Mafia Player of the Year (Tie)

Town: 47 Games, 24 Wins
Scum: 22 Games, 13 Wins
11x MVP: M30, M83, ZM16, M25, M38, M61, M76, RMM5, RMM41, RMM46, M51

ragingduckd

  • Board Moderator
  • *
  • Offline Offline
  • Posts: 1059
  • Respect: +3525
    • View Profile
Re: Log Search Engine
« Reply #94 on: July 17, 2013, 12:37:19 am »
+4

AI, can you and DougZ have a baby that will grow up to create the Virtual Reality Live Action Dominion-torium for us all to battle in Yu-Gi-Oh style?

That's a remarkably patient feature request!
Logged
Salvager Extension | Isotropish Leaderboard | Game Data | Log Search & other toys | Salvager Bug Reports

Salvager not working for me at all today. ... Please help! I can't go back to playing without it like an animal!

ragingduckd

  • Board Moderator
  • *
  • Offline Offline
  • Posts: 1059
  • Respect: +3525
    • View Profile
Re: Log Search Engine
« Reply #95 on: July 18, 2013, 08:03:53 am »
+6

Updated.

Removed features:
- Case-sensitive username search
- Overall win-loss record

I'll reimplement the win-loss record elsewhere. It's a nuisance when I'm trying to code for reasonable search speed.

Added features:
- Open log in new tab/window
- Date range defaults to Goko's entire history
- correct Goko's "JackOfAllTrades" nonsense
- allow searching without specifying a player (it's rather slow though)
- filter by draws
- filter by resignations
- filter by cards not in kingdom
- more elegant date picker

If you bookmarked a search with your preferred search paramters, you'll need to replace it. I changed some of the parameter names and formatting.

Can anyone confirm that this is working on Chrome or Chromium? I developed in Firefox and I think I may have loused up the JS or something.
« Last Edit: July 18, 2013, 09:01:39 am by ragingduckd »
Logged
Salvager Extension | Isotropish Leaderboard | Game Data | Log Search & other toys | Salvager Bug Reports

Salvager not working for me at all today. ... Please help! I can't go back to playing without it like an animal!

florrat

  • Minion
  • *****
  • Offline Offline
  • Posts: 542
  • Shuffle iT Username: florrat
  • Respect: +748
    • View Profile
Re: Log Search Engine
« Reply #96 on: July 19, 2013, 08:02:09 pm »
+1

I tested all new features in Chrome. The features work good.

Some comments:
-When putting a non-existing card in the "In Kingdom"/"Not in Kingdom"-fields, I get a blank screen except for the message "500: Internal Server Error". Maybe give a little more subtle error message?  :)
-If I specify no player but a lot of cards in "In Kingdom", why don't you want to search it? Checking for an additional card in the kingdom should only slightly increase the search time, shouldn't it? Because you only have to check the hits for the new card.
Logged

ragingduckd

  • Board Moderator
  • *
  • Offline Offline
  • Posts: 1059
  • Respect: +3525
    • View Profile
Re: Log Search Engine
« Reply #97 on: July 19, 2013, 08:19:58 pm »
+2

I tested all new features in Chrome. The features work good.

Thanks! I'm not facile with regression testing on web apps, so this is very good to know.

Some comments:
-When putting a non-existing card in the "In Kingdom"/"Not in Kingdom"-fields, I get a blank screen except for the message "500: Internal Server Error". Maybe give a little more subtle error message?  :)

The nicely formatted error messages are things that I'm explicitly checking for. The "500" errors happen when there's something wrong with my code or a situation I didn't think of. So I can give you a clearer generic error like "AI's code farted" or a stack trace, but that's about it. Thanks for the bug report tho.

-If I specify no player but a lot of cards in "In Kingdom", why don't you want to search it? Checking for an additional card in the kingdom should only slightly increase the search time, shouldn't it? Because you only have to check the hits for the new card.

Actually each additional card is an enormous increase in search time.

My database is indexed by player names, but not by supply cards. That means that I actually have to look through every individual game and check the supply against the search parameters, which is very slow. It keeps looking until it finds N matches, so the more cards you specify, the longer it has to look. Asking for 20 games with any particular set of 4+ cards requires looking through the entire database.

I'll eventually add indexing by supply cards, so those searches will be faster, but I don't see a good way to index by cards that aren't in the supply. If someone has a good suggestion, I'll be happy to implement it, but I also suspect this isn't at the top of many people's feature request lists.
Logged
Salvager Extension | Isotropish Leaderboard | Game Data | Log Search & other toys | Salvager Bug Reports

Salvager not working for me at all today. ... Please help! I can't go back to playing without it like an animal!

florrat

  • Minion
  • *****
  • Offline Offline
  • Posts: 542
  • Shuffle iT Username: florrat
  • Respect: +748
    • View Profile
Re: Log Search Engine
« Reply #98 on: July 19, 2013, 08:30:22 pm »
0

The nicely formatted error messages are things that I'm explicitly checking for. The "500" errors happen when there's something wrong with my code or a situation I didn't think of. So I can give you a clearer generic error like "AI's code farted" or a stack trace, but that's about it. Thanks for the bug report tho.
I meant adding a nicely formatted error message for this particular error. I understand that if something happens you didn't think of cannot give a very meaningful error message to a casual user.



Thanks for the explanation about the search time. I've never programmed a search function myself, so I naively thought you would generate all results and then display the first 20. Now that I actually think about it, I understand that this is not the way things work :-X
Logged

serakfalcon

  • Apprentice
  • *****
  • Offline Offline
  • Posts: 278
  • Shuffle iT Username: serakfalcon
  • Respect: +229
    • View Profile
Re: Log Search Engine
« Reply #99 on: July 19, 2013, 09:42:32 pm »
+1

Quote
I'll eventually add indexing by supply cards, so those searches will be faster, but I don't see a good way to index by cards that aren't in the supply. If someone has a good suggestion, I'll be happy to implement it, but I also suspect this isn't at the top of many people's feature request lists.
Create a join table of the game ID and card IDs
for example, if game is 1, then the two columns will be
1,cardID(smuggler)
1,cardID(scout)
etc.
then do an inner join of the game table to the cards per game table. (inner join cards per game table on game id, add a where clause restricting the results where the card id is present) It should be faster since both columns will be indexed integers as opposed to a text search.
Its a little tricky to set up, after thinking about it a bit more but that should be the general right way to get started
« Last Edit: July 19, 2013, 09:56:11 pm by serakfalcon »
Logged
Pages: 1 2 3 [4] 5 6 ... 8  All
 

Page created in 0.136 seconds with 22 queries.