It must be more like you're picking a memory of a card they gained at the time they gained it. Similar to what Treasure Map and Ritual look at.
How does the same issue arise with Treasure Map and Ritual?
Play BoM as Treasure Map; trash it and trash a "real" Treasure Map from your hand. You get four Golds because, regardless of what one of those cards looks like
now (a BoM), it was a Treasure Map at the time you trashed it, and Treasure map is checking the
memory of what the cards were at the time you trashed them.
Inherit Village. Play Quarry; your Estates cost $0. Buy Ritual, trashing your $0 Estate. The Estate stops being yours, so it stops being an action, so it changes to costing $2. But Ritual says "per $1 it
cost" (past tense), so it's been ruled that Ritual checks the
memory of what the card looked like at the time you trashed it. You get 0VP instead of 2VP.
These examples are in contrast to how everything else works in Dominion: you check what the card looks like right now; you don't care what it looked like at the moment you trashed it or whatever. So, in the case for Ritual above, if you had instead Villa'd back and trashed the Estate with Remodel (which uses present tense), it wouldn't care that what you trashed used to cost $0; the card you trashed
now costs $2, so you can turn it into a $4.
(The fact that the rules work as described in the previous paragraph is my Dominion pet peeve. Nothing's broken yet, but it seems liable to create situations that lead to, as you called them, unresolvable tracking issues.)
EDIT:
Some trivia: M:tG has a concept of looking at the "memory" of an object; it's called "last known information." I think I read in an interview somewhere that Donald was somehow the one who proposed that idea.