The rulebook for base refers to them as placeholders and included them even for the non-kingdom cards, something that the expansions don't do. I think the idea was to have them on the table as a reminder of what was in any piles that had already run out. From my experience this is a bad idea though. It leads to confusion as people forget that they aren't supposed to gain the last card in the pile.
It wasn't too confusing. When all we had was the base set (and then with the first couple of expansions), we used the "placeholder" cards. You turn them upside down, so if the last card of a stack is gained, then you have a Dominion back staring you in the face. If you see three Dominion backs on the table (aside from players' decks of course), then the game is over. This worked for Estates and Duchies as well. Technically, the Province placeholder wasn't really needed.
Then somewhere along the line, we learned (I think I read it somewhere) that the cards could be used to randomize your kingdom. I liked that idea, so I pulled all those placeholder cards. Instead of having a card at the bottom of the deck, we just grab some card and put it face-down in the pile that is bought out. Usually, it's from the trash, but a Copper is just as good. In fact, it'll more likely be Coppers since Dark Ages makes the contents of the trash more relevant. And if we get something like a Beggar or a Trader or a Tunnel or anything else that could contribute to a stack going empty when it traditionally doesn't, then we'll use some other card. After all, what are the odds that you have a kingdom where all three base treasures have the likelihood to run out? But if it happens, then there's either Shelters or excess Estates.
I'd probably be fine using the randomize deck for the Black Market just because I generally don't look at my deck, so I wouldn't even know if the next card would be a Black Market card or not. And if I know that I have Black Market, I'll very consciously look away from my deck while shuffling.