The official rule is this:
In rare circumstances an effect may try to move a card that is not where that effect expects the card to be. In those cases the card does not move - the effect has "lost track" of the card.
So here the card is Fortress, the effect is Prince's "setting it aside again when you discard it from play", the place it expects the card to be is the play area. We can fulfill all of these conditions, so I would expect that Prince is able to set Fortress aside again.
I would argue that whatever happens in the meantime is irrelevant as long as you can find the exact card that you initially played with Prince and are able to set it aside again.
However, a lot has been written both by Donald and others about how lose-track applies to these kinds of things, so I'm not exactly sure what the ruling would be, I can only argue about what I think would make the most sense.
I once Princed a gear and accidentally set aside cards with gear. I was sad.
You know, I only noticed because of your comment that it's "up to 2", not: "exactly 2".
You learn something everyday.