That would be really cool, actually.
I'm pretty sure it wouldn't be.
I think it would work better than it might seem at first glance. There aren’t THAT many Kingdom Victory cards and Treasures. Obviously terrible games can happen in there but it could be sometimes interesting.
This variant could be interesting if you first included the 'ordinary' victory/treasure/curse cards that were named by a specific kingdom card, e.g. Silver if Bureaucrat is present, Curse if Witch is present. This isn't necessary but prevents the kingdom from having cards that can't be used to the normal extent. Then, decide how to pick the remaining victory/treasure/curse cards, for example: while you have fewer than two victory and two treasure cards in the kingdom, draw from the 'ordinary' randomizers. I don't remember if there's a better word than 'ordinary' for cards like Duchy and Gold.
You might need an additional house rule for Mine, Rebuild, and a few others though ...