Overall, I'm pretty sure that Adventures is the most complex set and will remain that way.
Let's compare the two expansions.
Adventures
Tokens: Okay, this is a big one, but irl, it's easy to forget about token abilities
Reserves: It can be pretty easy to forget about these, especially irl.
Events: Adds more components/extra complexity. Also, easy to forget about irl. Though, Empires has Events, but this set introduced them which makes Adventures more complex since people are seeing them and getting used to them here.
Inheritance: Easy to forget irl that your estates are Inherited. See tokens.
Travellers: Easy to forget to upgrade, plus adds 5 cards per kingdom. Technically BM and Tournament do that as well, but it's a lot to keep track of, especially if both Travellers show up in a Kingdom.
Duration-Attacks: Potentially easy to forget an attack like Swamp Hag is in play. We've seen Enchantress, but again, introduced in this expansion.
So, yah, that's a lot of complexity, and I have not named everything.
Empires
Landmarks: I'm pretty sure these are easier than Events. You have to go in thinking more about strategy, but not an extra card to buy.
Debt: You can pay off a card later. Seems simple enough.
Split Piles: These seem the most complex thing of the expansion. You now have to be aware of an extra card underneath and think about whether it is worth digging to get to the card underneath.
Crown: Not really complex. I mentioned earlier this could have appeared in Intrigue, and it would have fit in well there. The effect is pretty straightforward and easy to grasp.
VP tokens: We've seen these in Prosperity. Most people buying this will have likely played with/own Prosperity.
So, overall, I'm pretty sure that Adventures is more complex than Empires by a pretty huge margin. Nonetheless, Empires is still a pretty complex expansion, at least for Dominion.