It seems that every match would basically influence the rating of every player within the system
Uh, why?
In the current system, we treat games as a stream of information of the form "Player A just beat Player B", and that gets updated into the existing skill estimates. The entire stream of games so far is summarized into the (mu, phi) that each player has, or in other words, after N games, the only information we need to care about is the rating system's current (mu, phi) estimate for each player.
In your proposed system, you want game order to not matter. So suppose Player A just beat Player B. This affects the skill estimate for A and B. However, consider all the people Player A has played against in the past. Like X1, X2, X3, X4, ..., X1000, for example. If game order doesn't matter, then these two orders should give equivalent rating.
(1000 games between A and Xi)
(game bewteen A and B)
vs
(game between A and B)
(1000 games between A and Xi)
In order to do this, we need to retroactively update the skill estimate of every player X1, ..., X1000, to account for the information "A just beat B", because this indirectly influences how much we care about the information "A beat X1, lost to X2, beat X3, beat X4, ..."
Game order also matters because player skill isn't static, it changes over time. If the skill changes over time then it makes sense to consider the chronological order. I don't think you want to say something like, "Magnus Carlsen's rating should go down because he lost to a 1600 player when he was 5 years old". This is an absurd example because in reality such a rating system would probably not look back that far, but if you want to assume player skill is static over the time window used, using "time window = 1 game" seems reasonable enough, and even if you used a slightly larger time window, I would expect your ratings to be almost identical to using a time window of 1 game.