Direct peer to peer delegations have been thought of, and while it does bring a lot of simplicity, it also lacks a lot of the flexibility that pools bring (oversubscription), and when you think about it how often do you find yourself in a position where you want to delegate rc to people ? That happens very rarely for me as an user, as a dapp owner though that happens quite often when I create accounts (which is handled by design).
I think rc delegations will be something very few users will actually do. And if you want to contribute yourself to someone else, I bet there will be front ends to do it to hide some of the complexity.
And finally I envision that pools will be smart and always overdelegate because nobody uses all of their rc. So if you want to contribute to an user, you could just delegate that pool. Because in the long run, pretty much everyone will either have hp or be fed RC by a pool until they have hp.
My point is, I think there is a use case for direct delegations but it's a very niche one, and I don't think it's worth the added complexity of supporting both at the same time, or the time spent to undo the current system and make it a direct peer to peer one.