We are the result of all our interactions we have had.
When we talk of affecting our experience, be it positively or negatively, we immediately place a dualistic point of view - a 'me' in here
Yes, we need to put a "me" in there because we are talking about how each of us, individually, develop a different perception, probably making the interpretation of our world different as well.
As far as I know, we are the only tool the universe has to experience itself.