i am learning too for over a year now! how long are you learning?
From my perspective till now if i watch without subs i get only a small part of it and one huge difference that i came to realize is that the language is a bit different from what they speak in real life and what in anime. a simple example is the anime with the historic attribute samurai, swords ancient era etc etc. a lot of the words used there in real life if you use them they will make fun of you :P
yeah I figured that part out already, its important to have friends who speak japanese so that I can run it all by them before I use it. I have only been learning seriously for about a month I guess. Lots of progress to make.