I don't think I understand it clearly. I just changed my recovery account a week ago and also I have no proxy in witnesses because I removed it already and voted for myself.
Also, it's kind of rude. I only have 0 scores despite being active all the time or keep on posting. Just kidding.lol
Your recovery account will change officially in 30 days from the transaction, after that happens, your score will be approximately
438.The other thing that's losing you points is the amount of accounts that you've trusted with your posting key
Even some old steem dApps (steempeak.app when it's now peakd.app). I'm not saying all of these have to go, just that you should review them and consider whether you use them regularly or at all, and if not, remove their authority!
If you need an easy way to revoke authorities, go here:
https://peakd.com/@mrnightmare89/permissions and then go to the Authorities tab.
Hope this helps!