Oh wow! See?! You're already part of the way there! A lot of people would only dream of having a fruit producing tree right in their back yard! I have seedlings of avos but they're a far way off from my plate ;)
OK so, I know you feel rushed for time but if cheese and dairy are a problem, there are tons of premade vegan sources which you can either try to order online or you could check out some recipes, they're super easy and yo just throw them in the fridge and eat them when a craving comes! I have tons of cheese recipes and have one on hand in general at all times in the fridge ;)
I'm hoping you are able to find a way to make it work! I know dairy can be the hardest, most addictive part but it's also proven to be one of the harder things on our bodies and environment so it's incredible if substitutions can take up the slack :)