It is coming together. I have searched a bit and it could be a fungus called Blossom wilt. I need to get a professional to check it out. Luckily my brother and sister inlaw are gardeners.
I added a woven mess cloth under my planters to try and help them retain water better. We will see how it goes.