I imagine that its unavoidable to fend off visitors/vultures who nostalgically visit a dying institution.
To mediate it and perhaps even benefit from it, I think a concise and open plan of visitor communication is necessary, ie what will happen to our collections? Where can you still visit them? What would you like to see done with them? If visitors didn't connect with the collections in this life, they will connect with them in the next, and maybe this funereal visit will inspire them to seek out other cultural history institutions to support and connect with while they still live!
Side note: If the PHM ever does merge its collections with Temple University, I personally think it would further stifle the life of those collections and the city's history unless marketed and exhibited properly--so unless Temple engages with and entrusts power to the community. Skills they're not known for.