Over time I could not say which anime genre has been the least popular but at least currently a genre that has lost a lot of prominence and even today there are very few new anime of that genre are the horror anime.
In the past I remember that there were very good horror anime and even I have been recommended very good iconic anime where you really ended up terrified after each chapter but at least in recent years there have been very few anime that decide to focus on horror and even fewer that stand out exclusively with a true horror and manage to be entertaining, I'm not a big fan of horror but it would be interesting to see some new anime of this genre from time to time.
have you watched "the another"? it's pretty old but fits horror. or suspense