Well that's philosophically debatable! Trolley problems etc.!
Some people btw don't understand why philosophers bother with thought experiments. Many times, as in this case, they are meant to help you figure out your stance regarding whether inaction (among other things) can be immoral. So say for instance I know someone is coming to your house to kill you, and I don't notify you or the police. According to what you said above, a person who does notify you is neither better, nor worse, morally, than me. My intuition says that's false. Yours might say it sounds fine. What matters is that the thought experiment has helped us figure out where we stand on this issue.
It could definitely be viewed as a trolley problem because it's basically pitting two different properties against one another. Without action, your body will be destroyed but with an action, the window will be destroyed. To "pull the lever" is to choose to destroy someone else's property in order to save your own. Because of the vast disparity of value between the two properties, I would be willing to "pull the lever" and I believe the window's owner would understand and forgive me.
I know it can be emotionally tempting to say that it's morally wrong to not help someone in need but we have to keep our emotions out of these things and work it out rationally. Only actions can be moral or immoral. Think of it this way: Only things can have physical properties. A ball can be red. A ball can be heavy. A ball can be smooth... The absence of a thing cannot have physical properties. The absence of a ball cannot be red, heavy, or smooth. Actions are the same. Something has to exist in order to have any defining characteristics.
You could think of it another way too: If we were responsible for every death that we could prevent, we'd me mass murderers every single day. Think about it. We know that there are people dying right now of starvation and we could go and feed them right now, if we really wanted to. In every second of every day, there's an infinite number of good things we could be doing and an equally infinite number of bad things we could be preventing. Fortunately, it's only the actions we take that can have defining characteristics. Our actions can be fast or slow. They can be deliberate or accidental. They can be moral or immoral. The absence of an action is just nothing and cannot be said to have any such defining characteristics.
If person A is drowning, and person B refuses to save him, I'm not the kind of person who will say this should be legally punishable. B might be scared, or he might not know how to swim, or he might not want to endanger his own life for that of A, etc. But what I am saying is that the sentence "it is immoral not to do X" has meaning. Within reason, I may pass moral judgement on a person who doesn't do certain things. Those things won't include not helping starving kids, but they might include not doing anything to change the states of affairs that allow things such as starving children to exist.
Your examples about physical properties apply to physical objects. A woman who complains that her husband doesn't sweet-talk to her, is complaining about something that is quite present, and that's his indifference. A husband who says "lack of tenderness cannot be a bag thing, it's an absence of something, not a presence of something, and as such it can at most be called neutral", will merely add to her list of grievances. Just as cowardice is often lack of action, similarly immorality can be due to a lack of action.
What you're describing is a preference for positive action. I'm sure most people would share that preference but it shouldn't be confused with morality.
The absence of sweet talking is the same. The woman has a preference for sweet talk but its absence cannot be said to be immoral.
Cowardice isn't immoral either. It's a word used to indicate one's subjective preference for action where there is none.
Well okay, at least you're 'biting the bullet', in the sense that your intuitions about the examples I gave are in line with your opinions about what can and can't be moral. And that's what extreme hypotheticals are all about!
But just a last one cos I can't help it! : A mother who gives birth to a child, but refuses to nurture it, hears its cries but refuses to move a toe, is - all other things being equal - not morally bad, to you, just morally neutral?
Above you said something about having the duty to be a provider for your daughter, so maybe that might relate to this.
I'm glad you brought this one up. Most anarcho-capitalists revere the writings of Murray Rothbard and he argued that it was morally acceptable to leave the child to die... but I disagree with him. Rothbard neglected to consider the responsibility that parents incur when conceiving a child. One could use the analogy of a house. Having sex could be thought of as "inviting a child into existence" the way one might invite someone into their home. Once the guest has arrived, you're responsible for its well-being until it leaves. In the case of a newborn child, it's in a state of complete dependency so you might think of it as your house guest that was injured and rendered unable to walk or take care of itself. As long as that helpless guest is in your home and in a state of dependency, you're obliged to take care of it until it can get up and leave on its own.
Acknowledged.