I usually confine my discussions to sane adults
Sure. If I force adults to cross the road only at pedestrian crossings, how does that harm them? If I force drivers to stop at a red light, how does that harm them? There's an endless list of things we force sane adults to do that don't harm them. But then again, it depends on what you mean by "harm". If I force someone to pay taxes, how does that harm them? Just because you don't want to do something doesn't necessarily mean that by doing that thing you are harmed.
I oppose the idea that one person (or group of people) can be morally superior to another in a way that allows one group to control another group against their will (I am talking exclusively about sane adults)
Well, I was just going by your statement:
Ethical people avoid initiating harm against peaceful people.
Are you suggesting that all sane people are ethical? Or are you saying that ethical people are not morally superior to unethical people? No, I think I it get now. You're saying that ethical people should not control unethical people against their will. In other words, unethical people should be allowed to do whatever they want.
So now, back to your statement:
The person who gets to define the harm is the intended victim
If anybody and everybody can define harm, do you not see how messy that gets? Force me to cross the road at a designated place? No, I don't want to. That's harming me. Force me to stop at a red light? Nope, that's harming me too.
So when I say "Ethical people get to decide what constitutes harm," I'm not saying that only a few select people get to decide. What I'm really saying is that I don't trust just anyone to define for themselves what constitutes harm. Otherwise, anything unpleasant to any individual will be called "harm".
In another response, you said that if one is not happy with their salary, find another job. No one is forcing you to accept the salary, right? So you are free to work somewhere else. But if you want to work at that particular place, you accept the rules of that place.
How is that different from taxes? You don't want to pay taxes, find another place to live where you don't have to pay taxes. No one is forcing you to pay taxes. But, if you want to live here then these are the rules.
Would I pay taxes if I didn't have to? Nope. But I do not consider myself a victim for having to pay them either. Furthermore, I don't consider paying taxes as harmful to me. They're simply an inconvenience that I need to accept for wanting to live and work here.
To me, calling someone who doesn't want to pay taxes a "peaceful person" and a "victim" who is "harmed" sounds overly and unnecessarily dramatic.
If being asked to play by certain rules constitutes as forceful harm then one should probably find another game to play.
I think you are thinking of the end result as opposed to the entire process. If the mayor of New York outlaws sugary drinks greater than 16 ounces, or outlaws fried foods, you could say that everyone will be healthier and that there is a "net benefit." Such laws prevent people from doing what they want to do. That is loss of liberty. Nobody "needs" a cookie or ice cream, but it is a loss of liberty when those things are outlawed and people want them. Would it really "harm" you if I forced you to wear a hijab or a skirt? How about if I prevented you from wearing a hijab or a skirt? I say it is loss of liberty, and that constitutes harm the way I have been describing it.
In the street-crossing example, there are places near me where it is a mile to walk the long way (1/4 mile to the corner, then 1/4 back to your destination, then repeat when you come home) vs crossing the road in the middle. The "loss" might be 20 minutes for one person, or an hour for somebody who can't walk quickly. It could also mean being late for work or not having time to play with the kids or not having time to see grandma on her death bed before she dies.
Based on my prior paragraph, I would say another "harm" is establishing a pattern of behavior in which people pass laws based on failure to consider liberty or a tendency to confuse "I don't see any reason to stop controlling your behavior" with "there is no reason to stop controlling your behavior." Does that make sense? I say this because your question indicated that you see no harm in preventing someone from crossing in the middle of the street, but I suspect that now that you thought about it, you can understand some "adverse effects" of such a rule or similar rules. I don't know your final ruling on this type of laws. Restrictive laws made on hasty lack of consideration could easily abound if they are tolerated.
Some people don't mind being micromanaged by others (e.g., being told that they are not allowed to eat cookies and ice cream due to the health risk). Others do mind. It is a perspective-taking exercise to understand how others experience life. Some people are better than others and understanding the perspective of others.
As for escalation, how exactly would you prevent me from crossing in the middle of the street? I start to cross the street, you say "don't cross here," I start to cross, then what? Do you feel justified in escalating to a knife fight if I fail to comply with your ethical idea or a law that says I can't cross at that spot? What's the harm then? Do you just blame the victim for not complying and feel justified in whatever follows?
I should probably clarify that I include "loss of liberty" as a type of "harm" at the top of these posts.
As for letting the victim define harm, the words above are a reason why: because an observer might not know what the victim perceives as harm, so let the victim define it. This is absolutely essential for anything that I would view as ethical or prosocial behavior. Religious laws are not based on this idea, they, like you, believe they can determine the correct action without any input from the victim.
other games: try to get the person to understand that loss of liberty=harm; move to another town (has no effect where I live); shoot out. None of these are inconsistent with your actions because you do not respect the autonomy of others and see yourself as in a position of moral superiority that justifies controlling the behavior of others or harming them if they don't comply.