The reason this argument doesnt work is because pro-life people believe the fetus is an entity of its own and that abortion is interferring with its life. To them, your argument supports the opposite conclusion to your own and demonstrates why abortion is wrong since its allowing one person to decide the life of another.
Do you think we should abolish laws against murder, rape, battety, etc? Afterall, our beliefs should guide our own lives, not the lives of others, right? Nobody actually believes we should mind our own business when someone defenseless is being harmed, so this argument is only valid if you dont think the fetus is an entity of its own, in which case it changes nothing since people who believe this are likely in favor of abortion already.