To build on my comment on there being no free will and no individual in science (these are concepts from philosophy), because of that then morality is subjective rather than objective.
Maybe I'm perceiving of the "science of morality" thing wrong but science is about objective facts not subjective opinions or subjective values. If we assume morality is to be a science, and that wellbeing is something everyone must agree on, at what point does science determine there is no individual, no free will, and simply let AI dictate what everyone must do?
Couldn't hypothetically there be a scenario where the government makes it a crime to disobey the moral suggestions of your AI? I mean if the AI knows you better than you know yourself then it knows whats best for you, it knows what you should value, it's smarter than you, and it the "science of morality" could be used to promote unlimited moral authority to the super computer.
That is an entirely hypothetical scenario but you never defined clearly what "wellbeing" is, or how we can maintain individuality and free will if science alone determines what everyone should do. Finally you haven't confronted the problem that science by how it is set up would have to determine free will doesn't exist, so many of the concepts behind morality break down (personal responsibility doesn't exist under that reality). Who and what is responsible for mental states even?
I try to stretch my thinking to figure out how you can have a science of morality and I cannot see how it can be done without sacrificing free will, individualism, personal responsibility, the concept of a self, because science supports none of these concepts which means any of them could be declared immoral or not be valued by science.
According to neuroscience the brain has no free will. You don't actually make decisions consciously. Yet science is supposed to tell us about how we should use our free will which is according to science just an illusion?
According to physics the entire universe is determined. Determinism rules the Standard Model and because the Standard Model is our most accurate description of reality supported by science we would have to also accept by science that there is no such thing as choices because there is no free will because everything is determined. So of course if all of that is true there is no individual, no self, etc.
For studying cells this is fine to not worry about these concepts because it's cells, it's atoms, it's not necessarily a person. How would science distinguish a difference in value (morality is based on value) between a collection of atoms which is a person and a collection of atoms which is a rock? Atoms are atoms in science because science doesn't give value to anything, not to atoms, not to people, not to any of the concepts we use to get along with each other.
I’ll start by saying what the “is/ought” divide is, in case you haven’t heard of this before. It’s an old idea, tracing at least to David Hume, and its gist is that there is no way to reason from facts about the way the world is, to statements about the way the world should be. You can’t derive values from data. I’ll use one example to illustrate and then move on.
Example. It’s a fact that rape occurs in nature — among chimpanzees, for instance; and there are some evolutionary arguments to explain its existence in humans and non-humans alike. But this fact tells us exactly nothing about whether it’s OK to rape people. This is because “natural” doesn’t entail “right” (just as “unnatural” doesn’t necessarily mean wrong) — indeed, the correct answer is that it’s not OK, and this is a judgement we make at the interface of moral philosophy and common sense: it’s not an output of science.
I'm not the only one who disagrees with Sam Harris's approach to this topic. While I have not dug deep into Moral Landscapes, if it's anything like his Ted Talk I probably will find the same faults in it. I would like to see him debate this with philosophers to see if it passes peer review.