Divers worked desperately to pull the five passengers from the helicopter but two of them died at the scene while the other three died in hospital.
The pilot managed to free himself and was rescued by a passing tugboat.
The helicopter, which had been chartered for a photo shoot, crashed near Roosevelt Island.
Police commissioner James O'Neill told reporters it had been privately leased by a group of photographers.
Video posted on social media showed a red helicopter appearing to make a semi-controlled descent at around 19:00 (23:00 GMT) before landing with some force in the water. It then began to tip over, its rotors still spinning and slicing into the water.
Divers from the emergency services pulled people from the water
At a press conference, New York Fire Commissioner Daniel Nigro called it a "great tragedy".
"We are told the five people were all tied tightly in harnesses that had to be cut and removed," he said.
"It took a while for the divers to get these people out. They worked very quickly, as fast as they could. That's 50ft [15m] of water there."
The divers had to work against currents and water temperatures of below 4C, he added.
Journalists at the scene reported seeing CPR being performed on the rescued passengers as they were rushed to ambulances.
The Federal Aviation Authority said it was investigating the incident.
The National Transportation Safety Board would determine the cause, it added.