The Reds, who are now five points clear of fifth-placed Tottenham, had lost at Swansea in the league and West Brom in the FA Cup in their previous two games.
"It was not a brilliant performance, but it was very mature and exactly what we needed at the right moment," added Klopp.
"You need to win all the time, we know that. Coming back on track is the most important thing. There is an open bill and we need to pay that back."
Emre Can's deflected long-range effort found the bottom corner to put the Reds ahead midway through the first half, after the hosts' Laurent Depoitre had missed the best early chance.
Roberto Firmino threaded a finish between goalkeeper Jonas Lossl and his near post for a second just before the break.
Mohamed Salah took over penalty-taking duties after Firmino's missed effort in the FA Cup defeat by West Brom and put the outcome beyond doubt after Philip Billing's foul on Can.
The defeat leaves Huddersfield just one point above the relegation zone.
