Spurs re-took the lead through Kane ahead of the hour mark when he guided a whipped cross from Son Heung-min into the far top corner, and the England striker looked to have added another minutes later but the attempt was ruled out for offside.
City pressed to find another equaliser and looked to have done so when Cristian Romero made a handball in the box, granting City a stoppage-time penalty. Riyad Mahrez stepped up and converted from the spot, but Spurs took the lead for the third time moments later as Kane scored to secure all three points.
Spurs had lost their previous three matches in the Premier League before Saturday's narrow victory over leaders City, who are six points ahead of second-place Liverpool with one more game played.
"We kind of thought we'd rescued it. If you look at the whole game we deserved to win. But football is sometimes crazy and today we made easy mistakes," Gundogan said.
Kulusevski sprung an early surprise when he netted his first goal since joining the club in the final days of the January transfer window. The Sweden winger looked sharp on the counter-attack and calmly slotted home into an empty net following a selfless layoff from Son.
Gundogan struck back for City after a period of sustained pressure when took advantage after opposing goalkeeper Hugo Lloris parried a cross onto the floor in front of him, leaving Gundogan with a comfortable finish to level the scores.
Kane, who attempted to push through a move to City last summer, then put Spurs ahead again on 59 minutes with a well-taken finish to convert a whipped cross from Son.
The Spurs striker then had a glaring chance to extend Spurs' lead minutes later when he was sent through one-on-one with goalkeeper Ederson, but he was denied.
The action-packed affair continued as Gundogan found himself back in the heart of the action with another curled effort into the far top corner on 65 minutes, forcing a spectacular diving save from Lloris.
However, Kane remained potent at the other end and even had the ball in the back of the net once again after he took advantage of a deflected Kulusevski shot, but it was ruled out by the Video Assistant Referee (VAR) for an offside in the build-up.
City were granted a lifeline as the game headed into stoppage time when Romero slide to defend a cross into the Spurs 18-yard box but handled the ball with his raised arm, causing referee Anthony Taylor to check the on-field monitor and award a penalty, which Mahrez converted.
But there was still time for Kane to deliver a hammer blow, leaping onto to a floated cross from Kulusevski to nod a 95th-minute winner.
Spurs move up to seventh, four points behind fourth-place Manchester United with two games in hand, as they continue their bid for Champions League qualification.