Walker's third goal for the club came four minutes from time to complete a second-half comeback ahead of the crucial clash with Liverpool at Anfield next weekend.
James Ward-Prowse had capitalised on Ederson's mistake to put Southampton ahead after 13 minutes, and despite an impressive defensive performance from the visitors, Sergio Aguero equalised on 70 minutes from Walker's cross before the England right-back won the game with his first goal of the season.
Walker's position at right-back came under threat in the summer with the £60 million arrival of Joao Cancelo from Juventus. The 29-year-old has suffered a dip in form that has seen him omitted from the past two England squads but he was City's match winner here. First, he found Aguero with a drilled cross from the byline for the equaliser before beating Alex McCarthy with a striker's finish after the Southampton goalkeeper had dropped Angelino's cross.
Ederson rarely makes mistakes, but he was at fault for Southampton's goal. The wet, slippery conditions weren't ideal for a goalkeeper but the Brazilian would still have been expected to gather Stuart Armstrong's shot from the edge of the area. He could have been bailed out by his defenders had they reacted quicker but instead the ball fell to Ward-Prowse to score from two yards out.
Manager rating (out of 10)
7 -- The City manager was becoming increasingly frustrated before the second-half fightback and even took on the role of ballboy to speed up the game. When his team were looking for a winner, he could have thrown on Riyad Mahrez but instead chose Phil Foden and was rewarded when the 19-year-old was involved in the move which led to Walker's late winner.
Player ratings (1-10; 10 = best, players introduced after 70 minutes get no rating)
GK Ederson, 5 -- Fumbled Stuart Armstrong's shot allowing Ward-Prowse to put bundle in the opener for Southampton.
DF Kyle Walker, 8 -- Might be a little disappointed he let Ward-Prowse ghost past him to score but created the equaliser with a drilled cross into Aguero and then scored the all-important winner.
DF Angelino, 7 -- Making his first Premier League start, he delivered the cross from the left that fell to Walker to put City in front.
MF Ilkay Gundogan, 6 -- Created a chance for Stones in the first half and otherwise kept the ball moving from the base of City's midfield.
MF Kevin De Bruyne, 6 -- Worried Southampton's defence with a series of whipped crosses from the right and went close with an indirect free kick after McCarthy had inexplicably picked up a back pass.
FW Bernardo Silva, 5 -- Would have put City ahead with a fierce drive from the centre of the area but for an acrobatic save from McCarthy to push it over the bar.
FW Raheem Sterling, 6 -- Made a good chance for Bernardo Silva in the second half with a jinking run down the right and a cutback into the box.
FW Sergio Aguero, 6 -- Missed the chance to equalise in the second half with a header but conjured a smart finish to put away Walker's cross.
FW Gabriel Jesus, 5 -- Made a nuisance of himself for Walker's goal.
MF Phil Foden, N/R -- Rewarded Guardiola's trust by playing a role in the move that ended up in Walker's late winner.
DF Nicolas Otamendi -- Brought on late to help City see out the victory.