Olyroos coach Graham Arnold had promised his players would produce "something special" in Japan and they duly delivered under the closed roof of the Sapporo Dome.
Australia's win, which sees them top the Group C standings, avenged 1-0 defeats to Argentina at the 2004 and 2008 Olympics when the South Americans went on to win gold.
The game began in a frantic fashion with Daniel Arzani bursting past several defenders in the first minute and Argentina only just keeping him out.
Harry Souttar produced a big last-ditch tackle for the Olyroos at the other end before the deadlock was broken after 14 minutes.
Duke missed his kick but Wales, the Western United forward, came in behind him to slot home.
But Australia were lively and Riley McGree, a 2018 Puskas Award nominee with an incredible scorpion kick effort for Newcastle Jets, almost scored another memorable goal.
This time, however, his powerful overhead kick was held by Argentina goalkeeper Jeremias Ledesma.
The Olyroos were handed a crucial one-man advantage on the stroke of half-time.
Ortega was booked for fouling Nathaniel Atkinson and, just seconds later, the Argentina full-back was shown a second yellow card for pushing McGree.
Duke nearly doubled the Olyroos' lead within seconds of the restart as his chip clipped the crossbar. Argentina edged their way back into the contest but the flow was disrupted by a raft of bookings.
Duke did well to set up Tilio and the substitute swept home from 20 yards just moments after coming on to the pitch. Australia almost extended their lead in injury time when Duke struck the outside of a post.