Rangers took the lead through Steven Davis’ 20-yard strike but Joe Ledley headed Celtic level on 31 minutes.
The second half was tense with Celtic’s Fraser Forster saving from Steven Whittaker and Rangers keeper Neil Alexander denying Ledley.
But Jelavic fired in off the post in the 98th minute to win it for Rangers.
Rangers’ win ends Celtic’s hopes of a domestic treble but, with both sides chasing the Scottish Premier League and Celtic still in the Scottish Cup, each Old Firm club remain in contention for a trophy double.
And the victory also means a 20th trophy win for Walter Smith in his last cup final as Rangers manager. Captain Scott Brown and defenders Thomas Rogne and Mark Wilson returned to the Celtic line-up, replacing Ki Sung-Yeung, Glenn Loovens and Efrain Juarez, all of whom featured on the substitutes’ bench.
Goalkeeper Alexander retained his place in the Rangers team, Allan McGregor left out as is custom in this competition, while there were also starts in attack for Steven Naismith and Jelavic as the cup-tied Richard Foster and injured Kyle Bartley dropped out.
After the previous Old Firm match’s highly-publicised flashpoints, referee Craig Thomson’s first big decision came on the quarter-hour mark.
Wilson burst into the Rangers box and went to ground as Davis and Sasa Papac closed the Celtic full-back down but no penalty was forthcoming.
Celtic almost went in front when Kris Commons’ teasing cross found Brown diving in but the skipper’s header flew past the post.
And it was Rangers who got their noses in front when Davis pounced upon Charlie Mulgrew’s defensive header, drove at the Celtic defence and guided a low left-foot shot into the net via the post in the 24th minute.
But Neil Lennon’s side levelled when Emilio Izaguirre’s deflected cross was nodded on by Georgios Samaras and Ledley arrived to glance the ball past Alexander.
Rangers thought they had been awarded a penalty when Jelavic went to ground following Rogne’s tackle and referee Thomson initially appeared to point to the spot. However, after advice from one of his assistants, the whistler changed his mind and booked the Rangers striker for simulation.
The Ibrox side started the second period positively and Whittaker forced Forster into a save with a stinging drive.
Celtic went back on the attack and Papac was inches away from scoring an own goal after his attempted clearance from Commons’ centre spun across the face of goal. Maurice Edu tried to force his way past Wilson and the Rangers players claimed in vain that the Celtic defender had used his arm to block the ball’s path but referee Thomson again waved play on.
Another Wylde delivery, this time from a free-kick, nearly caught out Forster, the Celtic keeper retreating to palm the ball over.
Brown and Rogne had made way for Ki and Loovens respectively and Rangers made an enforced change of their own when Madjid Bougherra came off to be replaced by Kyle Hutton.
Loovens’ presence at a Celtic corner caused problems for the Rangers defence and Ledley got a shot away only to be foiled by Alexander’s positioning.
Vladimir Weiss came on for the final seconds of regulation time and, after the match moved into extra-time, the winger played a big part in the final reckoning.
The Slovakian was fouled on the edge of the box and Jelavic’s effort from the free-kick was deflected off the wall but Forster reacted superbly to claw the ball over.
But Jelavic was not to be denied with his next attempt, the striker brushing off the challenge of Mulgrew to latch on to Weiss’s quickly taken free-kick and sliding a shot past Forster, the ball rebounding off the right-hand post but spinning back over the line.
Jelavic tried to get the better of Mulgrew again as the pair went one-on-one at a Rangers attack and, though the Celtic man committed a foul on the edge of the box, referee Thomson produced a yellow card, not judging the offence the denial of a clear goalscoring opportunity.
El-Hadji Diouf replaced Jelavic for the closing stages and set up Weiss for a shot that required a last-ditch save by Forster.
Diouf was then booked for simulation near the halfway line.
Celtic pressed for late leveller and Gary Hooper was denied by an excellent sliding block by Whittaker.
Celtic were reduced to 10 men when Izaguirre was shown a straight red card for barging Weiss over as the Rangers player tried to take advantage of Forster’s absence from his goal, minutes before the final whistle.