Stephen McLaughlin gave the visitors something to defend with his third-minute opener.
But former Doncaster striker Alfie May levelled in the 73rd minute to send the game to extra time.
And it looked destined to be decided on penalties before Will Boyle’s 110th-minute winner sent the Stags crashing to an agonising exit at Whaddon Road.
Only finishing missing in another fine Mansfield Town display says boss Nigel Clough, but injury list lengthens for his depleted squad in Leyton Orient defeat
One more new face in for Mansfield Town as early injuries mount?
Mansfield Town made to pay for missed chances as they lose at Leyton Orient
See who you can pick out in our fans gallery from Mansfield Town v Derby County
Former Mansfield Town boss Mick Jones dies aged 75
Clough named two changes from the side which beat Salford City last time out.
James Perch fell ill on the morning of the game and was replaced by Corey O’Keeffe, with McLaughlin coming in for injured skipper Ollie Clarke.
And McLaughlin got the Stags off to the perfect start as he opened the scoring in the third minute.
The midfielder ended a three-month goal drought as his first-time finish from the edge of the box was helped in by hosts’ keeper Josh Griffiths.
Harry Charsley did well to clear George Lloyd’s header off the line after Ben Tozer’s long throw caused the defence problems.
And Michael Duffy’s side came even closer to an equaliser when Lloyd rattled the woodwork.
Potent May fired straight at Stags’ stopper Marek Stech as the hosts continued to threaten.
And the Robins squandered another great chance when May blasted over the bar.
The Stags were under the pump after the restart as the Robins upped the ante.
Ryan Sweeney did well to dispossess marksman May before he had the chance to level from mid range.
Stech got down well to deny the ever-present May from 25 yards before he missed the target from inside the box.
But the striker made up for countless missed chances as he levelled 17 minutes from with a fine curling finish after cutting inside from the left.
Sweeney fired home a great strike only to be denied by the linesman’s flag as the game went to extra-time.
The Stags had the best chances of the first half as Jamie Reid forced a good Griffiths save before George Lapslie’s deflected effort went close.
The gutted Stags were beaten 10 minutes from penalties as a Tozer long throw was diverted in by Boyle.
Stech superbly kept out substitute Alex Addai in the closing stages, with the Stags unable to find a late leveller.
Cheltenham: Griffiths, Tozer, Raglan (Addai, 71), Boyle, Hussey (Freestone, 46), Thomas, Blair, Chapman (Clements, 105), May, Williams, Lloyd (Azaz, 56).
Unused subs: Harris, Long, Campbell, Bowry.
Stags: Stech, O'Keeffe, Benning (J. Clarke, 116), Sweeney, Rawson, Charsley, Maris (O'Driscoll, 108), McLaughlin (Cook, 79), Lapslie, Bowery, Reid.
Unused subs: Stone, Menayese.
Referee: Tom Nield (Yorkshire)