Stephen McLaughlin gave the visitors something to defend with his third-minute opener.
But former Doncaster striker Alfie May levelled in the 73rd minute to send the game to extra time.
And it looked destined to be decided on penalties before Will Boyle’s 110th-minute winner sent the Stags crashing to an agonising exit at Whaddon Road.
But Clough was quick to praise the spirit and fight shown by his players and hopes they can take that into their remaining league fixtures.
“The effort and the spirit shone through for me. The players put their bodies on the line and couldn't have given any more,” he said.
"There is a way to lose if you lose, you give everything and we did that.
“Hopefully the lads take from it what we did and that is the spirit.
“We have to show that for the rest of our league games, starting at Oldham.”
Clough admitted he was also very disappointed to be out of the FA Cup after seeing chances go begging.
“We are very disappointed to be out of the FA Cup,” he said. “We put so much effort in and battle so hard with a bit of a patched up team.
“To concede in the way we did was very disappointing.
“We started very well and got the goal we deserved. They penned us back a bit after that and we got away with one of two in our box, but really we dealt with what they threw at us.
“I thought we were comfortable with 15-20 minutes to go and had dealt with what they chucked at us. I felt that we might be able to pick them off and get a second.
“At the end of 90 mins we said our chance was still there. We had some good situations in the first half of extra-time. If we'd got the goal we would have gone on and won.
“We did not play anywhere near as good as we have in the last few games but credit to Cheltenham.
“They have had 10 days off and they were full of energy and swarming all over us, they made it very difficult for us.”