Former Chesterfield player Ollie Banks hit the winner with a second half penalty after Mark Ellis had been pushed in the back.
Barrow then survived some late scares with Ollie Clarke, Danny Johnson and Oli Hawkins all going close.
But Cooper was upset at his side’s wasteful first half finishing which saw Josh Kay hit the bar when well placed and Patrick Brough have a shot cleared off the line.
“It was not a great performance from us,” he said. “We surrendered the ball and we needed to play with the ball better, that's how we are set up.
Derby County boss Liam Rosenior has high praise for Mansfield Town after winning Carabao Cup thriller
Mansfield Town boss Nigel Clough proud of players and fans as Stags are pipped by Derby County in Carabao Cup classic
Battling 10-men Mansfield Town push neighbours Derby County all the way in thrilling Carabao Cup exit
See who you can pick out in our fans gallery from Mansfield Town v Derby County
We are a match for anyone at our level believes Mansfield Town midfielder George Lapslie after Stags push League One giants Derby County all the way in cup clash
“If I’m honest, we were a little lucky to come out at the end with a win.
“We should have been ahead early, we had three really good opportunities to score and put pressure on Mansfield.
“It was a good opportunity for us to get in front, you don’t get anything for free and we have to put those chances away as you know what is coming the other end.
“We will have to be much better than that going forward but credit to the players, they dug it out.”
But Cooper was quick to praise his players for their brilliant back-to-the wall defending in the closing stages of the contest, which saw six minutes of stoppage time.
“I am pleased with the result,” he added. “It was a difficult afternoon with the wind and the rain, especially in the first half.
"Mansfield started to play after we missed our chances, they have some really good players.
"When they start firing they will be a dangerous side for a lot of teams.
“We started to play in the second half, we kept the ball better and got the goal.
“It was a penalty and it was a great strike from Banksy.“The players stuck in and defended magnificently after that, Mark Ellis was outstanding again.
“I am pleased with how we managed the game at the end.”
Get more Stags news here