Clarke pointed out it was early days yet and spirit in the camp remains high ahead of Saturday's home clash with Rochdale.
“It's not doom and gloom,” he said. “You have seen it before how quickly it can turn for other teams. They go on and have successful seasons.
“It's nothing to worry about where we are at the moment. It's just something we need to put a stop to fairly quickly.
“Once we do that we will look a much better outfit.
“You have to dig deep, work as hard as you possibly can Monday to Friday and hope that you get that bit of luck on a Saturday to turn things round.
“We are not a million miles away. We are giving teams good games and creating a lot of chances. We are probably playing a lot more attacking football than most teams.
“But at this moment in time if you are not going to keep a clean sheet then you are going to struggle to win games.
“We are now trying to raise spirits and go again on Saturday. It's one of those where the tide will turn and we knew we are capable of going on a run.
“It's only six or seven games into the season so we're not worrying too much yet. We've just got to keep working away at it and it will change.
“We have a lot of good characters in there – a lot of boys who work their socks off day to day and training is always of a high standard.
“It's not like we've got any passengers in our team. Everyone is trying to do the right things to get us where we want to be come the end of the season.”
He continued: “We knew that we had a tough start against teams that were probably more favoured in the league.
“Unfortunately in the last two or three weeks we've had some indifferent results, down to luck and individual errors.
“I don't think the performances have been too bad at times. We are creating chances and we're not shy in front of goal.
“It's just we are not keeping the ball out of our net at the moment and it's highly frustrating for everyone.
“In recent games we have probably not had the decisions go our way at times either and we have made individual errors all over the pitch, not just pinpointed to one person. It's costing us.”
Clarke knows how important a first clean sheet of the season would be on Saturday.
“It's quite easily seen from outside – we need that clean sheet,” he said.
“That would build confidence in ourselves whether that's a 0-0 draw or whether we can keep a clean sheet and score a couple of goals ourselves to get ourselves back in winning form.
“We desperately need a clean sheet for everyone's sake just to solidify us at the back and give us something to build on.
“In the games we have conceded, we have conceded quite early on in the last few. That's frustrating as you have got to go and score another goal.
“It puts pressure on you for the rest of the game and that's probably what is killing us at the minute. We are giving ourselves mountains to climb.
“A clean sheet would be massive and help us settle down into the season a little bit.
“Once we get one we'll probably go on a nice little run, turn a corner and hopefully kick ourselves back up the league to where we want to be.”
Clarke returns to the side on Saturday from a one game ban for his red card against Harrogate Town and admits the controversial dismissal may have been justified, the midfielder being ruled as the last man as he went in.
“In real time I knew I had given him a little tug,” he said.
“But I felt I had got my leg across to actually win the ball in terms of the tackle that I made. So I was a bit surprised to see the red card come out.
“I thought someone might have been covering round, but when you watch it back it is one of those that can go either way.
“It looks worse from one angle to another. So when you analyse it it's probably one of those that might have been justified and that's probably why we didn't appeal it in the end.”
On his overall season so far, he said: “I have been okay – a bit up and down really. I have got a goal and an assist.
“I enjoyed the first couple of games. Everyone was on a high because we'd won two and drew one. You now look back on the last three or four and analyse it as to how you can do better.
“I know I have made a couple of errors for goals in terms of marking and against Preston I slipped and gave the ball away. You can't be too pleased with your performances when the results are not going your way.”
Despite Clarke's return, Stags will still be without their two most experienced players on Saturday with James Perch injured and Stephen Quinn suspended.
“They will be a massive loss,” said Clarke.
“You look at their careers previously and they have played at the highest level and done well there.
“To have their experience and nous in the positions they play on the pitch is quite important to us.
“It was probably the nucleus of our team down the middle of the pitch that was missing at Walsall.
“It is frustrating to miss these players but it is an opportunity for other players now to step up and make their mark on the team. Over the next couple of weeks we need that to happen.”
On opponents Rochdale, Clarke said: “They have made a tidy start to the season after starting a bit slow.
“We expect them to be a bit more possession-based than most in this division.
“They've just come down and, I can remember playing against them previously, they knock the ball around nicely.
“I am not too sure what to expect in terms of their players as they have obviously changed quite a few.
“Over the next couple of days we will look how we can impose ourselves on them and play our game. If we do that we are more than a match for anyone in this division.
“But at this moment in time for us we need to work solely on keeping a clean sheet and hopefully the match will then take care of itself from there.”