In a post on his Facebook page, Mr Anderson said: I might not be everybody’s cup of tea – think I am on the spectrum as I struggle to keep my mouth shut.”
However, Simon Harris, who has a seven-year-old autistic son, said such comments annoyed him.
The dad-of-three, whose Man Behaving Dadly blog has 185,000 followers, said: “We can’t allow ‘I might be on the spectrum’ to become a go-to excuse for behaving like an a*se.”
Mr Harris said: “As many autistic adults, or parents of autistic children will tell you, it’s infuriating when people use ‘I think I am on the spectrum’ as some sort of throwaway remark to try to excuse certain behaviour.
“In Mr Anderson’s case, this is his mouth seemingly engaging a few weeks before his brain in a manner that upsets people.
“Getting a diagnosis of autism is a process that can take many years. This sort of attitude from someone with a large audience is incredibly unhelpful.
“I encourage Mr Anderson to get a formal assessment if possible.”
Mr Anderson is no stranger to controversy, having refused to watch the England national football team in protest at the players taking a knee before matches to fight racism and suggesting nuisance council tenants should be forced to live ‘in a tent, in the middle of a field’ and pick crops all day.
Regarding his latest comment, the Ashfield and Eastwood MP said: “If I want to talk openly about my own mental health, I will.
“Since being elected I have helped many families across Ashfield with children who have all sorts of health issues, including autism, which would automatically label someone as being ‘on the spectrum.’
“In adults, it affects a person’s behaviour, it makes communication and social interactions hard, which describes some of my behaviour to a tee; the post receiving criticism is proof of my point.
“I got in trouble at school for speaking out of turn. It’s followed me all my working life, that’s why I made the comment. I know several MPs on the spectrum, including two good friends of mine.”