Speaking to the Today show on BBC Radio 4, Mr Austin said: “I’m not a Tory, this isn’t where I want to be.
“I wouldn’t say Boris Johnson is unfit in the way I say that about Jeremy Corbyn.”
Writing in his local paper the Express and Star Mr Austin said: "The Labour Party has been poisoned by racism, extremism and intolerance under Corbyn’s leadership."
Mr Austin, a former special advisor to Gordon Brown, left the Labour party earlier this year citing the party’s failure to tackle anti-semitism.
He told the BBC's Today programme this morning: "Look, the public has to make this choice. The British people have to decide this.
"Lots of traditional Labour voters are going to be grappling with this question.
"If they have got to face up to that, then I don't think people like me should have the luxury of running away from it.
"What Jeremy Corbyn has done to the Labour Party, I don't want him to be able to do that to the country."
When he resigned from the party in February, Mr Austin said: "I always tell them the truth and I could never ask local people to make Jeremy Corbyn Prime Minister.
"I am appalled at the offence and distress Jeremy Corbyn and the Labour Party have caused to Jewish people."
Mr Austin is not standing in this election.
The statement comes after Labour deputy leader Tom Watson announced he was stepping down.
Speaking this morning about the anti-semitism scandal within the Labour party and Mr Watson's resignation, Mr Austin said: "I think it is enormously significant.
"If Tom thought that Jeremy Corbyn was fit to lead our country and fit to form a government, then he would have been in that Cabinet. Would he really be standing down?
"Anybody who has spoken to Tom knows what he thinks about Jeremy Corbyn.
"More importantly, they know how appalled he is, like so many other people, by the scandal of anti-semitism that has poisoned the Labour Party under Jeremy Corbyn's leadership."