The close ally of Mary Coughlan released a statement saying he had decided to stay on.
However Mr Byrne said he is disillusioned with the direction the party is taking and with the leadership of Mr Martin.
The former mayor of Donegal said Fianna Fáil was “still relevant in Irish politics but was not relevant in the eyes of the Irish nation.”
He said the party had not connected with the Irish people and needs strong leadership to do this, but Mr Martin cannot provide that.
On a personal level, Mr Byrne said Mr Martin was approachable and a nice man.
But asked if he should lead the party Byrne said: “No.”
It’s understood Mary Coughlan spoke to Byrne and persuaded him to stay on.