The father of two from the Waterside area was blasted to death in Buncrana at 9.20pm last Thursday week.
The killing was carried out by Republican Action Against Drugs.
Several hundred people gathered in the Top of the Hill area of the city where Mr Allen was from.
One of the organisers, civil rights activist Eamonn McCann, said people in the city were angry at Mr Allen’s death.
“I think there was a very strong feeling, in particular around the area where Andy came from that there had to be some sort of show of opposition of defiance made in public,” he said
“Holding a demonstration like this provides people with an opportunity to show what they think without undertaking retaliatory actions in response.”
Mr Allen’s sister-in-law Leona told the crowd she did not want “any other mother or father to have to experience this heartache”.
She added: “We would like to stress for these people who are hiding behind masks and guns to stop destroying our families, to leave our children alone and to stop these murders.
“We want these death threats to stop. We want these death threats lifted.”