Safet Vukalic, 42, said the last few years have felt “more and more frightening” as Britain becomes increasingly polarised since the EU referendum campaign.
As a Muslim teenager he was forced to flee Prijedor in Bosnia, where non-Serbs were forced to wear white armbands and put a white flag on their homes.
He and his family came to the UK as refugees in stages during the early 1990s, and he describes himself now as “officially more Londoner than Bosnian”.
He said: “People, especially during Brexit, people have used refugees to push their card for whatever their agenda was, and that was really hard to see, difficult to take.
“And ever since then I’ve been more and more worried about me being a Muslim in the UK and that should not be right, and that’s certainly not how I felt when I arrived first here.
“It’s only the last few years that I have genuinely started thinking that it certainly doesn’t feel as safe as before, and I am afraid of things that have happened in other countries, and sometimes things happen here.”
Mr Vukalic, who works in accounting, lives with his wife and two daughters, aged nine and one, in Sutton, south London.
He was speaking at the launch of the theme for Holocaust Memorial Day 2020: Stand Together, alongside other genocide survivors.
Next year will mark 25 years since the massacre at Srebrenica, where around 8,000 Muslim men and boys were murdered.
He said: “I’m grateful that I survived but I also try to do what I can, try and support the Holocaust Memorial Day Trust, because we don’t want these things repeating, people do not want to go through this, people do not want to be the victims of genocides.
“Nobody deserves to live like that. It’s something we need to continue to fight for.”
Olivia Marks-Woldman, chief executive of the Holocaust Memorial Day Trust, said it was important to be historically aware of how the Nazis deliberately made Jewish people “other” and to show solidarity.
She said: “The more we see swastikas and the more we hear it (anti-Semitism) the more normalised it becomes, so people’s tolerance to it increases.
“We’ve seen hostility-based hatred – Islamophobia, anti-Muslim hatred – is on the increase, anti-gay prejudice hasn’t disappeared, and when these things become normalised and tolerated that’s a very frightening place to be.”