The Simpsons had been accused of racism over its stereotype of Asians.
Azaria, who has played the character since 1989 and also voices Moe Szyslak, Chief Wiggum and Comic Book Guy, told Slashfilm:
"All we know there is I won't be doing the voice anymore, unless there's some way to transition it or something.
"We all made the decision together... We all agreed on it. We all feel like it's the right thing and good about it."
Known for his catchphrase "Thank you, come again," Apu first appeared in The Simpsons's first season.
The show's writers had attempted to address the issue but were accused of making the issues worse.
During a 2018 episode, Marge was reading a novel that had been adapted for modern audiences to make it less offensive.
The scene ended with Lisa turning to the camera and saying: "Something that started decades ago and was applauded and inoffensive is now politically incorrect. What can you do?"
She then looked at a picture of Apu on her bedside table that was inscribed with Bart's catchphrase, "Don't have a cow".
Marge added, "some things will be dealt with at a later date," while Lisa replied: "If at all".
In 2018, Matt Groening, the show's creator dismissed criticism of the character, saying: "I think it's a time in our culture where people love to pretend they're offended."
It was rumoured in 2018 that the show would soon drop the character.
The controversy has grown since Hari Kondabolu's 2017 documentary, The Problem with Apu, in which Kondabolu explored the impact of the character on Asian people in the United States, arguing that the character is a form of "brownface". Other comedians in the documentary relate childhood stories of being called "Apu" or having lines from the show quoted at them by strangers.