Police identified the assailant as being a "convert" to Islam, who was born in 1999 in the town of Mladenovac, some 50 kilometres (30 miles) from Belgrade.
The attack happened around 11:00 am (0900 GMT) on Saturday morning in the Serbian capital, when the attacker shot the officer who was on duty outside the Israeli embassy.
The wounded policeman shot the attacker "who died as a result of his injuries", said Interior Minister Ivica Dacic.
Authorities said some arrests had been made and a number of people known to the security services were suspected of being linked to the attack.
Police also said they were carrying out searches in a number of locations.
"There are several more persons that we are looking for, certainly for one who is on the territory of Serbia," President Aleksandar Vucic told reporters after visiting the wounded officer.
The attacker and his associates had been tracked by authorities before Saturday's attack but there was not enough evidence to arrest them, he added.
The officer, who underwent surgery in hospital, was in his guard booth when the attack happened.
According to authorities, the assailant had moved to live in Novi Pazar, which is a historical and political centre of the Bosniak Muslim minority in Serbia, and a centre of Islam in the country.
Dacic told reporters that there were early indications connecting the attack with people suspected of being linked to the Wahhabi movement -- an ultra-conservative branch of Islam that dominates in Saudi Arabia.
He said several people had been arrested for "prevention reasons" and that overall security had been stepped up in Belgrade. Special prosecutors had taken over the case, added Dacic.
'Terrorist act'
Serbian Prime Minister Milos Vucevic strongly condemned what he labelled a "heinous terrorist act".
"This was an act of insanity, which cannot be attributed to any religion and any nation. It is a crime of an individual," he said, according to quotes from the Beta news agency.
The prime minister said his country would firmly respond to the threat of terrorism and stressed Serbian citizens could "feel safe".
The Israeli foreign ministry called it an "attempted terrorist attack in the vicinity" of the country's embassy in Serbia.
"The embassy is closed and no employee of the embassy was injured", it said in a statement, saying the circumstances are still being investigated.
Israeli ambassador to Serbia Yahel Vilan on X voiced his gratitude to the wounded officer, "who courageously prevented the attack".
"I am convinced that the investigation by the competent authorities of this shameful attack will identify all responsible persons and further contribute to the preservation of Serbia as a safe country."
Meanwhile, Serbia's top Islamic cleric Senad Halitovic strongly condemned the attack and wished the wounded officer a quick recovery.
"Such crimes are against all religious teachings, especially the teachings of Islam. Today's crime is the work of a mindless individual," he said in a statement.
The Balkan nation has continued arms sales to Israel after the war in Gaza started with Hamas's October 7 attack on southern Israel, which claimed 1,195 lives, mostly of civilians, according to an AFP tally based on Israeli figures.
The militants also seized hostages, 116 of whom remain in Gaza although the army says 42 are dead.
Israel's retaliatory offensive has killed at least 37,834 people, also mostly civilians, according to data from the health ministry in Hamas-run Gaza.
