"Richard Bennett was informed of the decision that he would not be welcome to return to Afghanistan several months ago," a diplomatic source confirmed to AFP after local media reported the ban, citing a Taliban government spokesman.
Bennett marked two years in the role on May 1.
Since returning to power in August 2021, Taliban authorities have enforced rules based on a strict interpretation of Islamic law.
Women have borne the brunt of restrictions the United Nations has labelled "gender apartheid" which have pushed them from public life.
They have been barred from secondary and higher education, as well as blocked from working in many jobs or entering public parks, gyms and travelling without a male relative.
The Taliban government remains unrecognised by any other state, with its restrictions on women a key sticking point.
Taliban authorities have systematically dismissed criticism of their policies from the UN and the international community.
However, when the ban was apparently issued months ago, the Taliban government stressed that their issue was not with human rights monitoring and reporting, but with Bennett personally, according to diplomatic sources.
Earlier Tuesday, Afghanistan's Tolo News quoted chief Taliban government spokesman Zabihullah Mujahid as saying that Bennett had been banned "because he was appointed to Afghanistan to spread propaganda and he is not someone whose words we can trust".
"He took small issues and exaggerated them for propaganda," he said.
Strong statements
In recent months, Bennett has issued strong statements on women's rights in Afghanistan at moments when the country was in the international spotlight.
Last week, as the Taliban authorities celebrated the third anniversary of their takeover of Afghanistan, Bennett joined 29 other UN experts in a statement urging the international community to "not normalise the de facto authorities or their appalling human rights violations", he said on X.
In late June, Bennett condemned the decision to exclude rights issues from the agenda and Afghan women and civil society representatives from the table at UN-hosted talks in Qatar -- a condition of Taliban representatives' attendance at the meetings with the international community.
"The cost is too high," he wrote in a New York Times opinion piece.
In New York, Stephane Dujarric, a spokesman for UN Secretary-General Antonio Guterres, did not confirm or deny the ban Tuesday but said: "Special rapporteurs play a very critical part in the global human rights architecture. We encourage full cooperation with them."
Special rapporteurs like Bennett are independent experts within the Special Procedures body of the Geneva-based UN Human Rights Council.
The UN Assistance Mission in Afghanistan (UNAMA) maintains a human rights monitoring and reporting function in the country.