Simon Msuva scored after 11 minutes for Tanzania, but Zambia overcame the red-carding of captain Roderick Kabwe before half-time to equalise on 88 minutes through Patson Daka.
Earlier on Sunday, Morocco and the Democratic Republic of Congo also drew 1-1, leaving all four teams with a chance of qualifying for the knockout stage.
Morocco have four points, DR Congo and Zambia two each and Tanzania one ahead of the final group matches on Wednesday in the Ivory Coast.
Tanzania were coached by caretaker Hemed Suleiman after Adel Amrouche was suspended by the Confederation of African Football (CAF), then fired by the Tanzanian Football Federation.
CAF banned the Algerian coach for eight matches after he claimed in a TV interview that Moroccan officials were influencing the appointment of match officials and kick-off times.
Both teams showed three changes after disappointing debuts four days ago in a group including shock 2022 World Cup semi-finalists Morocco.
Tanzania collapsed against Morocco in the closing stages when reduced to 10 men by a red card and fell 3-0 while Zambia were fortunate to draw 1-1 with DR Congo.
Since stunning the Ivory Coast in a 2012 final decided by a penalty shootout, Zambia have failed to win in eight Cup of Nations matches, and did not qualify for the past three editions.
Dismal record
Tanzania went into this match with the dismal record of one draw and six losses from three appearances at the African football showpiece.
Zambia began the match in the southwestern coastal city as favourites to win their first Cup of Nations meeting with lower ranked Tanzania, but were soon trailing.
The Chipolopolo (Copper Bullets) surrendered possession cheaply inside their half and a swift counter-attack set up unmarked Msuva to slam the ball over goalkeeper Lawrence Mulenga.
Msuva scored two of the three goals that qualified the Taifa Stars for the tournament, but he has been without a club since leaving Algerian side JS Kabylie last September.
A disappointing opening half for Zambia worsened one minute before half-time when Kabwe was sent off after being yellow-carded a second time.
The second caution came after Kabwe elbowed rival captain Mbwana Samatta, and the Zambian protested furiously for several minutes before leaving the pitch.
In the aftermath of the sending-off, the Benin referee yellow-carded angry Zambia coach Avram Grant, the former Chelsea and West Ham United manager.
Daka, the most high-profile Zambian having played in the Premier League with Leicester City, burst toward goal just past the hour mark and forced a good save from Aisha Manula.
Zambian pressure finally paid off through Daka, whose powerful near-post header from a corner gave Manula no chance.
