South Africa were 233 for four when bad light stopped play after being sent in to bat.
South African captain Elgar made 67 in an opening partnership of 113 with fellow left-hander Sarel Erwee (41) but Bangladesh fought back before Bavuma and Kyle Verreynne steadied the innings with an unbeaten fifth wicket partnership of 53.
Bavuma, playing in his 50th Test, made 53 not out and Verreynne was unbeaten on 27. It was Bavuma's 18th half-century. He has hit a single century.
The Bangladesh bowlers failed to take advantage of a well-grassed pitch at the start of the match, bowling too many loose deliveries as South Africa scored 60 runs without loss in 13 overs in the first hour.
But the tourists came back strongly, taking three wickets between lunch and tea and keeping the run rate down.
Off-spinner Mehidy Hasan played a key role for Bangladesh, bowling 26 overs in the day and taking one for 57. He bowled 22 overs in his first spell, which enabled captain Mominul Haque to rotate his fast bowlers in short spells.
Mehidy's outstanding contribution of the day, however, was a sensational run-out.
Fielding at backward point he sped across the ground to make a diving stop when Bavuma drove into the covers. He picked the ball up cleanly and threw down the stumps from a kneeling position to dismiss Keegan Petersen, who had looked in good form in making 19.
An easy-paced pitch offered minimal help to the bowlers and Elgar was the only victim of a ball which behaved unpredictably, gloving a catch to wicketkeeper Liton Das off a Khaled Ahmed delivery which lifted sharply off a good length.
South Africa, missing five players who chose to play in the Indian Premier League, fielded two new caps in left-handed batsman Ryan Rickelton and fast bowler Lizaad Williams.
Rickelton looked solid in making 21 before playing an ill-judged pull shot against Ebadot Hossain and spooning a catch to mid-on.
Bangladesh were without opening batsman Tamim Iqbal because of illness.
The start was delayed by 35 minutes after sightscreen malfunctions at both ends, with ground staff struggling to cover both screens with white cloth.
The umpires decided to move back the playing times by half an hour but it was not possible to complete a full day's play at South Africa's most easterly Test ground where bad light is often a problem.
