The former captain took more than two minutes to take strike and opposition skipper Shakib Al Hasan appealed, which was upheld by the on-field umpire.
Mathews came into bat after the fall of Sadeera Samarawickrama in the 25th over but was unable to secure his helmet strap tightly enough, an action which delayed the resumption of play.
Following the Bangladesh appeal, the umpires held a lengthy discussion with the players including Mathews.
They ruled Mathews to be out and the 36-year-old veteran reluctantly trudged off with most spectators appearing bemused by the decision. Some Bangladesh fans applauded.
Former South African fast bowler Dale Steyn writing on X, formerly Twitter, said: "Well, that wasn't cool".
Ex-india batsman Gautam Gambhir, on 'X', said: "Absolutely pathetic what happened Delhi today! #AngeloMathews."
Australian Test batsman Usman Khawaja also blasted the decision.
"Angelo made his crease then his helmet strap broke. How is that timed out? I'm all for timed out if he doesn't make his crease but this is ridic. No different then a batsman getting to the crease then taking 3 minutes to face up," he wrote on social media.
According to laws of cricket, after the fall of a wicket or the retirement of a batsman, the incoming player must be ready to receive the ball within two minutes.
The Association of Cricket Statisticians and Historians said there have been six instances of "timed out" in first-class cricket.
"There have been no instances recorded in List A or Twenty20 cricket until Mathews," they wrote on X, formerly known as Twitter.
The vastly-experienced Mathews was playing in his 225th one-day international on Monday in a career which has yielded 5,900 runs and 122 wickets.
He has also played 106 Tests and 78 Twenty20 internationals.
There are 10 modes of dismissal in cricket, the rarest of which is "timed out".
Former India captain Sourav Ganguly escaped being timed-out when he had to rush onto the pitch after the fall of a wicket during a South Africa Test at Cape Town in 2007.
India had lost a wicket and with next batsman Sachin Tendulkar not allowed to take the field owing the time he had spent off the pitch, and VVS Laxman taking a shower, the onus to come out fell on Ganguly, who was still in his tracksuit.
The timed-out rule in Tests was three minutes but Proteas skipper Graeme Smith did not appeal and waited patiently for Ganguly to stride out despite exceeding the time limit.
