New Zealand won the first of three matches on Saturday by eight runs.
In their attempt to wrap up the series New Zealand produced a steady performance until Hay opened up with three overs remaining.
In the final 18 deliveries New Zealand scored 43 runs, 35 of them coming from Hay as he atoned for his first-ball duck in game one.
It was again a shaky start for New Zealand with Rachin Ravindra, who has never settled as an opener, out for one in the third over.
Tim Robinson was more confident at the other end and with Mark Chapman the pair put on 69 for the second wicket.
But in the 10th over, after belting the first ball from Wanindu Hasaranga over the midwicket fence, Robinson misread the next delivery and was bowled for 41.
Another wrong'un from Hasaranga undid Chapman who was stumped for 42 off 29 deliveries.
Glenn Phillips belted a rapid 23 before he was caught attempting to pull Matheesha Pathirana over the boundary.
Daryl Mitchell, New Zealand's batting hero in the first match, was run out for 18 by Binura Fernando one delivery after Binura had missed a wide open chance to run out Hay on seven.
Hasaranga returned the best figures for Sri Lanka with 2-28.
