West Indies were 88-3 on the opening day when fast bowler Atkinson, who had already taken two wickets before lunch, came on to bowl the 35th over.
With his second ball of the over he had Alick Athanaze caught by Joe Root in the slips for 23.
Atkinson produced a superb delivery with the next ball that Jason Holder, returning to Test cricket after nearly a year's absence, could only edge to Harry Brook in the cordon.
Joshua Da Silva survived the hat-trick but next ball he too fell for a duck, an inside edge caught by wicketkeeper Jamie Smith, a Surrey team-mate of Atkinson who was also making his Test debut.
Atkinson had taken three wickets for no runs in four balls, giving him impressive bowling figures of 5-20 in 8.5 overs.
