The world's most populous country has become a key AI battleground in the last few years, as US tech giants look to find new users for their services and tap into fresh pools of talent.
In recent months, top executives including Nvidia boss Jensen Huang and Meta's chief AI scientist Yann LeCun have visited India.
On Tuesday, Nadella said the $3 billion investment would include the setting up of new data centres.
"India is rapidly becoming a leader in AI innovation, unlocking new opportunity across the country," Nadella said.
"The investments in infrastructure and skilling we are announcing today reaffirm our commitment to making India AI-first, and will help ensure people and organisations across the country benefit broadly."
The global embrace of AI has boosted sales of Microsoft's key cloud services, which have become the core of its business under Nadella's leadership.
The announcement comes less than a week after Microsoft president Brad Smith said the company was on track to invest $80 billion in AI this fiscal year.
Microsoft was on pace to invest about $80 billion this year to build out AI datacentres, train AI models and deploy cloud-based applications around the world, according to Smith.
"The United States is poised to stand at the forefront of this new technology wave, especially if it doubles down on its strengths and effectively partners internationally," he said in an online post.
