"Police have been told a 71-year-old man allegedly assaulted a 51-year-old man at Neutral Bay Wharf about 2:30 am, before leaving the location," police spokeswoman Alicia McCumstie told AFP.
"The younger man reported the incident and inquiries are now under way by officers attached to North Shore Police Area Command."
Australian police do not usually formally identify people accused or alleging crimes.
But a law enforcement source and the alleged victim, veteran photographer Ben McDonald, both told AFP the man was Swift's dad, Scott Swift.
McDonald said he had been photographing the US pop icon on a "super yacht" in Sydney Harbour, an apparent celebration after the last of her four sellout gigs in the city.
'Punched me in the chops'
McDonald alleges Swift's security detail put an umbrella in his face to prevent him from taking photographs as the singing star was walking down the jetty to a waiting vehicle.
After Swift departed, McDonald claimed a man confronted him and "took it further". "He punched me in the chops", McDonald said.
"I didn't know who he was," said McDonald, who said he initially believed it may have been an overzealous local security contractor "trying to impress the Americans".
"But I looked at photos and saw him holding hands with Taylor, and it was her dad," he said.
"It was a shock. That's never happened to me in 26 years."
A spokesperson for Taylor Swift responded to the allegation with allegations of their own, claiming there had been pushing and aggression from unnamed people at the waterside.
"Two individuals were aggressively pushing their way towards Taylor, grabbing at her security personnel, and threatening to throw a female staff member into the water," the spokesperson told AFP.
Swift's representative did not comment on the alleged assault by the star's father, or respond to requests for further clarification about what took place.
The megastar is currently in the midst of a worldwide blockbuster Eras Tour, which is expected to be the highest-grossing musical tour of all time at over $1 billion, according to Pollstar.
She is due to head to Singapore this week for further tour dates.
Police refused to comment on whether Scott Swift had been contacted or questioned in advance of the group's departure.
Online, fans rushed to his defence, who some fans dub "Papa Swift".
"Free Scott Swift," posted social media user @Soyytv.
