Both Arsenal and Brighton had maximum points from their opening two games and the Gunners looked set for another victory when Kai Havertz pounced on a Lewis Dunk error to lob in the opening goal.
However, the game turned just seconds into the second half when Rice controversially received the first red card of his career.
The England midfielder was deemed to have kicked the ball away as Joel Veltman shaped up to take a Brighton free-kick and was shown a second yellow card.
"I was amazed, amazed, amazed!" an outraged Arsenal boss Mikel Arteta said. "Amazed because of how inconsistent decisions can be.
"In the first half, there are two incidents and nothing happens. Then, in a non-critical area, the ball hits Declan, he turns around, he doesn't see the player coming and he touches the ball."
Brighton quickly made their man advantage count as Joao Pedro swept home the rebound after David Raya denied Yankuba Minteh.
Both sides had chances to win it as Bart Verbruggen denied Havertz and Georginio Rutter headed off target late on for the visitors.
But Arsenal had to settle for a point that could see them cede early ground in the title race.
Defending champions Manchester City can go top when the visit West Ham in Saturday's 1630GMT kick-off.
Duran delivers for Villa
Villa bounced back from their defeat to Arsenal last weekend to secure their second win in three games.
Amadou Onana stretched to turn in Jacob Ramsey's cross from a well-worked free-kick to open the scoring.
Ollie Watkins is yet to find the net this season after helping England to the Euro 2024 final, but his understudy Jhon Duran has started the campaign in fine form.
The Colombian's towering header just two minutes after replacing Watkins gave Villa the breathing space they needed.
Facundo Buonanotte pulled a goal back, but the Foxes remain winless on their return to the Premier League.
Southampton have found the jump up to the top flight just as hard.
Brentford eased past the Saints 3-1 with Bryan Mbeumo scoring twice.
Everton suffered another gut punch on their final season at Goodison Park as Bournemouth came from 2-0 down after 87 minutes to win 3-2.
Two quickfire goals from Michael Keane and Dominic Calvert-Lewin early in the second half had put the Toffees in control.
But Antoine Semenyo gave Bournemouth life before Lewis Cook and Luis Sinisterra ensured Everton remain without a point this season.
Ipswich got their first Premier League point in 22 years in a 1-1 draw against Fulham.
Liam Delap put the home side in front at Portman Road but Adama Traore secured a draw for Fulham.
Honours were also even in a 1-1 draw between Nottingham Forest and Wolves.
Manchester United host Liverpool in the blockbuster fixture of the weekend at Old Trafford on Sunday.
