The Crew, who play in the final of the CONCACAF Champions Cup against Mexico's Pachuca next Saturday, had a made a slow start to the regular season in MLS but Wilfried Nancy's team look to be hitting their stride.
Uruguayan Rossi put Columbus ahead, on the stroke of half-time, with a penalty awarded in unusual fashion.
It was Orlando who were awarded a penalty after Crew defender Steven Moreira brought down City's Colombian winger Luis Muriel inside the box but the video review spotted a foul at the start of the action, at the other end of the field.
After going to the monitor, referee Jair Marrufo ruled that Cesar Araujo had brought down Aidan Morris in the other box and awarded the spot kick for Columbus -- with Rossi gratefully converting.
Rossi's second, in the 61st minute, was less controversial -- the forward running on to a pass from Yaw Yeboah and then beating Pedro Gallese with a delightful chip with the outside of his foot.
There was a seven-goal thriller in Toronto were the home side fought back to make it 3-3 against Cincinnati through an 85th minute Lorenzo Insigne penalty only to see Cincinnati's Sergio Santos make it 4-3 in stoppage time.
There was still time though for ex-Italy striker Insigne to find the net again but his last gasp effort was ruled out for offside.
Former USA and Premier League goalkeeper Brad Guzan was sent off for Atlanta United in their 1-0 defeat at home to Los Angeles FC.
Guzan's attempted headed clearance, outside his box, struck LAFC's Mateusz Bogusz and the 39-year-old keeper's outstretched leg brought down the Polish attacker.
After a VAR review, Guzan was shown a red card and substitute keeper Josh Cohen's first job was to pick the ball out of the net after the deflected free-kick from Bogusz beat him.
The game ended 10 against 10 after LAFC's Eduard Atuesta was dismissed after a second yellow card as the visitors hung on for a 1-0 win.
Western Conference leaders Real Salt Lake are fast becoming the MLS comeback specialists - a week after they scored three late goals to beat the Colorado Rapids 5-3, they came back from three goals down to earn a 3-3 draw at FC Dallas.
Nelson Palacio got the equalizer with a left-foot drive in the last of eight minutes of stoppage time.
Later on Saturday, Inter Miami are at the Vancouver Whitecaps without Lionel Messi, Sergio Busquets and Luis Suarez.
Miami's Argentine coach Gerardo Martino said none of the former Barcelona trio were injured but that the team felt it was wise to rest them and avoid a 12-hour return flight ahead of a midweek game against Atlanta.
Thw Whitecaps had sold out their 55,000 stadium in anticipation of Messi playing and offered fans half-price food as compensation.
