[quote=@Amaranthine]
So they're driving around in their D-Wheel, on a racetrack, yes?[/quote]

Feels like I can answer it all by just clearing this up.

It doesn't [i]have[/i] to be a race track. At the beginning, Yusei and Ushio rode through the entirety of Satellite and even a pipeline. Since Yusei was on the run from police, and said police had a way to force him into a duel (imposed activation of Speed World), Ushio could keep Yusei in the pace he wanted by blocking out the speed counter generation. This was so Yusei couldn't reach the end of the pipeline for an easier arrest.

Outside of the situations where it was uses as a way for police to temporarily have a criminal on lockdown, Speed World has also been used in abandoned streets, on highways and even through a goddamn security building. The writers themselves knew it was a bit silly for speed to be determined on a racetrack, so instead of a speed limiter, they introduced a system where your speed counters increased by 1 for each lap you make while the other's in their pit-stop (which is there because the arc in question had a 3-man relay race). If your speed counters hit 12, the opposing speed counters start counting down. If their speed counters are at 0 and you finish a lap, they automatically lost.

So yeah, they got pretty creative with how speed counters were used in 5D's. They're also just cool and give an actual reason for being on a D-Wheel, flimsy as it may be.