Mark Martin has been retired for like 10 years, and was part time for years before hanging it up for good. He's just the one that started rigorous fitness training as part of his racing prep, and it's now the standard.
But, I'm not arguing auto racing is a sport, just arguing the claim that auto racing drivers "have zero athletic ability"
I'm on the fence on the sport department. I mean, you can be the best in the world and if your tire blows through no fault of your own, you lost the race and crashing hard most of the time, so there's a lot of things out of the driver's control(mechanical failures, aggression of other drivers, tires, pit crews, etc), but I'm just saying it's a lot more physically demanding than most realize.