Common, you know that Wacko Jacko lived in LA, he couldn't handle the extreme cold up here!

 

Military aviators are required to wear gloves at all times. The reason is that in case of (a ****pit) fire, you still have to keep your hands on the controls to get the bird down on the ground. Much less true when racing or tracking (only in that you don't have to 'fly it down'), but I'd always wear gloves. Always. Less for the grip and more for the protection. Fire is very very bad.
On the road, the danger is miniscule. And you definitely look like a dork.

 

I have two pairs of gloves. One for the track, and one for trips to the hire karts with my mates. Man the wheels on those karts get dirty and who know's what it is!!