I like my black leather seats. I find no problems with them, and mine's a DD in Central Coast of California, where its sunny 99% of the time (obviously excluding night).
(Silver, black top, black leather seats GT)
All you need is to use a Windshield reflector thingy and you are good.
So in that retrospect, if it ain't broke don't fix it?
Obviously i'm partial to the black seats.
I'd say drive it for about a month and see if you still want to do it.