Does this mean I need a coolant change? Or a new thermostat already?

Thanks.

 

Sounds like you are low on coolant.

 

+1 be sure to open the bleed screw to add coolant or you will create an air pocket.

 

I'm brain-storming here:
If your coolant is full, listen for a small hissing from under the dash. At idle, the engine vacuum is low and may not be sufficient to keep a gate valve open that channels the hot air into your ducts. You may have a vacuum leak.