I do not mind a little frosty. As long as they fill out. 2 of my molds give the best fill out when they drop with a little frost on them.
If it looks like an orange peel then it way too hot. Frost is ok. Rough to the touch is not.
Oh and I started getting way fewer rejects when I started using a thermometer. It was money well spend. (This comes from a poor working man, that is tight with a dollar as well.)
No matter how many times you do it and nothing happens it only takes something going wrong one time to kill you.