Here's how the cold start valve/thermo time switch works:

The thermo-time switch has a metal contact arm inside which is wrapped with a heating element. When it's cold, the arm contacts a ground connection. When you turn the key to start (the cold start valve ALWAYS receives power when the key is in start, and ONLY when the key is in the start position -- temperature is irrelevant as far as the cold start valve itself is concerned) 12v is sent to the cold start valve. If the thermo-time switch is cold, then the thermo-time switch grounds the cold start valve, allowing it to inject fuel. When the key is "on" the heating element in the thermo-time switch is warming up and slowly bending the contact arm, and so once it gets warm enough, it breaks contact and no longer grounds the cold start valve, shutting it off.

However, this applies only when cranking. The cold start valve never receives 12v when the engine is running, i.e. when the key is in the "on" position, only when the key is in the "start" position. So, to answer your question, no, the cold start valve is probably not staying on, unless you've got some weird wiring.

And the thermo-time switch does not supply 12v to the cold start valve, it merely supplies a ground for the cold start valve.