Temperatures below the mid 20s will freeze normal water bottles in an hour or less. If you go with insulated bottles, you may have to sacrifice capacity. Fill your bottles with the hottest water you can drink. It will keep them liquid longer.
Cold weather isn’t as big an issue when it comes to riding as is snow and ice. If the streets or sidewalks along your route aren’t clean, stay home.
Regulating your body temperature on a cold weather tour is very difficult. You will sweat. And when you sweat and you stop, you’ll get cold. Take the opportunity to dry your layers as often as needed.
Even in single digit temps, I had a synthetic T-shirt, thicker, long sleeve synthetic shirt, vest, lightweight running style jacket, and wind/waterproof shell. It was too much. By the time the weather hit double digits, I dropped my shell and still stayed warm.
Due to limited daylight hours and necessary stops to warm up and thaw water bottles, assume you’ll only accomplish 80% of your typical warm weather daily miles.
Being cold is one thing. Being tired is another. Being cold and tired can kill you. As the day wears on and you wear out, you’ll be too tired to ride fast enough to generate enough body heat to stay warm. At this point, you’re probably already sweaty. This is a dangerous time. Stop cycling. Find shelter. Get warm and dry immediately.
Plan to end you day thirty minutes before sunset. Once the sun drops below the horizon (assuming there was sun to begin with) temps drop dramatically.