Borrow the day's heat for the night
You can collect a little extra heat from the winter sun by placing a few bottles of water in a south-facing window. The bottles will heat up over the course of the day and slowly release that heat into the room when the sun goes down. The gently colored, translucent glass also does a lovely job of softening the light or screening an awkward view.
We put up a 3.5-inch-wide wooden shelf, using small L-brackets to attach the shelves to the sides of the window frame, and old glass bottles that we found in junk stores. Installing the shelf took about ten minutes.