PDAs with user replaceable batteries have a backup battery that preserves data when you swap out the main battery. As long as you don't take more than 5 minutes to remove the old battery and put in the new one, you should be fine.

.NET CF is built into Windows Mobile 2003 OS versions already