Generally, a user will enter new information sometimes on the computer, and sometimes on the PDA. Syncing refers to connecting the PDA to the computer and ensuring that the newest information from both sources is now on both devices.
Syncing may also involve installing new software on the PDA or backing up software from the PDA to the computer.
Palm OS devices use a program/technology called HotSyncing, whereas PocketPCs use ActiveSync.
Most Palm OS devices come with a rechargeable battery that is not user-replaceable. You can charge it hundreds of times, maybe more, and use the device for years, assuming you aren't intentionally abusing it. However, once the battery no longer holds a charge, you must send the PDA back to the factory to get it fixed, although by then you'll probably want a new PDA anyway. Most PocketPCs have user-replaceable batteries, which is primarily seen as an advantage, except that it does increase the price of the PDA somewhat, and by the time your battery stops holding a charge, it might become difficult to find that model battery.