There are quite a few, and they're reviewed on this site. Most all of the current (and older) Sony Clies have an MP3 player such as the TJ37, TH55 and UX40/50. The Palm Zire 31, 71, 72 and Tungsten T models as well as the E have an MP3 player.
The Tungsten T3, Sony Clie UX4, UX50 and TH55 have a voice recorder too.
None have FM tuners. However, if you get a device with an SD slot, you can but an SD FM radio card for around $50.