If your taste varies, you'll want a large-capacity MP3 player (32GB or more) to fit your music. If there's only a few bands/artists you enjoy, save some money with a 4GB or 8GB player.
Ensure you headphones suit your needs. For instance, during a workout you'll require stationary earbuds; watching videos? Large noise-cancelling headphones may provide better cinema-like audio.
Using MP3 players to watch videos has risen in popularity. If that's your want, ensure the screen is big enough for finer details. If not watching videos, then screen size should be an afterthought.
Many MP3 players these days have an integrated FM radio which is handy if you're bored of your current selection or if you want to hear the leatest weather, traffic or news updates while on the go.