Gaijin Games, the developers of Bit.Trip Beat have blogged about a bug/unintentional feature in the game.
The trouble being, the game will only chooses to save progress if you get a high score. This glitch is exclusive to the US version, both the European and Japanese games were fixed before release.
If this problem is affecting you, Gaijin Games are offering two save files which you can use:
Level 1 and 2 unlocked
All levels unlocked