Why does Lightroom 4 create a new file every time you want to back up? This will soon clog up my hard drive at this rate. Why doesn't it simply overwrite the previous save ?

Are you backing up your catalog to your main computer's HD, the same one that has LR?  If so, I wouldn't cuz if your HD fails you've lost everything.  I back up my catalog to a couple of different external hard drives and delete old backups every once in awhile.  Externals are relatively cheap nowadays and if your main HD fails you have an insurance policy in your externals.

And another thought - exclude your main LR catalog from any system backups. Just backup your backup. And your images.

The reason for this is that there are reports of incremental backups trashing the LR catalog if it is open when the backup software is running and the backup software accesses the catalog.

I've used Time Machine and SuperDuper (on a Mac) and exclusions are easy to set up.