How would you set up a decent backup setup?

victor2

Was it Microsoft Office that originally came on about twenty 5.25" floppies? The first unreadable one would be number 19. Happy days!  

TELLIT01

I hope it's OK to ask a variation on the question here rather than starting a new topic. 

All I want to do is find a way of updating the backup of my photo's on external hard drive.  Ideally just plug in the hard drive and then have a program to search for any images more recent than the last backup date.  Current backup is approx 160 Gb so I don't want to do a full backup every time.  Any suggestions appreciated. 

IvanOpinion

I used to use MS SyncToy for just that, but it is no longer available. There may be other equivalents to SyncToy available but I have never used them. Alternatively you could look at Xcopy or robocopy.

victor2

TELLIT01 said:

I hope it's OK to ask a variation on the question here rather than starting a new topic. 

All I want to do is find a way of updating the backup of my photo's on external hard drive.  Ideally just plug in the hard drive and then have a program to search for any images more recent than the last backup date.  Current backup is approx 160 Gb so I don't want to do a full backup every time.  Any suggestions appreciated. 

I used to use Cobian Backup. It's free but hasn't been updated since 2012, although you can still get it and documentation for it. As far as I remember it copies files either as they are or compresses them into a zip file.

I stopped using it in favour of Macrium because I had problems with scheduling it to wake up my PC in the night and run the backup to a NAS drive.

Sounds like it might meet your needs though, as you would only run it on demand.

gefnew

You could use file history in win 10 to do this.
How to use Windows 10 File History to back up data | Windows Central

TELLIT01

Thanks for the suggestion of File History.  On first glance I don't think it will do what I'm after, which is basically a differential backup, but I'll investigate further. 

[Deleted User]

TELLIT01 said:

Thanks for the suggestion of File History.  On first glance I don't think it will do what I'm after, which is basically a differential backup, but I'll investigate further. 

I don't think a differential backup is what you need - the amount of space it needs grows dramatically. Differentials are safer and quicker to restore from than incremental backups but will soon use an excessive amount of space.

Differential backup example:

1. You backup your photo collection of 1000 files on Sunday and this takes 10GB
2. On Monday, you have changed 100 of the existing files and added 100 new ones - 200 files are backed up taking 2GB on top of your original 100GB
3. On Tuesday you don't change or add any files, but the differential backup will still be 200 files 2GB as that is the difference since the last full backup
4. and so on - by day 6 your differential are 12GB + original 10GB backup so now 22GB in total
5. You complete another full backup on day 7 the following Sunday which will be 1100 files of 11GB. So now you have 33GB of backup and then the cycle starts again from step 2 above.

So you think ah well, after a few weeks I'll only keep my most recent backups to save space, but the problem is, you may have deleted a file 6 months ago without realising. That is in your 6 month old full backup only and not any of the newer differential or full ones so it is lost forever.

Incremental is probably what you need, and that is what Windows file history is, and it works like this:

1. Same as above, you backup you 1000 files on Sunday and it takes 10GB
2. Monday you change 100 of the 1000 and add 100 new ones - it will keep an old and new version of the 100 edited files and a copy of the 100 new ones - backup is now 12GB
3. Tuesday you don't change anything - nothing gets added because unlike differential which compares to the full backup, incremental compares files against the last incremental - so if you don't change anything, the backup doesn't grow.
4. Because the daily backups aren't growing in size like differential, you don't really need to start the cycle of full backup again - just continue with incremental forever.

You can tell Windows file history how long you want to keep incremental versions of the file - default is forever. In the event of realising you deleted a file 6 month ago it will be there.

Having said that, Windows File history isn't the best, it doesn't have an integrity check. Macrium or Veeam Community edition are better examples.

[Deleted User]

victor2 said:

That is set to do a full backup on the first of every month and differential backups daily. I hold 6 month's cycles with ease and it automatically deletes older ones than that. I have less than 100GB of data, including photographs.

What happens if you accidentally deleted a file 7 months ago without realising until today? Basically it is lost forever. You won't have a copy of that if you are purging older than 6 month cycles.

Or if a file gets edited 7 months ago and becomes corrupt without you realising - all your subsequent backups are of the corrupt file and you won't have the original good file from 7 months ago.

Differentials are safer in the sense that if one fails, the next one will have the files that changed the previous day unlike and incremental, however modern backup software will allow integrity checks of the backups and mop up any failed ones by just using version checks for each file and keep adding any changed files to the backup anyway.

The old differential / incremental with occasional full backups is a bit old hat now anyway, just simple "version history" taken care of by the backup software is preferable.

Because incremental backups are faster, they are easier to run more frequently - eg every 15 mins. Add in an integrity check as well and they become as safe as full / differentials.

victor2

[Deleted User] said:

victor2 said:

That is set to do a full backup on the first of every month and differential backups daily. I hold 6 month's cycles with ease and it automatically deletes older ones than that. I have less than 100GB of data, including photographs.

What happens if you accidentally deleted a file 7 months ago without realising until today? Basically it is lost forever. You won't have a copy of that if you are purging older than 6 month cycles.

.....

Agreed, I've only got 6 months to realise I've messed something up,  But I do have indefinite backups of my essential files with versioning in the cloud. Photographs are also in Google Drive and only synchronised manually when I need to update them.

JustAnotherSaver

[Deleted User] said:

The vital missing bit of information is how much data you want to back up - solutions differ depending on the amount.

For example but not hard and fast rules, I would consider:
100GB probably just cloud like one drive / google drive
1TB external USB drive
10TB NAS

Ideally you should backup both local and offsite for maximum resilience for precious files. I can advise more once you state how much you want to backup, but in the meantime I'll bore you with how I do mine:

1. PC - I store documents only - these backup using Windows File history automatically to my NAS every 15 mins in the background. 

2. NAS - 16TB QNAP - This contains above backup plus a photos / home videos / music  - about 2TB and around 9TB of movies and TV shows for my Plex server

3. 12 TB Seagate USB Backup drive connected to my NAS - backup of everything on the NAS (NAS is RAID5 so only 12TB of 16TB usable) auto backup daily with versioning.

4. NAS does automated job to Amazon Glacier to backup my photos / home videos / music (2TB=£3 a month). All my movies / TV shows can easily be re-downloaded via Sonarr / Radarr if needed so I don't do a cloud backup - 2 local copies enough.

5. 4 TB WD NAS - extra drive for TV shows and movies that I've watched and I'll probably delete - bit like a recycle bin - no backup.

Plex server = NVidia Shield Pro + 500GB SSD for database / thumbs etc

Raspberry PI + 500GB SSD = Radarr / Sonarr / Overseerr / Tautulli / Jackett / NZBGet / Qbittorrent 

All the automation software on the PI is exposed to the web via an Nginx reverse proxy running on my QNAP in a docker for security. Have my own domains with SSL and Plex SSO for external users to use Overseerr to request TV shows / movies.

Been a little bit away for a couple days but yes, as I was reading through the initial responses had me thinking of having to address space.

Put it this way ...

I have an 8TB external HDD which is full of movies.

I have a 6TB internal drive which is mostly full due to movies

I have another 4TB internal drive which has a mixture of movies and music.

Some 2TB drives with various files on, not necessarily movies

As well as another 4TB external drive with more files on

The above movies are in blu ray format or rather were & were then ripped using Makemkv and are now in .mkv format.

Oh and I have a further 100+ DVDs that are not yet ripped which I would like to.

And god knows how many music CDs, 200+ that I would also like to rip for a digital library & would like these in some kind of lossless format.

I've accepted that I may need to condense (I'm sure that's not the right term but you'll know what I'll mean) the blu-ray rips but my PC is 12 years old now and if I do 1 film then it's a set it up for through the night kind of job.

Additional issues ...

1) I don't know what settings to use. I don't particularly want to lose too much quality

2) When ripping the blu ray discs, anything that said English audio/subtitles, I just included as part of the rip as I didn't know which was 'the' one I needed.

That means for many of the movie files, there's x-amount of audio/subtitle files that I don't need ..... I just don't know which ones.

But then that's turning this in to a bit of a Plex thread & it's supposed to be a backing up thread.

So all in all, that's a long winded way of saying while I wont have as much (capacity) to back up as you, I'll still have more than what Joe Bloggs will be wanting to back up. We're not talking about 1TB, 2TB passport external drives sorting the situation here.