IPB

Welcome Guest ( Log In | Register )

2 Pages V   1 2 >  
Reply to this topicStart new topic
The best way to backup lossless audio data?, ...want to avoid re-ripping and re-tagging if hard drive fails...
jeremija
post Jan 25 2010, 09:04
Post #1





Group: Members
Posts: 328
Joined: 27-September 08
From: Zagreb
Member No.: 58696



I have about 50 gb of lossless audio that I want to backup from HDD. If my hard drive fails, all of my .flac files will be lost and re-ripping and re-tagging are both time-consuming processes which I would like to avoid.

So, which would be the best way to backup:
  • on multiple dvd discs
  • on one or two dual layer blu ray discs
  • on an external hard drive which would serves only for backup
  • something else?

The main pluses are that it lasts long (ie. I would want the medium not to self-destruct itself in 10 years due to the aging process) and that it's not too expensive.

The way I see it - I should iether buy a blu-ray recorder (which costs about $200 where I live) plus a few blu-ray discs, or buy a new external hard disk (around $80). But I don't know will blu-ray disc become unreadable in a few years, or will hard disk stop working... So what do you think, which lasts longer?
Go to the top of the page
+Quote Post
2E7AH
post Jan 25 2010, 09:40
Post #2





Group: Validating
Posts: 2424
Joined: 21-May 08
Member No.: 53675



My approach is backup on DVD with 1 level of correction files (stored also on DVD)
Keep them in vertical position
Something like annual check in this case is recommended
Go to the top of the page
+Quote Post
jeremija
post Jan 25 2010, 09:55
Post #3





Group: Members
Posts: 328
Joined: 27-September 08
From: Zagreb
Member No.: 58696



what exactly are the correction files and how to generate them?

This post has been edited by jeremija: Jan 25 2010, 09:57
Go to the top of the page
+Quote Post
2E7AH
post Jan 25 2010, 10:00
Post #4





Group: Validating
Posts: 2424
Joined: 21-May 08
Member No.: 53675



http://en.wikipedia.org/wiki/Dvdisaster
Go to the top of the page
+Quote Post
jeremija
post Jan 25 2010, 10:09
Post #5





Group: Members
Posts: 328
Joined: 27-September 08
From: Zagreb
Member No.: 58696



thank you!
Go to the top of the page
+Quote Post
flacflac
post Jan 25 2010, 10:53
Post #6





Group: Members
Posts: 56
Joined: 3-December 07
Member No.: 49229



Hi Jeremija,

While I also endorse the use of optical disc backups I also strongly recommend the following:

1) Get a second harddrive and make daily to weekly backups of your main drive - once your collection size increases you'll get dizzy recovering data from 20-50 DVDs, perhaps only to find out that one of them has disc rot and ends up useless.

2) Instead of using a rather proprietary tool such as DVDisaster I would recommend using Par2 Parity-files, which also use Reed Solomon algorithms to generate redundancy data. At 15-20% this can be read by many tools and recovered quickly, not only from DVDs but also if you have a bad sector on your harddrive that damages an audio file.

Easiest interface can be found at http://www.quickpar.org.uk/ .

ff smile.gif
Go to the top of the page
+Quote Post
Ardax
post Jan 25 2010, 12:39
Post #7





Group: Members
Posts: 233
Joined: 3-December 01
Member No.: 578



Par2 is just as "proprietary" as dvdisaster. Both use Reed-Solomon error correction, they just apply it differently. For optical media, I'd go with augmented images from dvdisaster.

But I'd rather use an external HD for backup purposes, simply because it's going to be a LOT faster for backup, verification, and restoring than optical media. What's more, you avoid having to deal with swapping discs. Connect it only when doing backups. Use a combination of Par2 (I like MultiPar) and hashes for verifying and recovering any bit errors. MD5 or SHA1 hashes (HashCheck are much faster to generate and verify than Par2 files, IME.

This post has been edited by Ardax: Jan 25 2010, 12:41
Go to the top of the page
+Quote Post
jeremija
post Jan 25 2010, 12:46
Post #8





Group: Members
Posts: 328
Joined: 27-September 08
From: Zagreb
Member No.: 58696



Thank you all for your replies. But from the medium's lifetime point of view, which has a longer lifetime (if treated properly): hard disk or DVD disc?
Go to the top of the page
+Quote Post
2Bdecided
post Jan 25 2010, 13:29
Post #9


ReplayGain developer


Group: Developer
Posts: 5138
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



How long is a piece of string?

Which are better - cats or dogs?


You're asking an impossible question, but you'll still get 10 pages of replies - all of which have been given many times before!

Cheers,
David.
Go to the top of the page
+Quote Post
GHammer
post Jan 25 2010, 15:19
Post #10





Group: Members
Posts: 224
Joined: 11-May 03
From: China
Member No.: 6546



QUOTE (flacflac @ Jan 25 2010, 04:53) *
1) Get a second harddrive and make daily to weekly backups of your main drive - once your collection size increases you'll get dizzy recovering data from 20-50 DVDs, perhaps only to find out that one of them has disc rot and ends up useless.

2) Instead of using a rather proprietary tool such as DVDisaster I would recommend using Par2 Parity-files, which also use Reed Solomon algorithms to generate redundancy data. At 15-20% this can be read by many tools and recovered quickly, not only from DVDs but also if you have a bad sector on your harddrive that damages an audio file.



DVDisaster has one advantage, it reads damaged CD/DVD.
With a PAR solution, you'll have to get something to read the damaged media.

For files on hard drives, I just use WinRAR with a recovery record.
Go to the top of the page
+Quote Post
Teknojnky
post Jan 25 2010, 15:21
Post #11





Group: Members
Posts: 335
Joined: 12-April 06
Member No.: 29453



50 gigs is a pretty small amount, you can easily and quickly keep 1 or more backup copies on external drives very cheaply and much easier than any amount of optical storage.

The primary thing to rememeber is, if you only have 1 instance of your data and that media fails or is damaged, you have no recourse. The safest data is backed up to multiple physical copies both locally and to a remote location (ie to protect from theft/fire/etc).


--------------------
Last.FM Nodes for your library @ http://build.last.fm/item/356
Go to the top of the page
+Quote Post
Fandango
post Jan 25 2010, 16:37
Post #12





Group: Members
Posts: 1548
Joined: 13-August 03
Member No.: 8353



I would rather use DVDisaster because it has all in one application. ISO augmentation, CD/DVD ripping without halt on errors, etc.

And therefore, because of its ease of use, I wouldn't use RS01 error data, but RS02. Because where do you want to store the ECC files? What if the medium you store the ECC files on fails, too? RS02 can deal with ECC data corruption as well, besides it's so much more convenient to use. The only drawback is that you cannot use a DVDs full capacity, which never was a big deal for me.

Regarding PAR2. The only GUI tool for Windows Quickpar has no multi-threading support and it also does not support sub-folders, AFAIK. No support for sub-folder is a show stopper in itself for music backups unless you have all your music rips in one single folder. Or you want to create a PAR2 file set for each rip.

But there's a MT-hacked version of the command line tool par2.exe (http://www.chuchusoft.com/par2_tbb/). I strongly recommend this one instead of Quickpar, since it's faster and supports sub-folders. In case you want to use PAR2 instead of DVDisaster...

So in case of DVD backup, I'd recommend DVDisaster. And in case of HD backup, I'd recommend the multithreaded version of par2cmdline.

And I would do both. Backup to another HD (thanks to eSATA, that's as easy as it can get), and backup to DVDs.
Go to the top of the page
+Quote Post
Ardax
post Jan 25 2010, 20:17
Post #13





Group: Members
Posts: 233
Joined: 3-December 01
Member No.: 578



QUOTE
Regarding PAR2. The only GUI tool for Windows Quickpar has no multi-threading support and it also does not support sub-folders, AFAIK.

See the MultiPar tool that I linked to in my previous post. I /think/ it does multi-threading as well, but it definitely handles subfolders. And it has a GUI, for those that prefer them.

Thanks for the pointer to a CLI par2 program that handles subfolders though.
Go to the top of the page
+Quote Post
Dirge
post Aug 9 2010, 23:53
Post #14





Group: Members
Posts: 14
Joined: 27-July 04
Member No.: 15811



I got to thinking it would be best, at least for my needs, to add PAR2 files to each album and include a checksum file. I would then back up to two separate external drives for redundancy.

I spotted a link to the HashCheck Shell Extension in this thread and got to wondering if I should generate .md5 or SFV files for each album. Are these file formats standardised or does each file verification app follow a different standard?
Go to the top of the page
+Quote Post
SCOTU
post Aug 10 2010, 08:31
Post #15





Group: Members
Posts: 118
Joined: 9-July 10
Member No.: 82156



Assuming you have a spare place for a hard drive and a DVD burner, the DVDs are definitely the cheaper option. However, the HDD offers the benefit of allowing you to much quickerly back up and restore things, less space, less hassle, and easier to identify if your backups are broken. If your data is on a Data drive (not the OS' hard drive) you can get another one of the same (internal) disks and (assuming you have the 50GB to back it up temporarily somewhere) make a RAID 1 set with your current drive and the new one (if your motherboard supports RAID 1). This would give you automatic backup that takes 0 time plus the other benefits of the hard drive.
Go to the top of the page
+Quote Post
odyssey
post Aug 10 2010, 09:21
Post #16





Group: Members
Posts: 2296
Joined: 18-May 03
From: Denmark
Member No.: 6695



Personally I've given up on the optical medias. They degrade rather quickly - Some faster than others though. I have much more faith in a secondary harddrive:

I have bought a NAS-solution which I'm locating somewhere else and use rsync to keep it updated as I extend my library. No extra work, no hassle, always updated!


--------------------
Can't wait for a HD-AAC encoder :P
Go to the top of the page
+Quote Post
viktor
post Aug 10 2010, 12:07
Post #17





Group: Members
Posts: 297
Joined: 17-November 06
Member No.: 37682



DVDs are much less reliable than HDDs, not to mention their worse price/capacity ratio. and of course you'll have to switch between disks all the time, while on HDD everything's in one place. and HDDs are faster than DVDs.

bluray is simply not an option, disks are way too expensive, writers are of bad quality, hard to find, and even more expensive.

i'd strongly encourage you to use 2 HDDs, one for storage, one for backup. you can put the backup disk in a mobile rack and make a sync weekly. for syncing rsync (linux) is the best option imho, i use its windows port called deltacopy with an hdd in a mobile rack.

or if you're really maximalist, you can set up a NAS with raid1 and 802.11n, for example.

This post has been edited by viktor: Aug 10 2010, 12:09
Go to the top of the page
+Quote Post
odyssey
post Aug 10 2010, 12:10
Post #18





Group: Members
Posts: 2296
Joined: 18-May 03
From: Denmark
Member No.: 6695



QUOTE (odyssey @ Aug 10 2010, 09:21) *
I have bought a NAS-solution...

It might seem overkill, but consider your house/apartment burned down to ground or something like that. Backup in a remote location is always a good thing smile.gif


--------------------
Can't wait for a HD-AAC encoder :P
Go to the top of the page
+Quote Post
SCOTU
post Aug 10 2010, 15:54
Post #19





Group: Members
Posts: 118
Joined: 9-July 10
Member No.: 82156



QUOTE (viktor @ Aug 10 2010, 07:07) *
DVDs are much less reliable than HDDs, not to mention their worse price/capacity ratio.


DVDs are actually the cheapest modern (idk how much tape costs) storage medium in terms of cents/GB. a 100pack of DVDs nets you 437GB for as low as $16. That's 3.7 cents/GB. Cheapest hard drive price ratio I could find was 1TB (actual 931GB) for $60, or 6.4 cets/GB, almost double the price of the DVDs.


QUOTE (viktor @ Aug 10 2010, 07:07) *
bluray is simply not an option, disks are way too expensive, writers are of bad quality, hard to find, and even more expensive.


This is where I'll have to revise my previous post. The cheapest solution for you may actually be blu ray (assuming you already have a blu ray writer, that is. For 50GB, you'd need 2 BDs @ $3 ea for a total of $6. For DVDs, you'd need a very inconvenient 11, which 15 Packs are sold @ $8.

QUOTE (viktor @ Aug 10 2010, 07:07) *
or if you're really maximalist, you can set up a NAS with raid1 and 802.11n, for example.


For backing up data, I would highly recommend gigabit Ethernet over any sort of wireless. Even using 802.11n and having a good connection will average 25.5 MB/s at best. Using gigabit ethernet, however, you may actually be limited by the write speed of your hard drive, with speeds that can be up to 100 MB/s average.
Go to the top of the page
+Quote Post
Akkurat
post Aug 10 2010, 16:43
Post #20


REACT Mod developer


Group: Developer
Posts: 929
Joined: 14-November 07
From: Finland
Member No.: 48750



RAID is not for backupping (a main storage), not any RAID. RAID is meant to increase data availability/reliability and/or I/O performance. Versioning is important in backupping, RAID doesn't offer that. RAID doesn't help if e.g. file system goes crazy, program/OS/user deletes a file, failing PSU/controller destroys both drives. I.e. Wikipedia - RAID - backups. EDIT: If one has a main storage and a separate backup of that, I personally don't see why using RAID in neither one would be beneficial (from backup POV).

I guess having an external off-site HD for backup would be ideal (for backupping purposes). Personally I've been _thinking_ about setting up a following system: main storage of big/"media" files, daily/constant versioning backup of "work files" (or all files) from my desktop PC, plus weekly/monthly incremental HD images in NAS. On-site sync of that to USB HD (easier than off-site) & an online (internet, encrypted) backup of "critical" files. Wow, that's a rather complicated looking plan now when written down.. backupping is tedious. sad.gif

This post has been edited by Akkurat: Aug 10 2010, 16:49
Go to the top of the page
+Quote Post
shadowking
post Aug 10 2010, 17:40
Post #21





Group: Members
Posts: 1527
Joined: 31-January 04
Member No.: 11664



Keep it simple (KISS):

Its a backup like any other. Don't worry about bit rot or corruption too much, Avoid RAID / DVD . Just backup on a spare drive . Having a yearly snapshot on an additional drive can provide extra security. The drives can even be 80gb models from used PC's - the chances of 2 ~ 3 drives going bad at once is next to zero.

In short just backup your windows profile that should contain your flacs or in unix type OS your /home


--------------------
Wavpack -b450s0.7
Go to the top of the page
+Quote Post
krabapple
post Aug 10 2010, 17:49
Post #22





Group: Members
Posts: 2274
Joined: 18-December 03
Member No.: 10538



QUOTE (shadowking @ Aug 10 2010, 12:40) *
Keep it simple (KISS):

Its a backup like any other. Don't worry about bit rot or corruption too much, Avoid RAID / DVD . Just backup on a spare drive . Having a yearly snapshot on an additional drive can provide extra security. The drives can even be 80gb models from used PC's - the chances of 2 ~ 3 drives going bad at once is next to zero.

In short just backup your windows profile that should contain your flacs or in unix type OS your /home



And keep one drive offsite in case there's a catastrophe at home.
Go to the top of the page
+Quote Post
SCOTU
post Aug 10 2010, 22:16
Post #23





Group: Members
Posts: 118
Joined: 9-July 10
Member No.: 82156



If you're backing data up remotely over the internet, make sure to keep a close eye on how much you're backing up. Some ISPs will deny you service if you use too much bandwidth. This can happen extremely easily if you're not paying attention to how much you back up in a month.
Go to the top of the page
+Quote Post
Akkurat
post Aug 10 2010, 23:46
Post #24


REACT Mod developer


Group: Developer
Posts: 929
Joined: 14-November 07
From: Finland
Member No.: 48750



QUOTE (SCOTU @ Aug 11 2010, 00:16) *
If you're backing data up remotely over the internet, make sure to keep a close eye on how much you're backing up. Some ISPs will deny you service if you use too much bandwidth.

That is a good point/advice.. for those unfortunate who live in a country where ISPs do that.
Go to the top of the page
+Quote Post
Slipstreem
post Aug 11 2010, 00:26
Post #25





Group: Members
Posts: 966
Joined: 7-July 06
Member No.: 32660



QUOTE (odyssey @ Aug 10 2010, 09:21) *
Personally I've given up on the optical medias. They degrade rather quickly - Some faster than others though. I have much more faith in a secondary harddrive...

I'd go for both decent optical media and a backup hard drive. Having lost 8 out of 10 hard drives over the past 18 months for no obviously apparent reason, and only 1 correctly stored DVD-R out of over 100 in the past decade, I wouldn't put my faith entirely in hard drives without another form of backup. YMMVC. tongue.gif

Lose a DVD and you've only lost just over 4GB of data. Lose a full 2TB hard drive and you've lost 2TB of data. The words "eggs" and "baskets" spring to mind. wink.gif

This post has been edited by Slipstreem: Aug 11 2010, 00:31
Go to the top of the page
+Quote Post

2 Pages V   1 2 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 19th September 2014 - 16:51