IPB

Welcome Guest ( Log In | Register )

Help ripping ~30,000 CDs, Was “Help digitizing […]” ;)
trail
post Mar 29 2012, 19:08
Post #1





Group: Members
Posts: 9
Joined: 29-March 12
Member No.: 98189



Hey! I'm a newbie around these forums, but hopefully I'll be around quite a bit. I really like the community here, and hope I can contribute in the future. But enough introduction, here's the interesting stuff:

I work for a college radio station, and we've decided to undertake the rather ambitious project of digitizing the CDs we've acquired over the years. This is a pretty monumental undertaking, so I'm looking to make this as painless and quick as possible. We have a very rough approximation of about 30,000 CDs that we're looking to convert to digital files, and it's my job to work out many of the more technical aspects of the project.

The problem with being a college radio station is that we're on a pretty limited budget. We can't afford any sort of robot or anything like that to help the process along, nor can we afford any sort of service, so we're stuck doing it ourselves. Thankfully, we have a bunch of people willing to put the time and effort in. We also aren't terribly picky about getting every rip totally 100% perfect. But I've done a fair bit of research, and here's the kind of plan I had in mind:

Ideally, we have one pretty decent quad-core desktop that we're planning to outfit with four CD drives. We have software that allows us to rip multiple discs at once to V0 MP3s which are stored on a small RAID 1 array inside the computer. I've done some informal ripping tests, and have narrowed down the two pieces of software that seem to work best to fre:ac and dBpoweramp. I have also tried EAC and simply ripping with MediaMonkey, but freac and dBpoweramp seemed the most efficient and easy to use. Now, if I decide to use one of these pieces of software (if anyone has any suggestions, I'm 100% open to them!) how can I configure them to make them as painless as possible? Would using multiple drives be an option? I found very little information about software that provided ripping from multiple drives simultaneously, so I'm assuming this is not a common feature. If not, would using different computers be our best bet? If anyone has any other suggestions about ripping multiple discs at the same time or other ways to improve efficiency then that would probably make my life much easier.

thanks for your time!
Go to the top of the page
+Quote Post
 
Start new topic
Replies
trail
post Mar 30 2012, 17:44
Post #2





Group: Members
Posts: 9
Joined: 29-March 12
Member No.: 98189



First off, I want to say thanks so much to everyone replying to this thread so far. It's been incredibly helpful.



QUOTE (dumdidum @ Mar 30 2012, 03:56) *
i second the suggestion of ripping to a lossless format. first, storage is cheap. second, generation loss could be an issue. after all, many if not most radio stations broadcast their show in a lossy format (internet radio, digital audio broadcasting, etc.).

This is a good point, we do broadcast online. I think that this is such a huge undertaking and going lossless will make it even huger, but I'm being convinced more and more that going lossless is worth the effort. Then it's just the issue of "how do we back up and make 10TB of data network accessible on the budget of a college radio station?"


QUOTE (Porcus @ Mar 30 2012, 05:07) *
I ripped about 7000 CDs to FLAC using dBpoweramp, a Sony XL1B2 200-disc mediachanger (well actually two, luckily since one wore out ... 2nd hand ones available for cheap at Amazon: http://www.amazon.com/gp/offer-listing/B00...;condition=used ).

You can probably use dBpoweramp's Batch Ripper. What I did -- this was at a time Batch Ripper was fresh and a bit immature -- was to hack together an AutoIT3 script that automated dBpoweramp. I did once post it at http://www.avsforum.com/avs-vb/showthread....86#post13939586 , but don't hold it against me, it is fairly lame coding. (And forget whatever I wrote there about HDCD. I regret using the HDCD DSP.)

I know that people have modified REACT to work with the mediachanger too.

I appreciate the links, and dBoink is a pretty awesome name. If at one point we do decide to get a dedicated ripper, the XL1B will be at the top of the list, thank you! Does Sony have any current version of this that they're selling? And how did you go about storing 7000 CDs of FLAC files?

QUOTE (LosMintos @ Mar 30 2012, 09:28) *
Sorry, I didn't read all posts carefully, nevertheless, let me add/emphasize some points.
  • You should insist on accurate, secure and lossless rips! You'll only do it once, do it right!
  • I've no experience with real batch rippers, but ripped a lot of CDs on ordinary PCs equipped with two CD-Rom drives. You will just open two instances of dbpoweramp or EAC and it's just fine. However, this is limited in terms of keeping a clear view on open CD cases on the desk and open program instances on the screen. With four drives you'll not gain much increase in over all speed, IMHO. The computer will wait for you (rather than you waiting for the computer).

This is something I didn't consider. I did some more tests last night, and it looks like three drives is the sweet spot. I also think I'll stick with dBpoweramp's batch ripper though.

QUOTE (LosMintos @ Mar 30 2012, 09:28) *
  • Metadata is a crucial issue. Databases are not correct anyway and every n-th CD will not be found. Then you'll have to enter the data by yourself, the most time consuming step.
  • If you can't effort a batch ripper, you probably can distribute the job to many volunteers. In advance you have to agree on a standard:
    • codec
    • folder hierarchy
    • tagging scheme incl. cover images
    • what to do with unknown or erroneous CDs
    When ripping manually I'll pre-sort a bunch of CDs: regular albums, sampler, soundtracks etc. This reflects my folder hierarchy and speeds up ripping in my case. You could consider things like this, when distributing to volunteers.

    I think I'll deal with bad or missing metadata by marking CDs that dB couldn't rip automatically and revisiting later to manually type the data in, but the folders for different types of music is an excellent idea. We get lots of promo material and compilation albums so it's probably a good idea to separate the music into broad categories.

    Here's the example folder hierarchy I was thinking of:
    \Library\Category\Artist\Album (ID number we add when we get it)\Track number. Artist - Title

    or for a real-life example:
    Library\Full Albums\Modeselektor\Monkeytown (30458)\04. Modeselektor - Evil Twin.flac

    QUOTE (LosMintos @ Mar 30 2012, 09:28) *
    I see, you're not likely to give CDs away (I fully understand!). That way, a large room with many computers equipped with max. 2 drives each will help more than few computers with a lot of drives each. Just my humble opinion ;-). Such a set up will likely be outperformed by a real batch ripper. In addition a network storage might be interesting for you. And a scanner for missing cover art.
    Just my thoughts, hopefully of some help for you :-)

    That's probably a great point. We have some older computers that may allow us to rip while keeping everything standardized, that's something I'll look in to. Coming up with a way to store the files efficiently and as cheaply as possible is another concern, too. But thankfully, cover art isn't terribly important, so we probably won't spend too much time on that.

    It would be nice to digitize at a rate of like 100CDs/hour, but that's likely unattainable without multiple people working at once.

    QUOTE (Porcus @ Mar 30 2012, 11:38) *
    QUOTE (LosMintos @ Mar 30 2012, 15:28) *
    You will just open two instances of dbpoweramp or EAC and it's just fine.


    Be careful. My experience with dBpoweramp is that it might from time to time switch to the most-recently-used drive. Probably not without telling me, but I have overlooked it (and gotten a few rips with absolutely wrong content). I don't think it is intended to have concurrent versions open.

    QUOTE (LosMintos @ Mar 30 2012, 15:28) *
    pre-sort a bunch of CDs: regular albums, sampler, soundtracks etc.


    Also:
    - remasters, if you want to have them distinguished. The metadata sources do not.
    - promos. Some of them have beeb sounds and talking interfering with the music.
    - I keep classical music away from the rest -- or rather: music sorted by composer, apart from music sorted by performer.

    It would be nice to be able to be incredibly specific about these things (classical music and promo material) I'm only here three more years! laugh.gif I think that we might overlook some of the more specific subfolder ordering in the interest of time. We'll be putting the entire library into MediaMonkey, too, so it'll be organized in that way.

    QUOTE (pdq @ Mar 30 2012, 11:40) *
    On the cost of storing lossless files - consider the tens if not hundreds of thousands of dollars that those 30,000 CDs cost originally. Storage space for FLAC runs about 5 cents per CD.


    That certainly puts it into perspective, yeah. Hard drives are pretty cheap in the long run.

    It just seems like the biggest hurdle now is to store and back up all these terabytes of data we're going to create by going lossless and still make them accessible to the other computers on our local network.
    Go to the top of the page
    +Quote Post
    Porcus
    post Mar 30 2012, 20:23
    Post #3





    Group: Members
    Posts: 1849
    Joined: 30-November 06
    Member No.: 38207



    QUOTE (trail @ Mar 30 2012, 18:44) *
    If at one point we do decide to get a dedicated ripper, the XL1B will be at the top of the list, thank you! Does Sony have any current version of this that they're selling? And how did you go about storing 7000 CDs of FLAC files?


    They discontinued the XL1B (and dumped the prices gradually down to $84 for the last ones -- compare that to an original price of $799, which was itself half the price of the Powerfile it was based on!), and I think they replaced it with a BluRay changer. Which does not interest me, so I haven't paid attention since.

    7000 CDs in FLAC, that fits on a single 3TB hard drive. (Plus backup and offsite backup.) 30 000 CDs should then fit on five. There are even cheap consumer-grade motherboards with 6 SATA connections.


    --------------------
    One day in the Year of the Fox came a time remembered well
    Go to the top of the page
    +Quote Post
    trail
    post Apr 2 2012, 01:42
    Post #4





    Group: Members
    Posts: 9
    Joined: 29-March 12
    Member No.: 98189



    Bad news, our budget means we probably won't be able to afford the equipment to go full FLAC. sad.gif We'll probably stick with V0.

    But for our RAID array I picked out this enclosure with four of these in RAID 5
    Go to the top of the page
    +Quote Post
    phofman
    post Apr 2 2012, 11:05
    Post #5





    Group: Members
    Posts: 309
    Joined: 14-February 12
    Member No.: 97162



    QUOTE (trail @ Apr 2 2012, 01:42) *
    Bad news, our budget means we probably won't be able to afford the equipment to go full FLAC. sad.gif We'll probably stick with V0.

    But for our RAID array I picked out this enclosure with four of these in RAID 5


    Just my 2 cents, coming from low-cost world.

    HP Proliant Microserver N40L (280 USD on amazon) + linux from usb flash drive + 5 x SATA 3GB drives in linux soft RAID5 (the 5th drive in the optical drive bay in SATA drive internal enclosure, SATA connector on board) = 12TB of reliable redundant file space. The drives are hot-swappable in linux (tested in internet forums). An extra eSATA connector available - possible to boot from another external SATA drive, or 6th drive for the array.

    The only issue is price of harddrives which is going down only slowly now.

    I would not rip to MP3s either, considering the amount of work the process will take.

    And BTW if you need MP3s from FLACs, perhaps the linux FUSE mp3fs would come handy, it works very good http://khenriks.github.com/mp3fs/
    Go to the top of the page
    +Quote Post

    Posts in this topic
    - trail   Help ripping ~30,000 CDs   Mar 29 2012, 19:08
    - - frozenspeed   I would use cueripper & foobar2000 but that...   Mar 29 2012, 19:11
    |- - trail   QUOTE (frozenspeed @ Mar 29 2012, 13:11) ...   Mar 29 2012, 19:13
    - - Dario   What about the dBpoweramp batch ripper? Is there a...   Mar 29 2012, 19:23
    |- - garym   QUOTE (Dario @ Mar 29 2012, 13:23) What a...   Mar 29 2012, 21:50
    |- - Roseval   You might try running multiple instances of dbPowe...   Mar 29 2012, 22:06
    ||- - trail   QUOTE (Roseval @ Mar 29 2012, 17:06) You ...   Mar 29 2012, 22:24
    |- - trail   QUOTE (garym @ Mar 29 2012, 16:50) QUOTE ...   Mar 29 2012, 22:36
    - - DVDdoug   QUOTE Ideally, we have one pretty decent quad-core...   Mar 29 2012, 23:33
    |- - trail   QUOTE (DVDdoug @ Mar 29 2012, 18:33) QUOT...   Mar 30 2012, 01:52
    - - Destroid   OP: I would concur with others to consider lossles...   Mar 30 2012, 00:08
    - - dumdidum   i second the suggestion of ripping to a lossless f...   Mar 30 2012, 08:56
    - - Porcus   I ripped about 7000 CDs to FLAC using dBpoweramp, ...   Mar 30 2012, 10:07
    - - LosMintos   Sorry, I didn't read all posts carefully, neve...   Mar 30 2012, 14:28
    |- - Porcus   QUOTE (LosMintos @ Mar 30 2012, 15:28) Yo...   Mar 30 2012, 16:38
    - - pdq   On the cost of storing lossless files - consider t...   Mar 30 2012, 16:40
    - - spoon   >My experience with dBpoweramp is that it might...   Mar 30 2012, 16:41
    - - trail   First off, I want to say thanks so much to everyon...   Mar 30 2012, 17:44
    |- - Porcus   QUOTE (trail @ Mar 30 2012, 18:44) If at ...   Mar 30 2012, 20:23
    |- - trail   Bad news, our budget means we probably won't b...   Apr 2 2012, 01:42
    |- - phofman   QUOTE (trail @ Apr 2 2012, 01:42) Bad new...   Apr 2 2012, 11:05
    |- - phofman   My experience with those SiliconImage SATA control...   Apr 2 2012, 11:22
    |- - Destroid   QUOTE (trail @ Apr 2 2012, 00:42) Bad new...   Apr 4 2012, 11:54
    |- - trail   QUOTE (Destroid @ Apr 4 2012, 06:54) QUOT...   Apr 5 2012, 20:41
    |- - shadowking   QUOTE (Destroid @ Apr 4 2012, 21:54) QUOT...   Apr 6 2012, 15:50
    - - .hx   Little side note - CD ripping is not digitizing.   Apr 2 2012, 03:06
    - - spoon   Word of advice, if you want a stress free life, st...   Apr 2 2012, 09:49
    - - Porcus   Since this is turning into a discussion of storage...   Apr 2 2012, 14:57
    |- - rick.hughes   QUOTE (Porcus @ Apr 2 2012, 09:57) - RAID...   Apr 2 2012, 15:40
    |- - phofman   We have been using linux raid in our company for a...   Apr 2 2012, 15:57
    |- - Porcus   QUOTE (phofman @ Apr 2 2012, 16:57) the r...   Apr 3 2012, 16:29
    - - LosMintos   QUOTE (spoon @ Mar 30 2012, 17:41) The la...   Apr 2 2012, 15:48
    - - krabapple   Nothing to add except again: go lossless if you c...   Apr 2 2012, 16:30
    - - pdq   For backup I would suggest AudioSAFE. If you never...   Apr 2 2012, 16:39
    |- - garym   QUOTE (pdq @ Apr 2 2012, 10:39) For backu...   Apr 2 2012, 22:25
    |- - Jan S.   QUOTE (garym @ Apr 2 2012, 22:25) QUOTE (...   Apr 3 2012, 10:42
    |- - Nessuno   QUOTE (Jan S. @ Apr 3 2012, 11:42) If the...   Apr 3 2012, 17:52
    - - rick.hughes   The original CDs could also be considered the back...   Apr 2 2012, 19:12
    |- - phofman   QUOTE (rick.hughes @ Apr 2 2012, 19:12) T...   Apr 2 2012, 19:25
    - - spoon   Your deletions are kept on audiosafe (the last cha...   Apr 3 2012, 19:33
    - - mixminus1   @Destroid: Yes, the OP's budget is limited, a...   Apr 5 2012, 21:06
    |- - trail   QUOTE (mixminus1 @ Apr 5 2012, 16:06) @De...   Apr 9 2012, 04:20
    - - JJZolx   If the budget is so limited that the station can...   Apr 6 2012, 01:43
    - - Porcus   OK, so OP cannot afford drives enough for lossless...   Apr 7 2012, 02:58
    - - gorob   I second dbpoweramp as well, it works great for ri...   Mar 11 2014, 03:16
    - - JJZolx   I wonder how this project turned out. Or if it eve...   Mar 11 2014, 04:01
    - - eahm   30,000 discs...I can't even imagine that many,...   Mar 11 2014, 05:43
    |- - Porcus   QUOTE (eahm @ Mar 11 2014, 05:43) It must...   Mar 11 2014, 08:44
    - - kennedyb4   Hi. I would like to add my advice to those suggest...   Mar 11 2014, 13:10
    - - yourlord   This thread is 2 years old and the original poster...   Mar 11 2014, 18:23
    - - eahm   QUOTE (Porcus @ Mar 11 2014, 00:44) Yep -...   Mar 11 2014, 23:46


    Reply to this topicStart new topic
    1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
    0 Members:

     



    RSS Lo-Fi Version Time is now: 2nd October 2014 - 12:45