Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Steps to rip/encode HDCD to FLAC/MP3? (Read 17656 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Steps to rip/encode HDCD to FLAC/MP3?

I have a few HDCD discs that I'd like to rip to FLAC and MP3. What are the steps I need to take to do this?
My usual CD ripping procedure is to rip tracks to WAV with Exact Audio Copy and then encode WAV files with a batch script I wrote that invokes flac.exe/lame.exe. With HDCD, I suppose I also have to decode the WAV files?
I have to admit that until a few months ago, I'd never even heard of HDCD.

I found the HDCD software decoder on Doom9's Forum <http://forum.doom9.org/showthread.php?t=129136>. I tried it on a WAV file I'd ripped from my Mark Knopfler - Sailing to Philadelphia HDCD, and it successfully output a HDCD-decoded 24-bit WAV file. The output WAV is not as loud as the input WAV. Is this the expected behaviour?

Is this output WAV what I am supposed to be encoding to FLAC/MP3, or am I first supposed to normalize it or boost its volume or something to counteract the drop in loudness? Or would that just be defeating the whole purpose of the HDCD decoding?

Sorry for my lack of understanding in this area. I did some searching/reading, but I wasn't able to find much to help me.

Steps to rip/encode HDCD to FLAC/MP3?

Reply #1 –
It is normal for the output file to be quieter. HDCD is essentially 16-bit audio with some extra information encoded that allows for dynamic range extension, for example higher peaks. That's going to, by necessity, require the overall volume to be lower.

The simplest thing to do would be to encode the quieter WAVs as-is, really the only downside to that is if you don't use ReplayGain it might sound quieter when played back-to-back with another album. What I would personally do is to normalize the quiet WAVs so that the highest peak on the album hits 0 dBfs. It's critical that the normalization be done across the entire album rather than individual tracks, or you will end up with tracks of varying loudness.

I do the same thing with my vinyl/tape rips and other recordings. Of course, there it's usually done as the last step before downconverting to 16-bit. 

Steps to rip/encode HDCD to FLAC/MP3?

Reply #2 –
If you're using foobar2000 to play back the FLAC files, you can also just rip the CD normally with EAC/CUERipper and use the foo_hdcd component to decode the HDCD information in realtime. The advantage would be the capability to check your ripped files against CTDB/AR, and being able to repair the rip in case of bit rot.

foo_hdcd also offers the option to decode the HDCD information on conversion, so you could convert it to MP3 that way. As for the volume, as slks said it's normal, I just scan the rip with ReplayGain (which calculates the gain for the decoded version if you have foo_hdcd installed), so I have no difference in volume when listening.

Steps to rip/encode HDCD to FLAC/MP3?

Reply #3 –
Thanks for the helpful replies.

Quote
It's critical that the normalization be done across the entire album rather than individual tracks, or you will end up with tracks of varying loudness.


Do you know of some Windows freeware for doing this?

Steps to rip/encode HDCD to FLAC/MP3?

Reply #4 –
Quote from: osullic on
Do you know of some Windows freeware for doing this?

foobar2000. (Select one or more albums and choose Replaygain -> Scan selection as albums (by tags) from the context menu. Or select one album and choose Replaygain -> Scan selection as a single album from the context menu.)

Steps to rip/encode HDCD to FLAC/MP3?

Reply #5 –
Quote from: osullic on
I have a few HDCD discs that I'd like to rip to FLAC and MP3. What are the steps I need to take to do this?
My usual CD ripping procedure is to rip tracks to WAV with Exact Audio Copy and then encode WAV files with a batch script I wrote that invokes flac.exe/lame.exe. With HDCD, I suppose I also have to decode the WAV files?
I have to admit that until a few months ago, I'd never even heard of HDCD.

I found the HDCD software decoder on Doom9's Forum <http://forum.doom9.org/showthread.php?t=129136>. I tried it on a WAV file I'd ripped from my Mark Knopfler - Sailing to Philadelphia HDCD, and it successfully output a HDCD-decoded 24-bit WAV file. The output WAV is not as loud as the input WAV. Is this the expected behaviour?

Is this output WAV what I am supposed to be encoding to FLAC/MP3, or am I first supposed to normalize it or boost its volume or something to counteract the drop in loudness? Or would that just be defeating the whole purpose of the HDCD decoding?

Sorry for my lack of understanding in this area. I did some searching/reading, but I wasn't able to find much to help me.



There are many 'HDCDs' where the only difference between coded and decoded is the overall level.  The 'dynamic range' (actually the crest factor, difference between average and peak levels) is not changed.  All decoding does is reduce the level by some dBs  (3 or 6, I forget which).  If you play the nondecoded version and just turn the volume down that many dB, it's the same affect.  (Why was the CD encoded with HDCD in the first  place then?  Good question.)

But when 'peak extension' has been used during encoding,  dynamically compressed content will actually be expanded when decoded. The dynamic range itself will increase.  The peaks will be 'higher' in relation to the average.

Was it necessary to encode the audio in HDCD to achieve that dynamic range?  Nope; CD can handle dynamic range of over 100dB.  So again, why HDCD?  Good question.




Steps to rip/encode HDCD to FLAC/MP3?

Reply #6 –
Quote from: osullic on
I have a few HDCD discs that I'd like to rip to FLAC and MP3. What are the steps I need to take to do this?
My usual CD ripping procedure is to rip tracks to WAV with Exact Audio Copy and then encode WAV files with a batch script I wrote that invokes flac.exe/lame.exe. With HDCD, I suppose I also have to decode the WAV files?
I have to admit that until a few months ago, I'd never even heard of HDCD.

I found the HDCD software decoder on Doom9's Forum <http://forum.doom9.org/showthread.php?t=129136>. I tried it on a WAV file I'd ripped from my Mark Knopfler - Sailing to Philadelphia HDCD, and it successfully output a HDCD-decoded 24-bit WAV file. The output WAV is not as loud as the input WAV. Is this the expected behaviour?

Is this output WAV what I am supposed to be encoding to FLAC/MP3, or am I first supposed to normalize it or boost its volume or something to counteract the drop in loudness? Or would that just be defeating the whole purpose of the HDCD decoding?

Sorry for my lack of understanding in this area. I did some searching/reading, but I wasn't able to find much to help me.


The version you used is obsolete. It's only designed to only decode CD's that are encoded with HDCD Peak Extension, which means that anything you create with it will be, by default, reduced in volume by 6 db. You really should use fb2k with the HDCD component installed that ChromeSphere mentioned. It'll automatically detect both HDCD & HDCD PE. You'll have to modify the Status bar readout within fb2k so that you can see what type a CD has during playback (File/Preferences/Default User Interface/Status bar) with this line:

$if(%ispaused%,Paused,Playing) | %codec% | %bitrate% kbps | %samplerate% Hz | $caps(%channels%) | %playback_time%[ / %length%][ |$if(%__hdcd%,' 'HDCD '('gain: %__hdcd_gain%',' peak extension: %__hdcd_peak_extend%')',)]

To create permanent HDCD or HDCD PE 24 bit files, you'd create a converting profile that outputs 24 bit files with the Additional Processing option checked off in the Processing section. You, unfortunately, can only choose FLAC, WavPack and TAK for this (which have blocksize options for controlling the size of HDCD PE)
ghostman

Steps to rip/encode HDCD to FLAC/MP3?

Reply #7 –
I regret that I ripped my HDCDs with HDCD conversion. Because HDCD is flagged in-signal - this provided it is stored lossless, so the signal remains intact! - it can always be detected upon playback if your player supports it (fb2k!) and also detected for later conversion (to lossy, or if you later decide to convert the HDCDs once-and-for-all).  Retro-checking with AccurateRip (and repair with CUEToolsDB) will be ruled out once decoded, for example.
Besides, there are compilation CDs with HDCD on some-but-not-all tracks. The volume shift thing is annoying on those - I do not really know what would be best on each, but leaving them intact leaves the option for later.




Quote from: osullic on
The output WAV is not as loud as the input WAV. Is this the expected behaviour?


Yes - but with reservations.
HDCD has a "peak extension" where the top 9 decibels are crammed into 3. Reversing this procedure requires 6 dB extra dynamic range, and since you cannot go above "digital 0" with PCM, you must reduce first.
The reservation: HDCD need not employ peak extension, in which case you should not reduce. (In fact, there are a lot of "HDCDs" which utilize no HDCD features, they only say "I am a HDCD and the following HDCD features are used:" [and then none]).

Steps to rip/encode HDCD to FLAC/MP3?

Reply #8 –
With regard to foobar2000 performing album-based normalisation, it didn't do what I intended. I am working with WAV files (for later encoding to FLAC/MP3) and what I want is for the normalisation to be "hard-coded" into the WAV files. foobar2000 just added ReplayGain tags to the WAV files, which I think is even non-standard for WAV files, no?

I have found a command-line program called normalize that has a "batch mode", that I think might do what I want (but I'm open to correction):
http://normalize.nongnu.org

Now back to the issue of decoding HDCD-encoded material. I followed the instructions from themanintheshadows_2451 for getting HDCD-related information to display in the foobar2000 status bar. For the five HDCDs I have to hand, when I play the tracks in foobar2000, the status bar says (for four of the five HDCDs):
"HDCD (gain: 0.0 dB, peak extension: yes)"
What is that telling me?
For Neil Young's Greatest Hits, during the silence between tracks, the gain value changes to -4.0 dB.

For the fifth HDCD (Bloodhound Gang - Hooray for Boobies), no HDCD-related information is shown in the foobar2000 status bar - does that mean that the disc is actually just a regular CD despite the HDCD logo on the rear cover?

As for converting/decoding HDCD-encoded 16-bit WAV files to 24-bit files with foobar2000, the converted files are still showing decreased loudness compared to the input files. Does this still make sense, given what I mentioned is displayed in the foobar2000 status bar above? Is foobar2000 decoding some sonic improvements into my output files, or are they just receiving a (possibly unnecessary?) loudness decrease?

Lastly, why do you say that I can only choose FLAC, WavPack and TAK for creating permanent 24-bit files? I tried the process with WAV output, and foobar2000 seems to have successfully decoded/output 24-bit WAV files.

Steps to rip/encode HDCD to FLAC/MP3?

Reply #9 –
Quote from: osullic on
With regard to foobar2000 performing album-based normalisation, it didn't do what I intended. I am working with WAV files (for later encoding to FLAC/MP3) and what I want is for the normalisation to be "hard-coded" into the WAV files. foobar2000 just added ReplayGain tags to the WAV files, which I think is even non-standard for WAV files, no?


Scan the file with replaygain, then right click, Convert and choose WAV with Replaygain processing.

Steps to rip/encode HDCD to FLAC/MP3?

Reply #10 –
Quote from: osullic on
With regard to foobar2000 performing album-based normalisation, it didn't do what I intended. I am working with WAV files (for later encoding to FLAC/MP3) and what I want is for the normalisation to be "hard-coded" into the WAV files. foobar2000 just added ReplayGain tags to the WAV files, which I think is even non-standard for WAV files, no?

I have found a command-line program called normalize that has a "batch mode", that I think might do what I want (but I'm open to correction):
http://normalize.nongnu.org

Now back to the issue of decoding HDCD-encoded material. I followed the instructions from themanintheshadows_2451 for getting HDCD-related information to display in the foobar2000 status bar. For the five HDCDs I have to hand, when I play the tracks in foobar2000, the status bar says (for four of the five HDCDs):
"HDCD (gain: 0.0 dB, peak extension: yes)"
What is that telling me?
For Neil Young's Greatest Hits, during the silence between tracks, the gain value changes to -4.0 dB.

For the fifth HDCD (Bloodhound Gang - Hooray for Boobies), no HDCD-related information is shown in the foobar2000 status bar - does that mean that the disc is actually just a regular CD despite the HDCD logo on the rear cover?

As for converting/decoding HDCD-encoded 16-bit WAV files to 24-bit files with foobar2000, the converted files are still showing decreased loudness compared to the input files. Does this still make sense, given what I mentioned is displayed in the foobar2000 status bar above? Is foobar2000 decoding some sonic improvements into my output files, or are they just receiving a (possibly unnecessary?) loudness decrease?

Lastly, why do you say that I can only choose FLAC, WavPack and TAK for creating permanent 24-bit files? I tried the process with WAV output, and foobar2000 seems to have successfully decoded/output 24-bit WAV files.


Well, of course WAV will work. But, obviously, you'd want to convert it to a lossless codec of some sort to save space. And when it comes to tracks encoded with HDCD Peak Extension, FLAC, WavPack and TAK are the only lossless codecs that have blocksize control that allow you to deal with the increased size that fully decoded HDCD PE files end up being.

It's also not unusual for HDCD CD's to have at least one track that's not encoded (like the 2000 Van Halen HDCD remaster of "Diver Down") Maybe it's a purposeful thing, a mistake...who knows?

FYI: The HDCD component will, by default, decrease the volume of tracks encoded with HDCD PE by 6db to prevent clipping. That's normal.
ghostman

Steps to rip/encode HDCD to FLAC/MP3?

Reply #11 –
Thanks again for the helpful replies.

I also today came across another post here that has some good information:
http://www.hydrogenaud.io/forums/index.php...9427&st=250

I now understand that, in the foobar2000 status bar, if %__hdcd_gain% has a value other than 0.0 dB, this indicates that the HDCD 'low level extension' feature is enabled, and if %__hdcd_peak_extend% ever has a value of 'yes', this indicates that the HDCD 'peak extension' feature has been enabled.
If either of these features have been enabled, then the audio should be HDCD-decoded.
If neither of these features have been enabled, then the HDCD-decoding process doesn't change the audio at all (other than to uselessly pad it to 24-bit).

What I had been concerned about was, in the case of my HDCD audio, even with the 'yes' for peak extension, that the low level extension value of 0.0 dB might be indicating that the audio didn't need decoding, and that by doing so, all I was doing was introducing an unnecessary decrease in loudness. But I now understand the situation better and know, from what I've said above, that my audio should indeed be HDCD-decoded.

Steps to rip/encode HDCD to FLAC/MP3?

Reply #12 –
Quote from: osullic on
I am working with WAV files (for later encoding to FLAC/MP3)


Except in exceptional cases (if you actually process the audio with software that likes WAV but dislikes e.g. FLAC) there is no use in keeping them as WAV, and there are disadvantages - like tagging, as you mention. (WAV can be tagged, but that is less likely to be supported across applications).


Quote from: osullic on
what I want is for the normalisation to be "hard-coded" into the WAV files.


If that is what you want - I would tend to think it is not a good idea, it could rather be applied upon converting to lossy if and when so you wish - then you need to convert.



Quote from: osullic on
For the five HDCDs I have to hand, when I play the tracks in foobar2000, the status bar says (for four of the five HDCDs):
"HDCD (gain: 0.0 dB, peak extension: yes)"
What is that telling me?


That means that the CDs employ the "peak extension" feature of HDCD, but not - at the instant you play - the low-level level reduction is not employed. To check whether there is any low-level gain you need to check the entire track.

As you see from the Neil Young CD, the low-level feature is -4 during silence. Check a track that fades out; then you should observe the number scale from 0 to -4. The idea is that a code to reduce level extends the dynamic range, as a digital 0000000000000001 gets lower volume.




Quote from: osullic on
For the fifth HDCD (Bloodhound Gang - Hooray for Boobies), no HDCD-related information is shown in the foobar2000 status bar - does that mean that the disc is actually just a regular CD despite the HDCD logo on the rear cover?


Most likely yes. But you cannot count on every track being the same, so you need to check them all.

Quote from: osullic on
As for converting/decoding HDCD-encoded 16-bit WAV files to 24-bit files with foobar2000, the converted files are still showing decreased loudness compared to the input files.


It is a bit unclear what you did and what you observed. Rather than replying to the "if you did ... " scenarios: what did you do and what did you observe afterwards?

Steps to rip/encode HDCD to FLAC/MP3?

Reply #13 –
@Porcus - If I'm going to permanently change anything that I've losslessly ripped, I keep the original lossless rip with an extra ".original" extension (so it doesn't show up as a FLAC). I can always go back to it if I need to, but I use the changed version otherwise (which sits in my music library as normal).

That way my library always has the best and easiest to use version of everything, and I don't have to worry that my player or transcoder supports X feature.

Pre-emphasis, HDCD, audible glitches (i.e. faults in the recording, not the rip), censorship etc all get "fixed" this way. My main library is (currently) all 16/44.1 stereo. It makes using it so simple

Cheers,
David.

Steps to rip/encode HDCD to FLAC/MP3?

Reply #14 –
I thought I was finished with this thread, but I want to return to the normalisation question - that is, normalisation of the HDCD-decoded audio.

slks says:
Quote
What I would personally do is to normalize the quiet WAVs so that the highest peak on the album hits 0 dBfs. It's critical that the normalization be done across the entire album rather than individual tracks, or you will end up with tracks of varying loudness.

and I agree that this seems like a good idea.

I've been trying to get foobar2000 to normalise the audio through ReplayGain processing, but I'd like the output to be louder than what I'm getting - closer in loudness to the pre-decoded audio. (I don't normally use ReplayGain, and I'm happy without it.) I know I can mess around with preamp in the ReplayGain settings of the foobar2000 converter, but that seems a bit arbitrary to me.

Now I'm thinking that the ideal situation might be to:
- Determine the peak of the audio across an entire album before HDCD decoding.
- Determine the peak of the audio across the entire album after HDCD decoding.
- Apply the necessary gain to the decoded audio to bring it to the same peak as before decoding, with every song on the album having the same gain value applied.

Is this logic flawed? If not, how could I go about this?

Steps to rip/encode HDCD to FLAC/MP3?

Reply #15 –
Quote from: osullic on
I know I can mess around with preamp in the ReplayGain settings of the foobar2000 converter, but that seems a bit arbitrary to me.
For peak normalisation, it's not arbitrary: you put the pre-amp to maximum, use album mode and enable clipping prevention. In virtually all cases this gives you peak normalisation by album.

In this specific scenario you need to make sure the HDCD decoder is in-line when ReplayGain scanning and applying.

I wouldn't worry about what the undecoded peak was, or trying to match it.

Cheers,
David.

Steps to rip/encode HDCD to FLAC/MP3?

Reply #16 –
Thanks, that seems to do just what I was hoping to achieve.

Steps to rip/encode HDCD to FLAC/MP3?

Reply #17 –
Quote from: themanintheshadows_2451 on
The version you used is obsolete. It's only designed to only decode CD's that are encoded with HDCD Peak Extension, which means that anything you create with it will be, by default, reduced in volume by 6 db. You really should use fb2k with the HDCD component installed that ChromeSphere mentioned. It'll automatically detect both HDCD & HDCD PE.
 

Why is it obsolete? The hdcd.exe version by Chris Key supports peak extension and gain adjustment, but not reconstruction filter switching. The version by Kode54, used in FB2K, supports the same features, according to the author himself. So why is the other version obsolete?

Quote from: cjk32 from Doom9 link=msg=0 date=
HDCD allows the use of three features: peak extend, gain adjustment and  reconstruction filter switching. Windows media player supports peak extend and  gain adjustment, and it has hence been possible to fully implement these  features.

Filter switching, however, only makes sense when you are  upsampling, something which wmp doesn't do, and there's hence no reference to  implement it from. Investigation of various test CDs and various (usually  contradictory) literature has led to the conlusion that there are only two  filters available at playback, the normal filter and the transient filter.