Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Ogg+Flac as lossy+recovery file! (Read 13808 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Ogg+Flac as lossy+recovery file!

Reply #25
@mithrandir:

I agree that xtreme is more efficient than insane. But since insane stores the whole spectrum, Dibrom pointed out (it could be the same thread you were referring to) that this would in fact lend itself very well to transcoding while xtreme may do worse. On the other hand I do believe that at your extreme settings the difference is neglegible. You could of course also argue the other way around: using insane instead of xtreme with your tmn nmt values won't hurt either. Whatever suits you best.

Ogg+Flac as lossy+recovery file!

Reply #26
Arrrrgh... this debate is going in circles.

Quote
Originally posted by kritip
What happens if you lose all your ogg's (recovery files) in the event of a hard disk faliure, won't the archived files then be of no use because they can't be recovered??


Yes, this has been pointed out repeatedly, and acknoledged by all parties involved. Please people, read the whole thread before posting.

Quote
Originally posted by mithrandir 
Sometimes we need to take a step back and ask "what exactly are we doing?"[/b]


So what you're criticizing isn't the lossy+correction file method of lossless backup, but the idea of any lossless backup at all. This, I think, is silly. With current hard drive sizes, large though they are, archiving a collection on your primary drive in a lossless format is overkill, I agree. I also agree that storing lossless backups on CDR is not too practical, only being able to store ~3 albums per CDR or so. If, however, I had a DVD-RW drive, or one of those 30gb tape drives, this idea would be very appealing.

Quote
Originally posted by mithrandir 
To go through this effort of making two files with two separate encoders, then burning one of them to media that can only handle 700MB...


I'll give you the point about burning to CDR, but as to making 2 separate files... a 2 line batch file or perl script would reduce this task to triviality.

Quote
Originally posted by mithrandir
If my original album were destroyed and I could never, ever get another original copy, I probably wouldn't "care" from an audio standpoint. What am I losing? Some wispy fine detail -60dB down in level that my ears can't hear anyway? Oh well, life goes on....


You're only focusing on the one criterion which is important to you: quality of the first generation lossy copy. For me there's more to it than that. Yes, MPC would give essentially transparent results... but how about MPC transcoded to MP3? *I* need to be able to play my files on a hardware player. As far as I know, no extensive testing has been done on the quality resulting from various transcoding scenarios, though it is guranteed to be less than a direct encode from original wav's. So OGG for me would probably be the best bet, as hardware player support has been promised by multiple vendors. OGG isn't quite as transparent yet as MPC though... so should I wait till 1.0 comes out and hopes it approches MPC levels of transparency? Even then, what if hardware support never materializes? So should I encode in MP3, knowing full well that even with Dibrom's improvements, my files will not be totally transparent?

Basically the problem isn't with these particulars, but with the fact that you're critcizing other's archiving methodology based on *your* requirements, rather than theirs. You need to take a long view and realize that the vast majority of people out there think that all of us who spend time on hyrdogenaudio and the like are crazy and wasting our time, since 128kbps is CD quality after all.

Ogg+Flac as lossy+recovery file!

Reply #27
lossless compression in general (and those lossy thumbs+recovery too) is very useful for trading in-concert recordings (too bad that shorten dominates there  )

but I think the attempt to use a common lossy format is dangerous since you usually can't rely on getting bit-identical output from different decoders/decoder versions...

edit: and you'd better watch out for limiting/replaygain/decoder-side clipping

(OT)
Quote
Tonight I encoded Live's A Distance To Here


way to go, mithrandir, your taste in music is good
(/OT)
A riddle is a short sword attached to the next 2000 years.

Ogg+Flac as lossy+recovery file!

Reply #28
Quote
Originally posted by Gecko
since insane stores the whole spectrum, Dibrom pointed out that this would in fact lend itself very well to transcoding while xtreme may do worse...using insane instead of xtreme with your tmn nmt values won't hurt either

I did investigate this and was a little surprised with what I found. I believe xtreme imposes a fixed lowpass of 19.5KHz but when I decode my xtreme files to WAVs and analyze them in a wave editor I notice that there is often information in the 20KHz+ region...sometimes even in the 21KHz+ region. When I use LAME aps, the frequencies get chopped off by 19KHz all the time, OTOH. Perhaps MPC's lowpass method is different than LAME's.

I did try my nmt/tmn settings with insane and bitrates generally went up by 15-20%. I decided it was not worth it considering that xtreme seems to have nearly full bandwidth anyway.
Quote
Originally posted by Randum
So what you're criticizing isn't the lossy+correction file method of lossless backup, but the idea of any lossless backup at all.

I actually trust the original pressed media more than burned dye CD-Rs, so I never seriously considered a backup plan. MAYBE for an album or two of mine that is rare, but in the worst case scenario, if I lose something I can generally rebuy it. Perhaps my logic is flawed, but I'm not losing any sleep over it.
Quote
Originally posted by Randum
Yes, MPC would give essentially transparent results... but how about MPC transcoded to MP3? *I* need to be able to play my files on a hardware player.

I suppose this is what I meant by "what exactly are we doing?"

If the purpose of transcoding my MPCs to MP3s is for playback on hardware players, then I don't see much of a problem at all. Why? Because these hardware players are devices used in environments that can hardly be considered conducive for audiophile playback. For example, if I must make a transcoded MP3 for use in a car MP3 receiver, what am I really losing? Unless you have a professional car stereo setup in a Lexus LS430, the noise from the road, the wind and the car itself will kill any of the little details that a super-high quality lossy encode may have preserved. Same thing with a portable like the Rio. If you listen on the go - on the train, on the bus, walking through town, whatever - are you going to be able to critically analyze the quality of the encode? Are you going to be able to tell the difference between a directly encoded MP3 and an MP3 transcoded from a high-bitrate MPC? On a portable or in the car, I should think not. Switching headphones may have a greater effect on sound quality. Is the portable device's output circuitry up to the task? Do you see my point? Diminishing returns. Obsessiveness over relative insignificance. I'm not trying to stunt progress but am suggesting that there may be other things of significance that deserve the effort.
Quote
Originally posted by Randum
you're critcizing other's archiving methodology based on *your* requirements, rather than theirs. You need to take a long view and realize that the vast majority of people out there think that all of us who spend time on hyrdogenaudio and the like are crazy and wasting our time, since 128kbps is CD quality after all.

It's not so much scathing criticism as it is an alternative perspective and a voice of reason. I well understand that most people here are very quality oriented. So am I. But I realize that you can get ahead of yourself in the search for perfection. I cannot say that what's "good enough" for me should be "good enough" for everyone else. But I do believe it is acceptable conversation to question methods and activities that others want to pursue...not necessarily to be condescending or elitist, but to present a perspective that adds to group thought processes and create a more well-rounded consensus.

Ogg+Flac as lossy+recovery file!

Reply #29
Even if this lossy + difference file method made the total file smaller (rather than larger) than just the lossless, I would not use it for backup purposes unless the lossy + difference file was significantly smaller.

Having a workable lossless file is much more useful; it can be played or re-encoded directly from the CD. Using the lossy + difference file requires temporary hard drive space to decompress and merge the files, not to mention a lot of extra processing and hard drive grinding time.

Also, I'd like to point out to Randum that Mithrandir did not criticize "the idea of any lossless backup at all." He says at the end of his post, "If [a] CD was known to be OOP/rare/etc, I would have used a lossless format like LPAC or MAC."

Personally, I think Mithrandir's backup methodology makes perfect sense for my needs, but then again there are people such as Randum and myself who want to be able to re-encode to other formats for portables or formats without the hassle of re-ripping.

Since my portable player is used in less-than-ideal playback situations, I have no problem with transcoding from "MPC --insane" to "Lame --ap fast standard" or "AACenc --normal." For those not willing to make the slight quality sacrifice, though, lossless is a viable option.

I've done my own tests, transcoding MPC > MP3 and MP3 > MP3, and I found that going from MPC --insane to Lame --ap 128 sounded like a direct --ap 128 encode, while Lame --ap standard to Lame --ap 128 had noticeable degradation. My ears aren't golden (both --insane and --ap standard sounded perfect to me), but as Dibrom and others have said, MPC is quite good for transcoding (to MP3, at least).

Edit: it looks like it took me a bit too long to type this: everything I had to say has already been said

Ogg+Flac as lossy+recovery file!

Reply #30
Quote
Originally posted by mithrandir

I actually trust the original pressed media more than burned dye CD-Rs, so I never seriously considered a backup plan.

Pressed CD's do last longer than burned CD's.... if they are both stored in the same way. But audio CD's get tossed around, sat on, left under the seat of your car for weeks.... etc. Backups would sit in jewel cases in a box in my closet. Also, keep in mind that a large portion of music I would be 'backing up' I wouldn't have access to the pressed CD for longer than it takes to get it to the nearest computer with EAC

Quote
Originally posted by mithrandir

Because these hardware players are devices used in environments that can hardly be considered conducive for audiophile playback.


For most people, probably. For me, exactly the opposite. My car has $1200 of audio equiment in it, and I am in the process of dynamating the interior. I frequently hear details in music when listening in the car that I don't hear elsewhere. If I had a handheld, I would be using it with my $200 Sennheiser cans, the closest to optimal listening environment you can get on a non-ridiculous budget. My home speakers OTOH, are shitty labtec $60 computer speakers - it just isn't worth investing in good home speakers when I live in an apartment and can't turn em up loud. So for me, I need HIGHGER quality on a portable than on my desktop.

Ogg+Flac as lossy+recovery file!

Reply #31
Quote
Originally posted by mithrandir

I did investigate this and was a little surprised with what I found. I believe xtreme imposes a fixed lowpass of 19.5KHz but when I decode my xtreme files to WAVs and analyze them in a wave editor I notice that there is often information in the 20KHz+ region...sometimes even in the 21KHz+ region. When I use LAME aps, the frequencies get chopped off by 19KHz all the time, OTOH. Perhaps MPC's lowpass method is different than LAME's.


Yes, MPC uses a variable lowpass in --xtreme. The lowpassing threshold is adjusted on-the-fly during encoding, depending on the HF energy and psychoacoustic calculations.

Ogg+Flac as lossy+recovery file!

Reply #32
Quote
Originally posted by CiTay
Yes, MPC uses a variable lowpass in --xtreme. The lowpassing threshold is adjusted on-the-fly during encoding, depending on the HF energy and psychoacoustic calculations.


This is sort of correct, except that it isn't actually lowpassing I think.  Instead it's just all based on the ath and various masking calculations, so in the end it has a similar effect to an adaptive lowpass.. just a nitpick I suppose, but an explicit lowpass would be a bit different than just not encoding up to a certain frequency cutoff which all beyond is deemed inaudible by the psymodel.

Ogg+Flac as lossy+recovery file!

Reply #33
isn't the bottom line is that:

this produces 2 files, one with the whole song encoded in ogg, and another with the .wav difference lossless encoded.

therefor, the total encode for backup purposes is smaller than just the full lossless encode, PLUS the added bonus of a easily distributable lossy version.

i don't know about you guys, but this seems like it has alot of opportunitys in various areas.

if your archiving, the end result is smaller.
and you have something you can toss to a friend!

pretty cool if you ask me.

would be the most useful with a DAT tape drive

 

Ogg+Flac as lossy+recovery file!

Reply #34
Quote
Originally posted by meff
isn't the bottom line is that:

this produces 2 files, one with the whole song encoded in ogg, and another with the .wav difference lossless encoded.

therefor, the total encode for backup purposes is smaller than just the full lossless encode, PLUS the added bonus of a easily distributable lossy version.

i don't know about you guys, but this seems like it has alot of opportunitys in various areas.

if your archiving, the end result is smaller.
No. This is just it. The result will not be smaller than just one lossless encode.
Ogg+lossless difference signal is always bigger than just single full lossless original.
Juha Laaksonheimo

Ogg+Flac as lossy+recovery file!

Reply #35
Quote
Originally posted by JohnV
No. This is just it. The result will not be smaller than just one lossless encode.
Ogg+lossless difference signal is always bigger than just single full lossless original.

No, you guys are still not getting the point of all this. Yes, lossy+correction file will be bigger than lossless alone... but it will be smaller than lossy+lossless... which is what you would need if your situation requires a lossless backup, and easy access to a playable version.

Ogg+Flac as lossy+recovery file!

Reply #36
Yeah I get it. I just didn't pay enough attention what was said..
Somehow I totally missed the "PLUS the added bonus of a easily distributable lossy version." -part.
Juha Laaksonheimo

Ogg+Flac as lossy+recovery file!

Reply #37
I wonder whether there's a possibility to compress the difference file more efficiently...

lower amplitude, but higher enthropy, hmmm :confused: ideas, anyone?
A riddle is a short sword attached to the next 2000 years.

Ogg+Flac as lossy+recovery file!

Reply #38
Quote
Originally posted by ssamadhi97
I wonder whether there's a possibility to compress the difference file more efficiently... 

lower amplitude, but higher enthropy, hmmm :confused: ideas, anyone?


My assumption is that "more efficiently" would mean lossy, and once you'll get here you'll end up with nothing else but garbage in the end...

Anyway, I see no point in storing the difference other than for "academic purposes" (What if the HDD crashes??)... recordable CD's are very cheap these days...

Ogg+Flac as lossy+recovery file!

Reply #39
Well, I've been experimenting with deltaS (the leftover) since I'm trying to find the most optimal way to store field recordings for my samples. I record a lot, but sometimes just don't have the time to manipulate. So I store the result, until I at least have the time to edit and take off the garbage (the long useless parts of a 2 hours recording....) In my case, I was particularly interested in compression closest possible to "bit perfect", since the material will, later, be manipulated and that psymodel can ruin an "artifact" that I may want to isolate later...

Someone have suggested that Lossless Audio compressor (Monkey's, flac, shorten and friends) MIGHT not be the most efficient for the nature of the data contained in a leftover file. He was suggesting that a general purpose compressor would PROBABLY be better than one specialised with AUDIO.  (In fact that was an interesting idea, because it's pretty true that DeltaS (leftovers) sound like fuzz or filtered white noise,; and the higher the bitrate of the lossy codec, the more fuzzy (and small) is the leftover. Try with Lame --freeformat 500kbps, fuzz-fuzz-fuzz.

I tried, yet only once, to compress the leftover with zip instead of Monkey . Here's the result:

Aphex Twin- I care because you do- 08 - Wet tip hen ax compressed with lame --alt-preset lowpass18 -F the result was 8 592 KB (original 54 655kb) the leftover compressed, in Monkeys audio at 43% the original size and in winzip, 55%, so I believe that Audio compressor are still more suited for leftovers than Genral purpose one. I may théorize that even if the result (leftover) sounds like fuzz, the samples in it still bear a strong correlation, that Monkey's audio is more able to exploit than WinZip.

I would permit myself to add this note: Compression is still FAR from being pushed at it's max, we will see lossless Audio Compressor that will perform at ratios like 40 or more, just because Music is a lot simpler than we may think, there is so much redundency that today's Codecs and compressors don't perceive.  Both Codec and compressor look at Audio data with a microscope, searching for extremely local redundency. They incorporate not much AI (if at all), they don't understand the nature of the sounds they compress....  Just look at MPEG4-SA  (structured audio) the plan is to determine what instruments are played, in what way, the recreate it in a midi-like way, and then, to correct the mistakes that have been made.... Pretty powerfull and extremely scalable....Better than CD quality at 32kbps....
Wait till AI puts its nose in audio Compression and we\'ll see lossless compression at 100:1 ratios....