IPB

Welcome Guest ( Log In | Register )

> foobar2000 General Forum Rules

This is NOT a tech support forum.
Tech support questions go to foobar2000 Tech Support forum instead.

See also: Hydrogenaudio Terms of Service.

 
Reply to this topicStart new topic
Replaygain data lost during conversion, To opus from flac
Xanikseo
post Jul 18 2013, 02:54
Post #1





Group: Members
Posts: 27
Joined: 14-April 09
Member No.: 68951



First of all I would like to thank all those who contributed to the development of fb2k. It's by far the most flexible music-playing application out there imo!

However, I have run into my first problem; the title reveals all.

Applying replaygain manually to the opus files after their generation seems to be the solution for now, presumably because opus' peculiar way of dealing with gain tags has been accommodated for in that department. However, this just adds (seemingly) unnecessary time and effort after the already painfully lengthy process of encoding a vast agglomeration of flacs.

This is all despite the fact that running opusenc on its own in the terminal does retain replaygain data when supplied a flac.

Am I right in thinking that foobar decodes the flac into a plain wav (or something similar) and provides this to opusenc via pipe? It confuses me though that the resulting opus files have all the other tags. Nonetheless, I tried using %s instead of pipe to no avail.

Anyway, could someone tell me if/what I am doing wrong. If this is simply an oversight, would it be possible to patch this for release in a future version?

Thanks in advance for any help!

This post has been edited by Xanikseo: Jul 18 2013, 02:57
Go to the top of the page
+Quote Post
TomasPin
post Jul 18 2013, 05:16
Post #2





Group: Members
Posts: 204
Joined: 5-June 13
From: Argentina
Member No.: 108508



Not really a solution to the problem, but in the meantime if the ReplayGain tags are not a necessity and you want foobar to handle the conversion you can try scaling the output of the encoder under the processing option of the converter. It will permanently change the loudness of the opus files according to track or album gain of the original FLAC files, if I'm not mistaken.

No idea what's causing the missing tag issue however, sorry.

This post has been edited by TomasPin: Jul 18 2013, 05:17


--------------------
A man and his music: http://tubular.net/
Go to the top of the page
+Quote Post
A_Man_Eating_Duc...
post Jul 18 2013, 10:22
Post #3





Group: Members
Posts: 943
Joined: 21-December 01
From: New Zealand
Member No.: 705



I think this has to do with the album and track peaks being different on lossy encodes.

FLAC


OPUS


Looks like the peaks aren't stored in the Opus tag, just the gain values. is this a bug?


This post has been edited by A_Man_Eating_Duck: Jul 18 2013, 10:25


--------------------
Who are you and how did you get in here ?
I'm a locksmith, I'm a locksmith.
Go to the top of the page
+Quote Post
Xanikseo
post Jul 18 2013, 16:47
Post #4





Group: Members
Posts: 27
Joined: 14-April 09
Member No.: 68951



It seems there are not supposed to be any peak tags as per recommendation from xiph: http://wiki.xiph.org/OggOpus#Comment_Header
QUOTE
To avoid confusion with multiple normalization schemes, an OpusTags packet SHOULD NOT contain any of the REPLAYGAIN_TRACK_GAIN, REPLAYGAIN_TRACK_PEAK, REPLAYGAIN_ALBUM_GAIN, or REPLAYGAIN_ALBUM_PEAK fields.


QUOTE (A_Man_Eating_Duck)
I think this has to do with the album and track peaks being different on lossy encodes.
I think this might be it. Replaygain tags are lost when converting to any other format! There should be an option somewhere to ask if replaygain tags should be kept (even if they are slightly different values on lossless vs lossy), rather than silently destroying them...
Go to the top of the page
+Quote Post
db1989
post Jul 19 2013, 15:44
Post #5





Group: Super Moderator
Posts: 5275
Joined: 23-June 06
Member No.: 32180



This is not a fault of, or caused by, Opus. The Converter does not transfer ReplayGain data between any formats.

This includes conversions from one lossless format to another, which by definition would not affect average loudness at all. Various members have requested that this ability be added for lossless files, but no reply has been given that I know of.
Go to the top of the page
+Quote Post
EpicForever
post Jul 19 2013, 17:24
Post #6





Group: Members
Posts: 724
Joined: 14-September 11
From: Szczecin, PL
Member No.: 93712



Again - this could be allowed as an additional option for those who want it, activated via checkbox in converter window, for example with additional info written in bold, that accuracy of this transfer is not assured. I was also bit confused and disappointed when found this Converter behavior for the first time, but I don't find it as a big problem.
Go to the top of the page
+Quote Post
ChronoSphere
post Jul 19 2013, 17:45
Post #7





Group: Members
Posts: 560
Joined: 11-March 07
Member No.: 41384



I'd be for such an option as well. The variation in perceived volume between lossy and lossless shouldn't be that big anyway? Or is it easily perceivable depending on music type?
Go to the top of the page
+Quote Post
EpicForever
post Jul 19 2013, 17:51
Post #8





Group: Members
Posts: 724
Joined: 14-September 11
From: Szczecin, PL
Member No.: 93712



No, it isn't. The amount of signal energy lost while lossy transcoding isn't that much - at least when you produce "listenable" lossy files (like at least 192 kbps mp3) that cut off at 19 kHz and don't cut too much in pass band. But if you remove 95% of bandwidth when converting to 1 kbps it can be completely different smile.gif

This post has been edited by EpicForever: Jul 19 2013, 17:51
Go to the top of the page
+Quote Post
Porcus
post Jul 19 2013, 18:15
Post #9





Group: Members
Posts: 1995
Joined: 30-November 06
Member No.: 38207



Another vote for tag transfer:
Sometimes one has scanned different parts of an album separately, say if a live album has bonus tracks that are not equally loud. Better transfer tags than giving up that grouping.


--------------------
One day in the Year of the Fox came a time remembered well
Go to the top of the page
+Quote Post
db1989
post Jul 19 2013, 18:18
Post #10





Group: Super Moderator
Posts: 5275
Joined: 23-June 06
Member No.: 32180



Others can debate lossy formats if they wish. At the very least, I canít think of any reason to omit to copy the tags between lossless formats.

In case someone mentions that foobar2000 used to use the original ReplayGain algorithm but now uses EBU R128, (i) the switch was made over two years ago, and (ii) users can easily re-scan the files before or after converting, and an automatic transfer would not hinder this ability or otherwise alter the workflow whatsoever.
Go to the top of the page
+Quote Post
Xanikseo
post Jul 20 2013, 16:33
Post #11





Group: Members
Posts: 27
Joined: 14-April 09
Member No.: 68951



Yes, the option of transferring gain tags between lossless formats should be a human right biggrin.gif! I am sure a great many people would be extremely grateful for the option to transfer gain tags between lossless formats smile.gif!

For lossless->lossy, perhaps a warning message next to the checkbox saying something like "Warning: Gain data will not be 100% accurate after encoding to lossy"? Surely, the user can decide on their own whether to be so intrepid as to go against the aversions of the foobar developers to the transferring gain tags between lossless and lossy; I am sure not too many of the user base will become too bellicose tongue.gif.

This post has been edited by Xanikseo: Jul 20 2013, 16:33
Go to the top of the page
+Quote Post
BenB
post Jul 20 2013, 19:05
Post #12





Group: Members
Posts: 778
Joined: 17-April 12
Member No.: 98921



For those utilizing a replaygain option involving "prevent clipping according to peak", it's better that the peak information be accurate.
Go to the top of the page
+Quote Post
ChronoSphere
post Jul 20 2013, 19:52
Post #13





Group: Members
Posts: 560
Joined: 11-March 07
Member No.: 41384



QUOTE (Xanikseo @ Jul 20 2013, 17:33) *
For lossless->lossy, perhaps a warning message next to the checkbox saying something like "Warning: Gain data will not be 100% accurate after encoding to lossy"? Surely, the user can decide on their own whether to be so intrepid as to go against the aversions of the foobar developers to the transferring gain tags between lossless and lossy; I am sure not too many of the user base will become too bellicose tongue.gif.
Considering they warn but still allow transcoding lossy -> lossy, I don't see why the same cannot be done for lossless -> lossy replay gain trasfer, which would also serve as a warning in the case mentioned by BenB
Go to the top of the page
+Quote Post
2Bdecided
post Jul 21 2013, 15:34
Post #14


ReplayGain developer


Group: Developer
Posts: 5364
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



I have always wanted this option for lossless and lossy.

For lossy, an option to only re-scan peak but not replaygain should also be included. It is only the change in peak that is EVER of significance when using good quality lossy encoding (unless you add something to intentionally scale the audio).

Cheers,
David.
Go to the top of the page
+Quote Post
Case
post Jul 21 2013, 17:05
Post #15





Group: Developer (Donating)
Posts: 2296
Joined: 19-October 01
From: Finland
Member No.: 322



Current alpha version has option to copy ReplayGain data, but peak information is dropped for lossy targets. I think some good arguments are required to change that decision, especially as foobar's ReplayGain scanner is barely slower than pure file decoding.
Go to the top of the page
+Quote Post
2Bdecided
post Jul 22 2013, 09:46
Post #16


ReplayGain developer


Group: Developer
Posts: 5364
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



QUOTE (Case @ Jul 21 2013, 17:05) *
Current alpha version has option to copy ReplayGain data, but peak information is dropped for lossy targets. I think some good arguments are required to change that decision, especially as foobar's ReplayGain scanner is barely slower than pure file decoding.
I see your point, and agree.

Anyone know what players do if they find ReplayGain data without peak data? I've never tried that. I've tried wrong data (with usually predictable results), but never no data.

Cheers,
David.
Go to the top of the page
+Quote Post
Peter
post Jul 22 2013, 12:19
Post #17


foobar2000 developer


Group: Admin
Posts: 3314
Joined: 30-September 01
Member No.: 84



QUOTE (2Bdecided @ Jul 21 2013, 16:34) *
I have always wanted this option for lossless and lossy.

For lossy, an option to only re-scan peak but not replaygain should also be included. It is only the change in peak that is EVER of significance when using good quality lossy encoding (unless you add something to intentionally scale the audio).

Cheers,
David.

My point of view:
ReplayGain scanner is pretty much always bottlenecked by reading+decoding now. Scanning for peaks alone will be somewhat faster than recalculating peaks+gain - but not much faster and usually HDD bottlenecked anyway.
Go to the top of the page
+Quote Post
ChronoSphere
post Jul 22 2013, 14:33
Post #18





Group: Members
Posts: 560
Joined: 11-March 07
Member No.: 41384



QUOTE (Case @ Jul 21 2013, 18:05) *
I think some good arguments are required to change that decision, especially as foobar's ReplayGain scanner is barely slower than pure file decoding.

My argument would be convenience, if the user is fine with the transferred peak not being as accurate as re-calculated peak, in turn not having to load files back to foobar and run the scan before transferring to a mobile device (or worse, having to scan from the mobile device), let them.

I see no harm in doing so, especially since foobar is all about options. It could even be a "hidden" tickbox in advanced preferences, but please let us do that (both lossy and lossless transfer)
Go to the top of the page
+Quote Post
Porcus
post Jul 22 2013, 14:56
Post #19





Group: Members
Posts: 1995
Joined: 30-November 06
Member No.: 38207



QUOTE (Peter @ Jul 22 2013, 13:19) *
ReplayGain scanner is pretty much always bottlenecked by reading+decoding now.


... so, that means that the option to RG scan after conversion is - provided no DSPs are applied - inefficient, as it would take almost no time had the scanning been done while the files were read first time? ;-)


--------------------
One day in the Year of the Fox came a time remembered well
Go to the top of the page
+Quote Post
ChronoSphere
post Jan 21 2014, 23:22
Post #20





Group: Members
Posts: 560
Joined: 11-March 07
Member No.: 41384



QUOTE (Case @ Jul 21 2013, 17:05) *
Current alpha version has option to copy ReplayGain data, but peak information is dropped for lossy targets. I think some good arguments are required to change that decision, especially as foobar's ReplayGain scanner is barely slower than pure file decoding.
After running into issues with rockbox not wanting to apply RG when peak tags are not present, I had a talk with rockbox devs on irc.
In the end I was told that files containing gain, but not peak tags are not conforming with the spec as said here.

That section is rather vague, if the peak tag is needed for volume normalization, then why isn't it listed in the corresponding section? And if it's not, then why does the spec say you need 4 tags?

As far as I see it, you actually only need one tag for volume normalization - either track OR album gain.
Additionally, if the user needs clipping protection, you need the corresponding peak tag - so it's a pair at most, not four of them.

As it stands now, I don't know who is right and who is wrong, since each side seems to think they're in the right.

Go to the top of the page
+Quote Post
2Bdecided
post Jan 22 2014, 11:21
Post #21


ReplayGain developer


Group: Developer
Posts: 5364
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



If you have a positive ReplayGain value, or a positive pre-amp value applied, or lossy coding that increases the peak, then simplistically you need the peak value to know if you're going to clip or not.

I think the rockbox audio pipeline might have more flexibility than most and be able to handle clipping smartly, but even so, it's reasonable to want the peak value.

It should be able to cope with only track_gain + track_peak, or only album_gain + album_peak being present. It should also fallback to which ever pair is in the tag (if there's only one pair in there) when the user has asked for the other one, rather than not applying ReplayGain at all. e.g. if user asks for album gain, but only track gain is tagged, use track gain, not nothing.


QUOTE (Xanikseo @ Jul 18 2013, 15:47) *
It seems there are not supposed to be any peak tags as per recommendation from xiph: http://wiki.xiph.org/OggOpus#Comment_Header

I didn't read this properly before, and I'm not up to speed on Opus, but unless I'm missing something they're making a mistake here. It's fine to do loudness normalisation based entirely on EBU R128, but it'll work far better if you have peak values, and clearly defined fields for track values, album values, and (if you choose to apply the gain, and that can be done losslessly like in mp3gain) "undo" values. Maybe they're trying to enforce EBU R128 album gain by default, which is a noble aim - but as one audio format in a sea of audio formats, that might be a step too far. I don't know. Is the peak value stored elsewhere? If not, it's going to make life difficult on DAPs.

Cheers,
David.
Go to the top of the page
+Quote Post
ChronoSphere
post Jan 22 2014, 13:53
Post #22





Group: Members
Posts: 560
Joined: 11-March 07
Member No.: 41384



QUOTE (2Bdecided @ Jan 22 2014, 11:21) *
If you have a positive ReplayGain value, or a positive pre-amp value applied, or lossy coding that increases the peak, then simplistically you need the peak value to know if you're going to clip or not.
Correct, but what if you don't care about clipping? Then peak should be irrelevant, should it not?

QUOTE
I think the rockbox audio pipeline might have more flexibility than most and be able to handle clipping smartly, but even so, it's reasonable to want the peak value.

It should be able to cope with only track_gain + track_peak, or only album_gain + album_peak being present. It should also fallback to which ever pair is in the tag (if there's only one pair in there) when the user has asked for the other one, rather than not applying ReplayGain at all. e.g. if user asks for album gain, but only track gain is tagged, use track gain, not nothing.
That's exactly what it does, which makes perfectly sense, the only case where it does not (for me), is that in case there is no peak, imho you should just skip the verification of whether it clips or not, instead of dropping RG completely.

Their argument is that the spec requires you to have a pair of peak+gain to do anything - which doesn't make sense to me in this particular scenario. Basically what I'm asking is whether it really needs peak for pure volume normalization, and if not, then whether the specification text should be changed to clarify that.

This post has been edited by ChronoSphere: Jan 22 2014, 13:55
Go to the top of the page
+Quote Post
2Bdecided
post Jan 22 2014, 15:00
Post #23


ReplayGain developer


Group: Developer
Posts: 5364
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



QUOTE (ChronoSphere @ Jan 22 2014, 12:53) *
what if you don't care about clipping? Then peak should be irrelevant, should it not?
It's irrelevant to you. I totally take your point, but they might have other concerns. e.g. the rockbox devs might feel it's a useful check of the tag's validity (irrespective of whether you're using the peak data or not), or might have written the code to only work with peak data present.

The original spec didn't anticipate there not being peak data available, and assumed players would use it by default...
http://replaygain.hydrogenaudio.org/proposal/player.html
...and when the idea of copying ReplayGain data from lossless to lossy files first came up, the suggestion was to store the fact it was approximate...
http://www.hydrogenaudio.org/forums/index....showtopic=15445
(item 3) rather than to drop the peak values entirely. That "it's approximate" flag has never been implemented AFAIK, and ReplayGain scanning is a lot faster in 2014 than 2003.

I don't know about Opus, but on any other format, if you've somehow ended up with a collection of files with ReplayGain tags but no peak values, and you don't care about peak values, I think you can drag your entire collection into mp3tag (or even fb2k), add REPLAYGAIN_TRACK_PEAK=1.000000 and REPLAYGAIN_ALBUM_PEAK=1.000000 tags to all your files, and everything will work as you want in your rockboxed player.

I now see the problem with Opus was raised a long time ago...
http://www.hydrogenaudio.org/forums/index....showtopic=97357
https://www.ietf.org/mail-archive/web/codec...t/msg02935.html
...I wish I'd had the time a few years back to take an active role in this. It seems it's too late now? Having to decide before encoding whether to hard-code album gain into the encoding is just wrong. Fine as an option; really poor as the only way a specific audio format allows album-based loudness normalisation! Yet there's a note that ReplayGain is half-complete for Opus...
http://wiki.xiph.org/OPUS_TODO
...have they done it "correctly"? I can't find any other info.

FWIW the way some ReplayGain tags are now written with a -23LUFS reference level is also a mistake I wish I'd prevented. Oh well. People got hung up on the original calculation (which was always intended to be improved), rather than the point...
http://www.hydrogenaudio.org/forums/index....mp;#entry736023

Cheers,
David.
P.S. sorry for derailing this thread. I should have searched for the others earlier.

This post has been edited by 2Bdecided: Jan 22 2014, 15:22
Go to the top of the page
+Quote Post
ChronoSphere
post Jan 22 2014, 15:34
Post #24





Group: Members
Posts: 560
Joined: 11-March 07
Member No.: 41384



QUOTE (2Bdecided @ Jan 22 2014, 15:00) *
It's irrelevant to you.
Or anyone who only wants to have volume normalization only... but I get what you are saying.

QUOTE
I don't know about Opus, but on any other format, if you've somehow ended up with a collection of files with ReplayGain tags but no peak values, and you don't care about peak values, I think you can drag your entire collection into mp3tag (or even fb2k), add REPLAYGAIN_TRACK_PEAK=1.000000 and REPLAYGAIN_ALBUM_PEAK=1.000000 tags to all your files, and everything will work as you want in your rockboxed player.
The main concern here is that applying tags on a relatively slow SD card, or even slower portable player memory is something that takes about the same amount of time as transcoding itself.

"One-click-convert" solution instead of having to rescan or add tags manually is what I'd ideally want to have. Else I might as well just continue copying the FLAC files over, which is a "one-click-copy" solution which takes about the same amount of time (few secs longer) but doesn't involve any interaction from me. Applying track/album gain on encode is less than ideal, because then you can't have a "track gain when shuffling, album gain otherwise"-type of flexibility anymore.

On a slightly different note, I thought about suggesting calculating peak tags on-the-fly, but that would only work for track peak, as you can't reliably detect which tracks belong to an album and whether the album is complete or not, so even going in that direction wouldn't change much. And switching to album gain would be impossible because peak is still missing for it.

What about an option to "add custom tags" in the converter? That's where people who don't care about peak could just make a preset which sets the peak to 1.0 and move on.

This post has been edited by ChronoSphere: Jan 22 2014, 15:36
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 27th December 2014 - 22:06