IPB

Welcome Guest ( Log In | Register )

2 Pages V   1 2 >  
Reply to this topicStart new topic
LAME 3.96b1 vs. 3.90.3 Discussion, Discussion Thread
tigre
post Mar 17 2004, 19:22
Post #1


Moderator


Group: Members
Posts: 1434
Joined: 26-November 02
Member No.: 3890



This thread has been split from a lame 3.90.3 vs. 3.96b1 test thread. It's for discussion about LAME 3.96b1 vs. 3.90.3, A new 'recommended version' for HA.org?! only. This thread is containing discussion related to the test too, so before posting here, you might want to read it.


--------------------
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello
Go to the top of the page
+Quote Post
emtee
post Mar 18 2004, 15:25
Post #2





Group: Members
Posts: 198
Joined: 18-October 02
Member No.: 3569



Shouldn't we wait for the developers to improve some of those regression examples? That topic had a lot of replies, and i'm sure the feedback will be useful to improve the codec. Maybe we should wait for a 3.96 beta 2 with some bugfixes before comparing it with the mighty 3.90.3, otherwise the regression thread was in vain.
Go to the top of the page
+Quote Post
john33
post Mar 18 2004, 16:31
Post #3


xcLame and OggDropXPd Developer


Group: Developer
Posts: 3761
Joined: 30-September 01
From: Bracknell, UK
Member No.: 111



I believe I'm correct in saying that 3.96 is scheduled to go final at the end of this month and that any further commits will be bugfixes only and not any further tuning, etc., at this stage. Gabriel, correct me if I'm wrong. wink.gif


--------------------
John
----------------------------------------------------------------
My compiles and utilities are at http://www.rarewares.org/
Go to the top of the page
+Quote Post
dev0
post Mar 18 2004, 16:58
Post #4





Group: Developer
Posts: 1679
Joined: 23-December 01
From: Germany
Member No.: 731



Testing has to be done anyway and if another version is released meanwhile another testing thread will be opened, it's as simple as that.
Go to the top of the page
+Quote Post
amano
post Mar 19 2004, 17:07
Post #5





Group: Members
Posts: 483
Joined: 1-December 02
Member No.: 3949



One problem is that with 3.96b bitrates aren't comparable with 3.90.3 anymore. With --preset 128 the difference between 116 and 138 kbps is quite significant. With --preset standard the differences are much bigger, up to 50 kbps.

Isn't there any chance for gabriel and the other devs to tune the presets to be comparable in size (at least the target kbps presets).

Other way I forsee the result that all samples are significantly smaller, but tend to sound worse. Then the results don't tell anything because the quality/size ratio is what counts and can only be guessed that way.

This post has been edited by amano: Mar 19 2004, 17:09
Go to the top of the page
+Quote Post
dev0
post Mar 19 2004, 17:13
Post #6





Group: Developer
Posts: 1679
Joined: 23-December 01
From: Germany
Member No.: 731



I assumed that --preset standard is still ment to be transparent on 99% of all samples, just like 3.90.3 --alt-preset standard.
The bitrate is a rather uninteresting factor here.


--------------------
"To understand me, you'll have to swallow a world." Or maybe your words.
Go to the top of the page
+Quote Post
kwanbis
post Mar 19 2004, 17:21
Post #7





Group: Developer (Donating)
Posts: 2362
Joined: 28-June 02
From: Argentina
Member No.: 2425



QUOTE (dev0 @ Mar 19 2004, 04:13 PM)
I assumed that --preset standard is still ment to be transparent on 99% of all samples, just like 3.90.3 --alt-preset standard.
The bitrate is a rather uniniteresting factor here.

totally agree on that one wink.gif


--------------------
MAREO: http://www.webearce.com.ar
Go to the top of the page
+Quote Post
evereux
post Mar 19 2004, 17:42
Post #8





Group: Members
Posts: 907
Joined: 9-February 02
From: Cheshire, UK
Member No.: 1296



QUOTE (dev0 @ Mar 19 2004, 04:13 PM)
I assumed that --preset standard is still ment to be transparent on 99% of all samples, just like 3.90.3 --alt-preset standard.
The bitrate is a rather uninteresting factor here.

I whole-heartedly dissagree. I don't think it's a good idea to have a new recommended Lame version that acheives transparency at the cost of file size (if that is indeed what happens on average) in comparison to an encoder that acheives transparency with a smaller file. Especially when that Lame version has many samples where it has regressed from 3.90.3 even in it's wrongly compiled state?


--------------------
daefeatures.co.uk
Go to the top of the page
+Quote Post
dev0
post Mar 19 2004, 17:53
Post #9





Group: Developer
Posts: 1679
Joined: 23-December 01
From: Germany
Member No.: 731



If it saves bitrate that's a nice bonus, but the new recommended version should be at least as transparent using --preset standard as 3.90.3 is now.
Sacrificing quality for bitrate is not an option!
Go to the top of the page
+Quote Post
Fede
post Mar 21 2004, 10:35
Post #10





Group: Members
Posts: 11
Joined: 8-February 02
Member No.: 1287



It's really progression to compress with the same sound quality at higher bitrate? It's progression to reach the same quality we can have @192kbps with the 3.90.3 at 256 kbps with another encoder???? If the answer is yes I think there's many other encoder working good.
We are talking about reach the best quality @ best size, if size isn't a priority go for lossless.

Just my 2 cents.
Go to the top of the page
+Quote Post
evereux
post Mar 21 2004, 14:26
Post #11





Group: Members
Posts: 907
Joined: 9-February 02
From: Cheshire, UK
Member No.: 1296



I see some samples are being tested with 3.90.2 (the user hasn't stated whether -Z was used) and not 3.90.3. Whilst this goes against the test requirements I actually think this is a better idea since we have little idea of what effect the compilation issues has had on 3.90.3 (I'm referring to this discussion).


--------------------
daefeatures.co.uk
Go to the top of the page
+Quote Post
[proxima]
post Mar 21 2004, 14:37
Post #12





Group: Members
Posts: 197
Joined: 12-October 02
From: Italy
Member No.: 3537



QUOTE (evereux @ Mar 21 2004, 02:26 PM)
I see some samples are being tested with 3.90.2 (the user hasn't stated whether -Z was used) and not 3.90.3.

if you are referring to me, i tested 3.90.2 only with abr 128 presets, so -Z is not included in 3.90.3.
I've only tested rebel.wav with --aps -Z and 3.90.2 and i specified this in my post.
QUOTE
Whilst this goes against the test requirements I actually think this is a better idea since we have little idea of what effect the compilation issues has had on 3.90.3

I think compilation issues should be very small since john33 compile include the same switches used by Dibrom. Nevertheless i agree with you, this is the reason because i tested 3.90.2 (with -Z when needed) wink.gif

This post has been edited by [proxima]: Mar 21 2004, 14:38


--------------------
WavPack 4.3 -mfx5
LAME 3.97 -V5 --vbr-new --athaa-sensitivity 1
Go to the top of the page
+Quote Post
tigre
post Mar 21 2004, 16:32
Post #13


Moderator


Group: Members
Posts: 1434
Joined: 26-November 02
Member No.: 3890



Some clarifications about ABX tests in this test:

Especially at low bitrates like ~128kbps many artifacts are so obvious that ABXing appears as waste of time, but even if it doesn't give the tester any additional security about the result, ...
- it gives security to others that a result is valid AND
- you can be sure that artifacts in one encoded version are really different from the ones in another version if you ABX two encoded versions against each other

To prove that one encoded version is worse than the other in such obvious cases, it should be enough to ABX
- Original vs. worse sounding encoded version
- better sounding vs. worse sounding encoded version

ABC/HR results without ABXing (especially the encoded versions that seem to be different against each other) can't be used for public tests like this because we need at least a minimum security (=ABX results) that the individual results of the participants are reliable. It would be different, if a big number of participants submitted ABC/HR results of the same sample(s). In this case the results could be evaluated using a statistical analysis similar to rjamorim's tests.


--------------------
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello
Go to the top of the page
+Quote Post
Jojo
post Mar 21 2004, 17:02
Post #14





Group: Members
Posts: 1361
Joined: 25-November 02
Member No.: 3873



QUOTE (evereux @ Mar 21 2004, 05:26 AM)
I see some samples are being tested with 3.90.2 (the user hasn't stated whether -Z was used) and not 3.90.3. Whilst this goes against the test requirements I actually think this is a better idea since we have little idea of what effect the compilation issues has had on 3.90.3 (I'm referring to this discussion).

I agree to that in some extend. If the goal was just to get the closest to transparent the recommended setting should be --preset 320
But then, --preset 128 offers the best ratio of quality and size...so it's a really tricky question. However, I use LAME 3.95.1 since it uses lower bitrates than LAME 3.92 or LAME 3.90.3 in most cases. If Lame 3.96 is found to be just as good as LAME 3.90.3 it should be recommended since it's faster and uses as lower bitrate.


--------------------
--alt-presets are there for a reason! These other switches DO NOT work better than it, trust me on this.
LAME + Joint Stereo doesn't destroy 'Stereo'
Go to the top of the page
+Quote Post
Jojo
post Mar 21 2004, 17:08
Post #15





Group: Members
Posts: 1361
Joined: 25-November 02
Member No.: 3873



QUOTE (dev0 @ Mar 19 2004, 08:53 AM)
If it saves bitrate that's a nice bonus, but the new recommended version should be at least as transparent using --preset standard as 3.90.3 is now.
Sacrificing quality for bitrate is not an option!

Why isn't --preset 320 recommended then laugh.gif


--------------------
--alt-presets are there for a reason! These other switches DO NOT work better than it, trust me on this.
LAME + Joint Stereo doesn't destroy 'Stereo'
Go to the top of the page
+Quote Post
PowerMacG4
post Mar 21 2004, 17:15
Post #16





Group: Members
Posts: 40
Joined: 25-January 04
From: Over there
Member No.: 11495



QUOTE (Jojo @ Mar 21 2004, 08:08 AM)
QUOTE (dev0 @ Mar 19 2004, 08:53 AM)
If it saves bitrate that's a nice bonus, but the new recommended version should be at least as transparent using --preset standard as 3.90.3 is now.
Sacrificing quality for bitrate is not an option!

Why isn't --preset 320 recommended then laugh.gif

Because "--preset-insane" wastes bitrate without much gain in quality at all. --preset-standard is designed to achieve transparency on a vast majority of samples at the lowest bitrate possible. Transparency doesn't mean "kinda close". It means "transparent".
Go to the top of the page
+Quote Post
evereux
post Mar 21 2004, 17:45
Post #17





Group: Members
Posts: 907
Joined: 9-February 02
From: Cheshire, UK
Member No.: 1296



QUOTE (PowerMacG4 @ Mar 21 2004, 04:15 PM)
QUOTE (Jojo @ Mar 21 2004, 08:08 AM)
QUOTE (dev0 @ Mar 19 2004, 08:53 AM)
If it saves bitrate that's a nice bonus, but the new recommended version should be at least as transparent using --preset standard as 3.90.3 is now.
Sacrificing quality for bitrate is not an option!

Why isn't --preset 320 recommended then laugh.gif

Because "--preset-insane" wastes bitrate without much gain in quality at all. --preset-standard is designed to achieve transparency on a vast majority of samples at the lowest bitrate possible. Transparency doesn't mean "kinda close". It means "transparent".

His point was, when do we draw the line and say that these bit-rates are becoming too inflated for a standard preset? Transparency in the true sense of the word will never be reached.

That being said, I'm in the process of encoding around 20GBs of wav files and the file size increase doesn't look alarming at all. I'll be more specific when the encoding is complete.


--------------------
daefeatures.co.uk
Go to the top of the page
+Quote Post
evereux
post Mar 21 2004, 17:48
Post #18





Group: Members
Posts: 907
Joined: 9-February 02
From: Cheshire, UK
Member No.: 1296



QUOTE ([proxima)
,Mar 21 2004, 01:37 PM]
QUOTE (evereux @ Mar 21 2004, 02:26 PM)
I see some samples are being tested with 3.90.2 (the user hasn't stated whether -Z was used) and not 3.90.3.

if you are referring to me, i tested 3.90.2 only with abr 128 presets, so -Z is not included in 3.90.3.
I've only tested rebel.wav with --aps -Z and 3.90.2 and i specified this in my post.

You are quite right, my apologies. smile.gif


--------------------
daefeatures.co.uk
Go to the top of the page
+Quote Post
evereux
post Mar 21 2004, 23:21
Post #19





Group: Members
Posts: 907
Joined: 9-February 02
From: Cheshire, UK
Member No.: 1296



I've encoded 22.4GB of wav files using the LAME versions 3.96b1 (--preset standard) and 3.90.3 (--alt-preset standard). The resultant directory sizes are as follows:

3.90.3 = 3.24GB
3.96b1 = 2.99GB

I can provide an album list if required (I first have some sorting to do to ease the compilation of it).


--------------------
daefeatures.co.uk
Go to the top of the page
+Quote Post
Jebus
post Mar 22 2004, 02:22
Post #20





Group: Developer
Posts: 1320
Joined: 17-March 03
From: Calgary, AB
Member No.: 5541



QUOTE (evereux @ Mar 21 2004, 08:45 AM)
QUOTE (PowerMacG4 @ Mar 21 2004, 04:15 PM)
QUOTE (Jojo @ Mar 21 2004, 08:08 AM)
QUOTE (dev0 @ Mar 19 2004, 08:53 AM)
If it saves bitrate that's a nice bonus, but the new recommended version should be at least as transparent using --preset standard as 3.90.3 is now.
Sacrificing quality for bitrate is not an option!

Why isn't --preset 320 recommended then laugh.gif

Because "--preset-insane" wastes bitrate without much gain in quality at all. --preset-standard is designed to achieve transparency on a vast majority of samples at the lowest bitrate possible. Transparency doesn't mean "kinda close". It means "transparent".

His point was, when do we draw the line and say that these bit-rates are becoming too inflated for a standard preset? Transparency in the true sense of the word will never be reached.

That being said, I'm in the process of encoding around 20GBs of wav files and the file size increase doesn't look alarming at all. I'll be more specific when the encoding is complete.

no, your definition of "transparency" is wrong. Transparency does not mean bitwise equivelency, it means the files are audibly indistinguishable. This is the case with --alt-preset standard, or at least the intention. This is ALSO the case with --alt-preset insane, but the extra bits don't make the files sound any better since transparency was already acheived using --alt-preset standard.
Go to the top of the page
+Quote Post
evereux
post Mar 23 2004, 14:15
Post #21





Group: Members
Posts: 907
Joined: 9-February 02
From: Cheshire, UK
Member No.: 1296



QUOTE (Jebus @ Mar 22 2004, 01:22 AM)
no, your definition of "transparency" is wrong. Transparency does not mean bitwise equivelency, it means the files are audibly indistinguishable. This is the case with --alt-preset standard, or at least the intention. This is ALSO the case with --alt-preset insane, but the extra bits don't make the files sound any better since transparency was already acheived using --alt-preset standard.

I didn't define transparency, I said in the true sense of the word. Using your analogy, what then is the point of the insane preset? I still think the whole point is being missed and for me this discussion is redundant now anyway.


--------------------
daefeatures.co.uk
Go to the top of the page
+Quote Post
evereux
post Mar 23 2004, 14:19
Post #22





Group: Members
Posts: 907
Joined: 9-February 02
From: Cheshire, UK
Member No.: 1296



QUOTE (evereux @ Mar 21 2004, 10:21 PM)
I've encoded 22.4GB of wav files using the LAME versions 3.96b1 (--preset standard) and 3.90.3 (--alt-preset standard). The resultant directory sizes are as follows:

3.90.3 = 3.24GB
3.96b1 = 2.99GB

I can provide an album list if required (I first have some sorting to do to ease the compilation of it).

Futher to this:

comparison

Be aware that the table is 412kb, I couldn't find a better way to trim the size down (html tables are crap, or at least my knowledge of them).

Generated from Encspot output original files here.


--------------------
daefeatures.co.uk
Go to the top of the page
+Quote Post
tigre
post Mar 23 2004, 14:43
Post #23


Moderator


Group: Members
Posts: 1434
Joined: 26-November 02
Member No.: 3890



Thanks evereux.

If possible, could you (or anyone else) please do a similar test with 3.96b1 VBR?

With 3.90.3 there are no recommended VBR settings for quality lower then --alt-preset standard -Y because ABR sounds better in most cases at comparable bitrate. AFAIK Gabriel has changed a lot to impove -V 3 and lower, so this could lead to 3.96b1 VBR replacing 3.90.3 ABR as recommended lame compile/settings combination for < 192kbps bitrates. To make ABR vs. VBR tests useful, we need some test similar to the one you did, evereux.

In theory V-settings should give those average bitrates:
-V 5 : 128kbps
-V 4 : 144kbps
-V 3 : 160kbps
-V 2 : 192kbps

With the samples I tried so far (mainly -V 5) the numbers have been 10-20kbps higher on average. So could you please do some mass-encoding at -V 2 - -V 6, maybe a (representative) part of the tracks you used for (preset)standard bitrate comparison is enough...


--------------------
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello
Go to the top of the page
+Quote Post
LoFiYo
post Mar 23 2004, 14:50
Post #24





Group: Members
Posts: 133
Joined: 2-January 04
Member No.: 10896



I think there should be an official guideline for a recommended version of LAME since HA will be endorsing it as its official recommendation. In other words what do HA admins consider a superior version of LAME to the current recommended version (3.90.3)?

I would think:
Version X is superior to Version Y when the sound quality of the encoded file is found to be higher 9 times (or more) out of 10 in all preset modes (from insane down to abr/cbr [x] kbps).

What would be the official HA stance on this? I am asking this, because we can keep testing and testing many many samples, but if there isn't a definition as to what constitutes a superior version, the test might never end...
Go to the top of the page
+Quote Post
evereux
post Mar 23 2004, 15:11
Post #25





Group: Members
Posts: 907
Joined: 9-February 02
From: Cheshire, UK
Member No.: 1296



QUOTE (tigre @ Mar 23 2004, 01:43 PM)
With the samples I tried so far (mainly -V 5) the numbers have been 10-20kbps higher on average. So could you please do some mass-encoding at -V 2 - -V 6, maybe a (representative) part of the tracks you used for (preset)standard bitrate comparison is enough...

Sure, I can do that. I'll start at V6, work my way down the numbers and post my results as they're completed.

If anyone else would like to start at V2 V3, please do so.

edit: V2 V3 was V2

This post has been edited by evereux: Mar 23 2004, 17:22


--------------------
daefeatures.co.uk
Go to the top of the page
+Quote Post

2 Pages V   1 2 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 22nd November 2014 - 04:16