IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
lame 3.98 alpha 2 testing thread, vbr new & athaa-sensitivity test at V5
guruboolez
post Oct 17 2005, 13:53
Post #1





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



I must be brief.
Here are the results of a blind listening tests performed on 150 samples, classical music only. I planned to use other samples, but my week-end is not unlimited.

• 3.98 alpha 2 -V5
• 3.98 alpha 2 -V5 --vbr-new
• 3.98 alpha 2 -V5 --vbr-new --athaa-sensitivity 1


Results
:









ABX logs:
http://audiotests.free.fr/tests/2005.10/LAME_398_V5.7z

Bitrate table:

http://audiotests.free.fr/tests/2005.10...Q%20Analysis.7z



I can comment it (I'm very sorry about, but there some troubles about some weird usage of internet here at my working place, and internet is currently "under control").


• --vbr-new is apparently same to use with -V5 profile
• --athaa-sensitivity really help to lower the level of annoyance (ringing mostly).
It's maybe not the right answer to current quality problems occuring with "pure" V5 preset, but the fix is very helpful and doesn't cost too much (~6 kbps on average).

This post has been edited by guruboolez: Dec 29 2005, 22:36
Go to the top of the page
+Quote Post
Synthetic Soul
post Oct 17 2005, 14:26
Post #2





Group: Super Moderator
Posts: 4887
Joined: 12-August 04
From: Exeter, UK
Member No.: 16217



Wow, your result formatting just gets better and better. Your tests have always been exemplary of course. smile.gif

I thought --athaa-sensitivity had gone. I know you were querying the decision.

Does this mean it's back in for 3.98 or was never out?

As a -V5 user, thank you very much for your time.

Edit: OK, discussions mentioned above are here. A spinoff thread is here. It is out in 3.97b1 but remains in alpha versions.

My re-reading of these previous threads leads me to believe that this test is to keep this discussion alive, and to attempt to find a resolve for the "requirement" for --athaa-sensitivity with -V5.

Good luck to guruboolez and Gabriel.

This post has been edited by Synthetic Soul: Oct 17 2005, 15:17


--------------------
I'm on a horse.
Go to the top of the page
+Quote Post
shadowking
post Oct 17 2005, 14:31
Post #3





Group: Members
Posts: 1529
Joined: 31-January 04
Member No.: 11664



An Xcellent read as always. What exactly are the changes from 3.97b1 > 3.98b2 ?


--------------------
Wavpack -b450s0.7
Go to the top of the page
+Quote Post
Gabriel
post Oct 17 2005, 15:11
Post #4


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



QUOTE
I thought --athaa-sensitivity had gone. I know you were querying the decision.

Does this mean it's back in for 3.98 or was never out?

As other experimental switches, it is available in alpha and debug compiles.
Go to the top of the page
+Quote Post
Synthetic Soul
post Oct 17 2005, 15:39
Post #5





Group: Super Moderator
Posts: 4887
Joined: 12-August 04
From: Exeter, UK
Member No.: 16217



Thanks Gabriel. I found that out myself as well. I decided I should do as I wish others would do sometimes, and actaully test instead of just asking for the answer.

Is 0.59 a marked improvement when talking about ABX? I remember guruboolez once stating that a 4.0 is "perceptible but not annoying". What is 3.0 please? "perceptible and annoying"? NB: I have tried to find the answer to this one.

I seem to remember seeing before that, if the two lines do not overlap, it can be said that there is a definite difference. They don't overlap in any of the subtotals. I guess this is why guruboolez makes the brief comments he has.


--------------------
I'm on a horse.
Go to the top of the page
+Quote Post
stephanV
post Oct 17 2005, 15:44
Post #6





Group: Members
Posts: 394
Joined: 6-May 04
Member No.: 13932



5.0 = imperceptible
4.0 = perceptible but not annoying
3.0 = perceptible and slighly annoying
2.0 = annoying
1.0 = very annoying


--------------------
"We cannot win against obsession. They care, we don't. They win."
Go to the top of the page
+Quote Post
guruboolez
post Oct 17 2005, 15:52
Post #7





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



Few words about my notation principles.

When a small difference was found= +/- 0.5 point
When a serious difference was heard = +1 point or more
When a very small or unsure difference was heard = < +/- 0.5 point.

Therefore, don't take too much attention on the absolute notation (i.e. it's not totally linked to ITU reference scale).


What matters is:
1/ that --athaa-sensitivity 1 bring more improvement than --vbr-new over defaulted VBR mode
2/ that --athaa-sensitivity could highly reduce severe audible flaws (high level of ringing/noise reduction with -V5/-V5 --vbr-new).
Go to the top of the page
+Quote Post
Synthetic Soul
post Oct 17 2005, 17:16
Post #8





Group: Super Moderator
Posts: 4887
Joined: 12-August 04
From: Exeter, UK
Member No.: 16217



Thanks stephenV.

Argh. Just as we get a new HA recommendation and I consider re-encoding all my MP3s (a shocking amount of which are Blade and Xing) up pops this development.

3.97b1 or 3.98a2 --athaa-sensitivity 1? Or do I wait until this anomaly has been properly diagnosed and ironed out?

I think I'm going to have to wait and see how this one progresses...


--------------------
I'm on a horse.
Go to the top of the page
+Quote Post
robert
post Oct 17 2005, 22:42
Post #9


LAME developer


Group: Developer
Posts: 789
Joined: 22-September 01
Member No.: 5



@guruboolez
is it possible to get one or two of your test samples?
Go to the top of the page
+Quote Post
Gambit
post Oct 17 2005, 22:52
Post #10


Burrrn developer


Group: Developer
Posts: 917
Joined: 25-November 01
From: Bratislava, Slovakia
Member No.: 534



QUOTE (robert @ Oct 17 2005, 11:42 PM)
@guruboolez
is it possible to get one or two of your test samples?
*

Not sure if those are the used ones, but some of guru's samples are here:
ftp://ftp2.foobar2000.net/foobar/


--------------------
Burrrn - http://www.burrrn.net/
MPEG Audio Collection - http://mac.sourceforge.net/
Go to the top of the page
+Quote Post
guruboolez
post Oct 18 2005, 09:52
Post #11





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



QUOTE (robert @ Oct 17 2005, 10:42 PM)
@guruboolez
is it possible to get one or two of your test samples?
*

The 150 samples are all downloadable:
EDIT: links have changed

This post has been edited by guruboolez: Dec 29 2005, 22:37
Go to the top of the page
+Quote Post
guruboolez
post Oct 18 2005, 09:56
Post #12





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



Some stats:



CODE
-V5 --vbr-new with and without --athaa-sensitivity 1

when difference is from: -1.0 to 0.4 point    (70 samples)    +2,84 kbps  in average with --athaa..1
when difference is from:  0.5 to 0.9 point    (29 samples)    +4,17 kbps  in average with --athaa..1
when difference is from:  1.0 to 1.4 points    (23 samples)   +7,70 kbps  in average with --athaa..1
when difference is from:  1.5 to 1.9 points    (13 samples)   +8,69 kbps  in average with --athaa..1
when difference is from:  2.0 to 2.4 points    (09 samples)   +14,44 kbps in average with --athaa..1
when difference is from:  2.5 points and more    (05 samples)    +19,00 kbps in average with --athaa..1



CONCLUSION: the perceptual quality improvment of --ahtaa-sensitivity 1 has a direct relation with the amount of bit added by this command line. When --ahtaa-sensitivity 1 has no impact on bitrate it has no impact on quality; when --ahtaa-sensitivity 1 "bloats" the bitrate it also "bloats" the quality.

10% of the tested samples reveal a tendency to "bitrate bloat" (+15 kbps or more). For these 14 samples, the average notation is going from 2.3 points (without --ahtaa-sensitivity 1) to 4.3 points (with --ahtaa-sensitivity 1).

____
Appendix:
RATIO
CODE
[average notation diff in points] / [average bitrate diff in kbps]

-0.01 / 2.84  =-0.0020
0.53 / 4.17  = 0.1264
1.03 / 4.70  = 0.1333
1.56 / 8.69  = 0.1796
2.00 / 14.44 = 0.1384
2.70 / 19.00 = 0.142
1

This post has been edited by guruboolez: Oct 18 2005, 09:59
Go to the top of the page
+Quote Post
Brigstocke
post Oct 19 2005, 06:30
Post #13





Group: Members
Posts: 8
Joined: 20-March 05
Member No.: 20758



Well, it looks like there's very very few instances of regression and that any increase in bitrate is well justified. Since I haven't done any testing myself, I can only say thank you very much for doing so much testing!

This post has been edited by Brigstocke: Oct 19 2005, 06:30
Go to the top of the page
+Quote Post
mad_arab
post Oct 23 2005, 11:58
Post #14





Group: Members
Posts: 87
Joined: 6-November 03
Member No.: 9691



3.97 was recommended while still in alpha stage. Are there any significant improvements that would make 3.98 a good choice yet?
Go to the top of the page
+Quote Post
Cartoon
post Oct 23 2005, 13:29
Post #15





Group: Members
Posts: 44
Joined: 6-May 05
From: Norway
Member No.: 21907



No, 3.97 was not recommended until it hit beta. Even that caused some noise, as in principle it should be GA code. But as the 3.x tree is pretty mature now, it is pretty safe.

Besides, it is just a recommendation, not the law smile.gif
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 20th December 2014 - 20:10