IPB

Welcome Guest ( Log In | Register )

2 Pages V  < 1 2  
Reply to this topicStart new topic
lame 3.97 alpha 6 testing thread
guruboolez
post Jan 20 2005, 10:19
Post #26





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



QUOTE (Gabriel @ Jan 20 2005, 10:10 AM)
I am wondering if there was a regression between 3.90.3 and 3.96.1 and no one spotted it, or if there is a regression between 3.96.1 and 3.97a6.
*

I could check it (quickly) but I fear that I can't post results before next monday.
Go to the top of the page
+Quote Post
uart
post Jan 21 2005, 14:01
Post #27





Group: Members
Posts: 810
Joined: 23-November 04
Member No.: 18295



QUOTE
I could check it (quickly) but I fear that I can't post results before next monday.


Great smile.gif. I'm very interested in the results as I'm currently using 3.96.1 exclusively.
Go to the top of the page
+Quote Post
Gabriel
post Jan 22 2005, 19:11
Post #28


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



The q1 and q0 problem has now been identified and corrected, thank you very much for the report.
Go to the top of the page
+Quote Post
guruboolez
post Jan 24 2005, 08:41
Post #29





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



APPENDIX TO TEST#4: --preset standard

QUOTE (Gabriel @ Jan 20 2005, 10:10 AM)
I am wondering if there was a regression between 3.90.3 and 3.96.1 and no one spotted it, or if there is a regression between 3.96.1 and 3.97a6.
*




It appeared previously that lame 3.97a6 -V 2 quality was inferior to -V 2 --vbr new and also to lame 3.90.3 --preset standard. The purpose of the following test is to verify the encoder’s evolution, from 3.96.1 to 3.97a6, in order to see if the cause of the measured degradations occurred during this developing phase or if it happened before it.

I've used the same 20 samples again, but I didn't test the three ones which sounded identical during the last test (BachS1007.wav: too transparent, castanets2.wav: too smeared, thear1.wav: too deafening).


Results






CODE
ATrain          3,5      3,5
BachS1007        not tested      
BeautySlept     3,0      3,5
Blackwater      4,0      4,0
castanets2       not tested      
dogies          3,5      2,5
FloorEssence    4,0      3,0
fossiles        4,0      3,9
SinceAlways     3,0      4,0
Layla           3,0      2,5
LifeShatters    4,0      4,0
LisztBMinor     3,0      4,0
macabre         2,0      3,0
MidnightVoyage  3,0      3,5
Orion II (2.1)  3,5      3,5
rawhide         3,0      4,0
thear1           not tested      
TheSource       3,0      2,5
Waiting         2,0      2,0
wayitis         3,0      2,0

Click here for log files


Results are unclear. Changes between 3.96.1 and 3.97a6 are sometimes audible, but they are equally distributed between both encoders. 7 samples sounded identical; at least no difference appeared during ABX phase. 5 were better with 3.96.1 (confirmed by positive ABX results) and 5 were better with 3.97a6 (also confirmed).

It seems that differences heard previously between -V2 and -V2 --vbr-new aren't correlated to a bug introduced in 3.97 alphas. I would therefore conclude that 3.90.3 --preset standard is better than 3.96.1 3.97a6 at same preset and finally that 3.97a6 -V 2 --vbr-new beats them all.

Of course, this conclusion is based on a limited set of sample (most are not critical for lossy encoders) and on my current sensitivity. It would be really helpful I suppose to hear other results confirming or infirming my own results.

N.B. According to ANOVA & Tukey parametric analysis, both encoders are tied.


EDIT: replaced 3.96.1 (now crossed) by 3.97a6.

This post has been edited by guruboolez: Dec 29 2005, 23:07
Go to the top of the page
+Quote Post
Jebus
post Jan 26 2005, 00:11
Post #30





Group: Developer
Posts: 1294
Joined: 17-March 03
From: Calgary, AB
Member No.: 5541



I know this is just one listening test, but would it be possible to remap --preset standard to use --vbr-new if it is determined to be of higher quality, sort of like --r3mix was changed in the last version?
Go to the top of the page
+Quote Post
ezra2323
post Feb 1 2005, 02:14
Post #31





Group: Members
Posts: 586
Joined: 17-July 02
Member No.: 2631



Its been over a week with no news of any developments. Any progress being made?

Yes - I am impatient biggrin.gif
Yes - the wonderful developers and testers do not "owe" me an answer biggrin.gif
Yes - I'm asking anyway tongue.gif

I would love to help test but fear my ears will only harm the process by muddying the test results. I try and aid the HA community in other ways - like helping newbies get their questions answered. I leav ethe listening tests to the golden eared and the programming to the programmers.

Looking forward to a stable 3.97 release soon!
Go to the top of the page
+Quote Post
dev0
post Feb 1 2005, 08:06
Post #32





Group: Developer
Posts: 1679
Joined: 23-December 01
From: Germany
Member No.: 731



QUOTE (Jebus @ Jan 26 2005, 12:11 AM)
I know this is just one listening test, but would it be possible to remap --preset standard to use --vbr-new if it is determined to be of higher quality, sort of like --r3mix was changed in the last version?
*

I'd prefer making vbr-new the default VBR mode, but that would need some additional testing.


--------------------
"To understand me, you'll have to swallow a world." Or maybe your words.
Go to the top of the page
+Quote Post
[proxima]
post Feb 1 2005, 22:49
Post #33





Group: Members
Posts: 197
Joined: 12-October 02
From: Italy
Member No.: 3537



guruboolez results about the new VBR mode are really interesting and unexpected! Maybe these findings could be a starting point for a new HA recommanded version. Certainly further, intensive testing is needed.
Unfortunately i'm busy with my exams at this time sad.gif


--------------------
WavPack 4.3 -mfx5
LAME 3.97 -V5 --vbr-new --athaa-sensitivity 1
Go to the top of the page
+Quote Post
Jechdery
post Feb 2 2005, 23:27
Post #34





Group: Members
Posts: 4
Joined: 2-February 05
Member No.: 19544



I am using 3.96.1 -V 2 rather than 3.90.3 aps for encoding speed reasons.

From guruboolez tests, 3.97a6 -V 2 --vbr-new > 3.90.3 aps > 3.96.1 and 3.97a6 -V 2.

Has anybody compared 3.96.1 -V 2 --vbr-new to 3.90.3 aps ? i.e. does the new VBR algorithm appear better than the old, both for 3.96.1 and 3.97a6 ?

This would allow me to know if I keep on using 3.96.1 or regress to 3.90.3.
Go to the top of the page
+Quote Post
Wombat
post Nov 28 2005, 19:01
Post #35





Group: Members
Posts: 1036
Joined: 7-October 01
Member No.: 235



After some reading it seems i finally found the thread that has dethroned 3.90.3 smile.gif

Maybe this is the right place to add something.
There is the new recommendation using 397beta together with vbr-new.
As the listening tests show there seems nothing wrong with it. One thing that makes me wonder is the artifacts that have to be tolerated when using it.
Here is what you developers and others that still can change lame behaviour may check.
At least 2 samples that produce real ugly articacts with 397b1 -V2 --vbr-new donīt do this with 3961 -V2 --vbr-new.
I talk about Birds and s53_wind_saxophone_a from guruboolez.
Maybe comparing behavior with these easy to hear artifacts can help in some way?
Go to the top of the page
+Quote Post
Wombat
post Dec 1 2005, 00:55
Post #36





Group: Members
Posts: 1036
Joined: 7-October 01
Member No.: 235



I donīt know where to post this and i am waiting for the 3.97 sample regression thread.
So i will add it here. Even Roels number 1 sample "velvet.wav" has some funny rhytmic plops added with 3.97b -V2 -vbr--new. Mostly at second 2-3 and 9-10.
Go to the top of the page
+Quote Post
robert
post Dec 1 2005, 11:02
Post #37


LAME developer


Group: Developer
Posts: 788
Joined: 22-September 01
Member No.: 5



@wombat
could you check 3.98a2 with the velvet.wav? The "sandpaper" problem with "birds" is still there in 3.98a2.
Go to the top of the page
+Quote Post
Wombat
post Dec 1 2005, 11:17
Post #38





Group: Members
Posts: 1036
Joined: 7-October 01
Member No.: 235



Sure i will! As soon as i am home tonight.
Go to the top of the page
+Quote Post

2 Pages V  < 1 2
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 19th September 2014 - 21:21