IPB

Welcome Guest ( Log In | Register )

2 Pages V   1 2 >  
Reply to this topicStart new topic
Lame 3.97 Alpha 8 Testing Thread
Gabriel
post Mar 6 2005, 17:35
Post #1


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



Updated V3 and V2 presets (vbr-new is unchanged)

What I am interested into is mainly results of V3 and V2 (especially V2) against either 3.96.1 or 3.97a7.
You can test it against 3.90.3 if you want, but I'd like to know the evolution from previous version first.

Results from 3.97a7:
http://www.hydrogenaudio.org/forums/index....showtopic=31255
Go to the top of the page
+Quote Post
rjamorim
post Mar 6 2005, 17:45
Post #2


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



Here's a quick and dirty compile (untested) done in MSVC6, without NASM optimizations.

Please don't link directly to it from elsewhere, it'll just stay there until John XXXIII comes up with a properly done compile.

Edit: look down

This post has been edited by rjamorim: Mar 6 2005, 23:42


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
lotr
post Mar 6 2005, 20:33
Post #3





Group: Members
Posts: 14
Joined: 26-February 05
Member No.: 20171



ABX results on trumpets sample.

a8 -V3 vs a7 -V3: 10/10 (0.1%) //a8 sounded worse.

a8 -V2 vs a7 -V2: 10/10 (0.1%) //a8 sounded worse.

3.97a8 -V3 vs 3.96.1 -V3: 10/10 (0.1%) //a8 sounded worse.

3.97a8 -V2 vs 3.96.1 -V2: 10/10 (0.1%) //a8 sounded worse.

In all four tests the latest, 3.97a8 version, sounded more "hoarse" during some parts of the sample, than the version it was compared with.
Go to the top of the page
+Quote Post
katharsis
post Mar 6 2005, 20:43
Post #4





Group: Members
Posts: 6
Joined: 6-March 05
Member No.: 20421



Does anyone know why I get a 30% faster lame (with preset standard) when I compile it myself?
I've uploaded it to www.geoshock.com/lame.zip , maybe it doesn't work as it should.

bye,
Michael

This post has been edited by katharsis: Mar 6 2005, 20:48
Go to the top of the page
+Quote Post
john33
post Mar 6 2005, 21:08
Post #5


xcLame and OggDropXPd Developer


Group: Developer
Posts: 3760
Joined: 30-September 01
From: Bracknell, UK
Member No.: 111



QUOTE (katharsis @ Mar 6 2005, 07:43 PM)
Does anyone know why I get a 30% faster lame (with preset standard) when I compile it myself?
I've uploaded it to www.geoshock.com/lame.zip , maybe it doesn't work as it should.

bye,
Michael
*

If you're referring to Roberto's compile, it's because it's (a) MSVC which is slower than ICL, and (b) mostly because it doesn't include the nasm assembler routines in the compile.

Anyway, new build now at Rarewares. smile.gif


--------------------
John
----------------------------------------------------------------
My compiles and utilities are at http://www.rarewares.org/
Go to the top of the page
+Quote Post
Acid Orange Juic...
post Mar 6 2005, 21:17
Post #6





Group: Banned
Posts: 69
Joined: 16-February 05
Member No.: 19879



I still can ABX this sample very easy:

aps_Killer_sample.wav; from here

My ABX results:

Settings: Lame 3.97a8 -V 2: ( 10/10 ) Very easy to ABX.

Settings: Lame 3.97a8 -V 2 --vbr-new: ( 10/10 ) Very easy to ABX again.
Go to the top of the page
+Quote Post
Gecko
post Mar 6 2005, 23:31
Post #7





Group: Members
Posts: 943
Joined: 15-December 01
From: Germany
Member No.: 662



Angelic sample 397a7 vs 397a8.

-V 2
ABX 6/10 --> no difference to me.

-V 3
1st run: ABX 9/10 (very hard, need longer pauses between trials)
But damnit, I pressed cancel when foobar asked me to save the log because I wanted to jump back and try something, but it quit the ABX dialog and now I don't know which file was which!

2nd run to find out which is which: ABX 5/5 and to my ears 397a8 sounded slightly worse than 397a7. There was more high pitched "sparkle".

Total: 14/15
Go to the top of the page
+Quote Post
jaybeee
post Mar 8 2005, 20:15
Post #8





Group: Members
Posts: 410
Joined: 20-October 04
From: UK
Member No.: 17750



My badvilbel.flac abx results:

Settings: Lame 3.96.1 --preset standard: ( 8/8 ) Very easy to ABX.

foo_abx v1.2 report
foobar2000 v0.8.3
2005/03/08 18:33:34

File A: file://C:\My Downloads\Encoders\Problem_audio_samples\badvilbel.flac
File B: file://C:\My Downloads\Encoders\Problem_audio_samples\3.96.1--aps\badvilbel.mp3

18:33:35 : Test started.
18:35:36 : 01/01 50.0%
18:36:41 : 02/02 25.0%
18:37:54 : 03/03 12.5%
18:39:37 : 04/04 6.3%
18:40:49 : 05/05 3.1%
18:41:56 : 06/06 1.6%
18:42:30 : 07/07 0.8%
18:43:04 : 08/08 0.4%
18:43:06 : Test finished.

----------
Total: 8/8 (0.4%)

---------------------------------------

Settings: Lame 3.97a8 --preset standard: ( 8/8 ) Very easy to ABX - although seemed to be slightly better (results do not support this at all though).

foo_abx v1.2 report
foobar2000 v0.8.3
2005/03/08 18:45:59

File A: file://C:\My Downloads\Encoders\Problem_audio_samples\badvilbel.flac
File B: file://C:\My Downloads\Encoders\Problem_audio_samples\3.97a8--aps\badvilbel.mp3

18:46:00 : Test started.
18:47:09 : 01/01 50.0%
18:47:31 : 02/02 25.0%
18:47:52 : 03/03 12.5%
18:48:10 : 04/04 6.3%
18:48:27 : 05/05 3.1%
18:48:47 : 06/06 1.6%
18:49:07 : 07/07 0.8%
18:49:25 : 08/08 0.4%
18:49:27 : Test finished.

----------
Total: 8/8 (0.4%)

---------------------------------------

Settings: Lame 3.97a8 -V 2 --vbr-new: ( 8/8 ) Very easy to ABX again.

foo_abx v1.2 report
foobar2000 v0.8.3
2005/03/08 19:08:32

File A: file://C:\My Downloads\Encoders\Problem_audio_samples\badvilbel.flac
File B: file://C:\My Downloads\Encoders\Problem_audio_samples\3.97a8-V2-vbr-new\badvilbel.mp3

19:08:33 : Test started.
19:09:19 : 01/01 50.0%
19:09:43 : 02/02 25.0%
19:10:00 : 03/03 12.5%
19:10:17 : 04/04 6.3%
19:10:34 : 05/05 3.1%
19:10:53 : 06/06 1.6%
19:11:10 : 07/07 0.8%
19:11:26 : 08/08 0.4%
19:11:38 : Test finished.

----------
Total: 8/8 (0.4%)


--------------------
http://www.health4ni.com/
Go to the top of the page
+Quote Post
lotr
post Mar 8 2005, 21:41
Post #9





Group: Members
Posts: 14
Joined: 26-February 05
Member No.: 20171



ABX results on castanets2 sample:

a8 -V3 vs a7 -V3: 10/10 (0.1%) // 3.97a8 sounded better than a7. Less pre-echo.

I couldn't hear a difference on any of the following tests:

a8 -V2 vs a7 -V2 (6/10)
a8 -V2 vs 3.96.1 -V2 (6/10)
a8 -V3 vs 3.96.1 -V3 (5/10)
Go to the top of the page
+Quote Post
yong
post Mar 10 2005, 10:54
Post #10





Group: Members
Posts: 202
Joined: 9-July 04
From: Malaysia
Member No.: 15210



LAME ACM bug
This probably is an old bug:
Here's example,
when the MIN bitrate is 8,
MAX is 320,
if i adjust the "step" slider from 16 to 12,
the "LAME MP3 codec:about" window will stop responding.


--------------------
http://foobar2000.xrea.jp/
Go to the top of the page
+Quote Post
guruboolez
post Mar 12 2005, 07:35
Post #11





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



LAME 3.97 alpha 8 TEST #1: -V 3


For the following tests, I tried to obtain the most accurate results. That’s why I didn’t include extra elements (like comparison with --vbr-new, or with an ancient lame version): my full attention was focused on difference between alpha8 and the previous one. For that reason, I put all efforts on ABX direct comparison between both encodings. 12 trials for each sample, no more, no less (except on training). No ABX tests between reference and encodings: unnecessary waste of time.

I’ve used previously 54 samples to test V4 preset. It was a luxury I can’t afford anymore, because my time is limited and my motivation quite over. Therefore, I returned to the general samples’ set used for alpha 5 and alpha 6 (20 samples, melting ‘general’ music, ‘classical’ and two critical samples). I’ve just replaced the cello’s sample (BachS1007.wav, too transparent at high bitrate) by another cello sample (Kodály.wav), more detailed and consequently harder to encode.

Other change: I’m now using the new java version of abc/hr, which is remarkable. It has a precious feature: it automatically saves the playback range. Now, developers could easily link all written comments to the corresponding part of the sample. I’ll precise all tested range in my synoptic table of results.
Other good point for schnofler’s software: it systematically cleans all comments wrote by the tester after each fresh test. ff123’s abchr beta 2 is buggy here, and often keeps previous comments. It’s sometimes funny (e.g. “distorted trumpet” on a solo piano sample…), but it’s often confusing. Comments are now fully reliable. I’ll recap the audible problems in my tables.


RESULTS





ABX log files are here.


COMMENTS

Gabriel’s tuning have the same impact on -V3 than on -V4: it lowers the bitrate, and increase the quality. It’s important to note that the progress are also very similar to the progress noticed by testing -V4: there are less distortions (noticeable on cymbals mostly), no more issues with low volume parts, and less pre-echo (at least on non critical samples: castanets2.wav is still heavily smeared). It’s probably not a coincidence… I suppose that -V3 alpha 8 benefits from similar tunings than -V4 alpha7.
I’ve only noticed two serious regressions (ATrain and MidnightVoyage.wav). But on average, the new alpha 8 appeared to me clearly better than the old one.

This post has been edited by guruboolez: Dec 29 2005, 23:12
Go to the top of the page
+Quote Post
guruboolez
post Mar 12 2005, 07:36
Post #12





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



LAME 3.97 alpha 8 TEST #2: -V 2, aka --preset standard



Contrary to tunings performed by Gabriel on V4 and V3, the modified V2 preset doesn’t lower the average bitrate. It’s now slightly higher than before, and with a great disparity between samples. The most problematic case is thear1.wav: bitrate jumps from 213 (alpha 7) to 247 kbps, and the most annoying thing is the chasm between -V3 (167 kbps) and -V2 (247 kbps): + 48%, for an average difference between two presets of ~15% only. Is it exceptional? People listening to metal should take care.


RESULTS





ABX log files are here.


I was glad to discover that the modified V2 is also better than the previous one. The progresses are similar to those noticed with -V4 and -V3: less preecho, less distortion, some rare ATH (?) issues corrected (no more ringing, or instability in highest frequencies). Two regressions: MidnightVoyage.wav (again), and castanets2.wav (no additional pre-echo, but an additional colouration which isn’t pleasant to my ears).

This post has been edited by guruboolez: Dec 29 2005, 23:13
Go to the top of the page
+Quote Post
guruboolez
post Mar 12 2005, 07:38
Post #13





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



LAME 3.97 alpha 8 EXTRA: comparison between different listening tests


It might be interesting to sum up the results of recent tunings made by Gabriel on defaulted VBR mode.
1/ -V4 was significantly inferior to -V4 --vbr-new (alpha 5), but after tunings (alpha 7), both produces similar quality encodings (with maybe a slight advantage for --vbr-new).
2/ -V2 was significantly inferior to -V2 --vbr-new (alpha 6). I didn’t compared the newer -V2 to -V2 --vbr-new, but the progresses noticed for -V2 alpha 8 are very similar (same genre, same level of improvement) to those noticed between alpha5 V4 and alpha7 V4. Therefore I strongly believe that -V2 and -V2 --vbr-new are now very close, and that 3.97 alpha 8 should reaches the same level of quality than 3.90.3 (see the following table comparison).




Important note: I don’t like comparison between distant listening tests. I have created this table in order to see how much VBR presets improved on average since Gabriel modified them. Progresses are very similar… Please don’t compare single notations. And don’t be surprised by apparent incoherencies with notation: -V2 is not inferior to -V3 because average notation is slightly superior with this one. My notation is relative (to the expected quality from given bitrate), floating (no anchors, low nor high).



Now, I’ll probably make a good break with listening tests. Hope they were helpful. Anyway, a big thanks to all lame developers smile.gif

This post has been edited by guruboolez: Dec 29 2005, 23:13
Go to the top of the page
+Quote Post
pest
post Mar 12 2005, 11:41
Post #14





Group: Members
Posts: 208
Joined: 12-March 04
From: Germany
Member No.: 12686



OT:

@guruboolez

i appreciate your work for lame.
imho lame would not be the same without your listening tests!

best regards
pest
Go to the top of the page
+Quote Post
Gabriel
post Mar 12 2005, 12:42
Post #15


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



Once again, thank you very much for those precious results.
Go to the top of the page
+Quote Post
dev0
post Mar 12 2005, 12:57
Post #16





Group: Developer
Posts: 1679
Joined: 23-December 01
From: Germany
Member No.: 731



Thanks a lot Guruboolez.
If more people could provide similiar test results (you don't need to test as many samples as Guru), 3.97 could easily become the recommended version once it's released.


--------------------
"To understand me, you'll have to swallow a world." Or maybe your words.
Go to the top of the page
+Quote Post
Gabriel
post Mar 12 2005, 14:44
Post #17


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



thear1.wav bitrate at V2 is caused by extreme sfb21 bloating:
-V2 --> 247.3 kbps
-V2 -Y --> 173.1 kbps
Go to the top of the page
+Quote Post
henkersmahlzeit
post Mar 12 2005, 15:24
Post #18





Group: Members
Posts: 110
Joined: 31-December 03
Member No.: 10840



@guruboolez
Overwhelming again! You are something of a living treasure!

Maybe a bit too overwhelming for a ABX newbie ... I think, it would be nice to have a small but defined sample-set with specific instructions for starters ... maybe this could motivate some people!

You lame developers are living treasure too wink.gif
Go to the top of the page
+Quote Post
mithrandir
post Mar 12 2005, 16:39
Post #19





Group: Members
Posts: 669
Joined: 15-January 02
From: SE Pennsylvania
Member No.: 1032



Ah, now -V2 and -V3 use x mode 9 instead of 3. This allows the maskingadjust and ath floor values to be set much lower (i.e. less masking and more sensitive ATH). This is essentially a "trading of the bits" and it appears the tradeoff is much better this way if guru's results are any indication.
Go to the top of the page
+Quote Post
indybrett
post Mar 12 2005, 17:06
Post #20





Group: Members (Donating)
Posts: 1350
Joined: 4-March 02
From: Indianapolis, IN
Member No.: 1440



These ABX tests were all done with vbr-old, correct?


--------------------
flac>fb2k>kernel streaming>audiophile 2496>magni>dt990 pro
Go to the top of the page
+Quote Post
crazychimp132
post Mar 12 2005, 17:48
Post #21





Group: Members
Posts: 22
Joined: 27-July 04
Member No.: 15827



QUOTE (henkersmahlzeit @ Mar 12 2005, 10:24 AM)
@guruboolez
Overwhelming again! You are something of a living treasure!

Maybe a bit too overwhelming for a ABX newbie ... I think, it would be nice to have a small but defined sample-set with specific instructions for starters ... maybe this could motivate some people!

You lame developers are living treasure too  wink.gif
*

I agree! I wish I could help with testing but I just don't know where to begin. The problem is getting someone who knows what they are doing to actually organize some sort of standard ABX test.
Go to the top of the page
+Quote Post
esa372
post Mar 12 2005, 20:35
Post #22





Group: Members (Donating)
Posts: 429
Joined: 5-September 04
From: Los Angeles
Member No.: 16796



QUOTE (crazychimp132 @ Mar 12 2005, 08:48 AM)
QUOTE (henkersmahlzeit @ Mar 12 2005, 10:24 AM)
I think, it would be nice to have a small but defined sample-set with specific instructions for starters...
I agree! I wish I could help with testing but I just don't know where to begin.
Cosign.
smile.gif


--------------------
Clowns love haircuts; so should Lee Marvin's valet.
Go to the top of the page
+Quote Post
lotr
post Mar 13 2005, 01:36
Post #23





Group: Members
Posts: 14
Joined: 26-February 05
Member No.: 20171



ABX results on awe32_20sec

3.97a8 -V3 vs 397a7 -V3: 10/10

a8 gave some artifacts that sounded higher in pitch than the artifacts on the a7 encoded sample.


3.97a8 -V2 vs 3.97a7 -V2: 9/10

I didn't hear the high pitched artifacts with a8 -V2. Overall less artifacting than a7, so a8 was better.


3.97a8 -V2 vs 3.96.1 -V2: 10/10

3.97a8 was better.



ABX results on OrionII (2.1)

3.97a8 -V3 vs 3.97a7 -V3: 8/10

Less smearing with a8, a8 was better.


3.97a8 -V2 vs 3.97a7 -V2: 10/10

Less smearing with a8.


3.97a8 -V2 vs 3.96.1 -V2: 3/10

They sounded the same.



ABX results on hihat

wav vs LAME 3.97a8 -V3: 9/10

Each hit on the hihat produced a less precise/more spread sound with LAME.


wav vs LAME 3.97a8 -V2: 10/10

The same alteration here.


3.97a8 -V3 vs 3.97a7 -V3: 9/10

a8 sounded less precise than a7, a7 was closer to the original.


3.97a8 -V2 vs 3.97a7 -V2: 7/10

I'm not too sure here. Sometimes a7 sounded better, sometimes they sounded the same.

This post has been edited by lotr: Mar 13 2005, 01:37
Go to the top of the page
+Quote Post
guruboolez
post Mar 13 2005, 15:39
Post #24





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



QUOTE (indybrett @ Mar 12 2005, 05:06 PM)
These ABX tests were all done with vbr-old, correct?
*

I've only used -V2 & -V3 command line. The VBR mode isn't --vbr-new.
Go to the top of the page
+Quote Post
Enig123
post Mar 14 2005, 07:38
Post #25





Group: Members
Posts: 208
Joined: 11-April 02
Member No.: 1749



Gabriel,

I tried to encode a track with this alpha today in a command line, and found there's a "switch" display between "long" and "short". I've checked the outputed mp3 and couldn't find any mixed-block, so it doesn't mean mixed-block.

I am curious what does this exactly mean.
Go to the top of the page
+Quote Post

2 Pages V   1 2 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 16th September 2014 - 20:34