IPB

Welcome Guest ( Log In | Register )

6 Pages V  < 1 2 3 4 > »   
Reply to this topicStart new topic
LAME 3.96 FINAL vs. 3.90.3 Test, Test/Result Thread
Lyx
post Apr 15 2004, 16:40
Post #26





Group: Members
Posts: 3353
Joined: 6-July 03
From: Sachsen (DE)
Member No.: 7609



I didn't do the test with 3.90.3 myself yet. there are two people involved.

The following happened:
- i helped another person (zolder) how to setup EAC with LAME and preset standard.
- we did setup encoder and drive -offset calibration in EAC and used it
- then zolder ripped the CD to wav. Secure Mode, no normalizing, no change to the ripped audiodata.
- He listened to the WAVs with foobar, 16bit non-dithered or waveshaped output. No problems audible. Everything was fine.
- after that, he encoded them the LAME 3.96b1 --preset standard. And we both listened to it again on foobar (same fb2k-settings). At some trackchanges problems were audible.
- we did the same again with LAME 3.96final - same problems
- then, zolder encoded the album with LAME 3.90.3 --alt-preset standard and listened to it with foobar(same settings again), especially to the trackchanges which we learned are notorious for problems. According to him, everything sounded okay with 3.90.3. I have talked alot about gapless playback with him in the previous days, and he knows what to look for.

"How it sounds like": mostly clicks and pops.... like a sudden displacement of the waveform - as if the two waves(of the 2 tracks) aren't aligned correctly at the trackchange. I may be able to get the sample earlier, no guarantees, but i'll try my best.

- Lyx

This post has been edited by Lyx: Apr 15 2004, 16:41


--------------------
I am arrogant and I can afford it because I deliver.
Go to the top of the page
+Quote Post
Gecko
post Apr 15 2004, 19:03
Post #27





Group: Members
Posts: 943
Joined: 15-December 01
From: Germany
Member No.: 662



QUOTE (ff123 @ Apr 14 2004, 10:56 PM)
None of this sounds like a bug, which is what the talk about dropouts made it sound like.  From the initial description, I was expecting something bad like the old Fraunhofer mp3enc 3.1 dropouts.

At this point, I think it's just normal artifacting which would naturally tend to sound worse in cbr than it would in abr or vbr.

Well, I didn't test the range in which the dropout problem occurs and I only tested (alt) preset standard. Iirc, the problem has only been reported for one of the 128k-ish settings.

I don't think you can draw any conclusions about the dropout problem from my test.
Go to the top of the page
+Quote Post
ssamadhi97
post Apr 15 2004, 19:10
Post #28





Group: Developer (Donating)
Posts: 1203
Joined: 10-February 02
From: Endless Water
Member No.: 1305



QUOTE (2Bdecided @ Apr 15 2004, 09:33 AM)
That's TV line whistle - you find that on lots of recordings.

How does this kind of noise make its way into studio tracks anyway? Just wondering..

(sorry 'bout the OT)


--------------------
A riddle is a short sword attached to the next 2000 years.
Go to the top of the page
+Quote Post
tigre
post Apr 15 2004, 23:22
Post #29


Moderator


Group: Members
Posts: 1434
Joined: 26-November 02
Member No.: 3890



RESULTS

Tested settings:
Lame 3.90.3 --alt-preset 128
Lame 3.96 --preset 128
Lame 3.96 -V 5

Tested samples from ff123's 64kbps test:

Blackwater
FloorEssence
Layla

All encoded files were ABXed successuflly (p < 0.01) against the original and against each other (or got equal ratings). For details (abx logs, descriptions of the audible differeces) see attatched .zip file.

Results (ABC/HR ratings):

Blackwater
3.7 Lame 3.90.3 --alt-preset 128
3.7 Lame 3.96 --preset 128
3.9 Lame 3.96 -V 5

FloorEssence
2.5 Lame 3.90.3 --alt-preset 128
2.2 Lame 3.96 --preset 128
3.8 Lame 3.96 -V 5

Layla
3.4 Lame 3.90.3 --alt-preset 128
3.0 Lame 3.96 --preset 128
4.0 Lame 3.96 -V 5
Attached File(s)
Attached File  ff123s_samples_part2.zip ( 1.95K ) Number of downloads: 396
 


--------------------
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello
Go to the top of the page
+Quote Post
ff123
post Apr 16 2004, 01:03
Post #30


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



QUOTE (tigre @ Apr 15 2004, 02:22 PM)
RESULTS

Tested settings:
Lame 3.90.3 --alt-preset 128
Lame 3.96 --preset 128
Lame 3.96 -V 5

Tested samples from ff123's 64kbps test:

Blackwater
FloorEssence
Layla

It's looking like tigre's results may conflict with mine regarding the comparison between the 3.90.3 and 3.96 128 settings (although that could change with the rest of the samples).

ff123
Go to the top of the page
+Quote Post
FatBoyFin
post Apr 16 2004, 12:29
Post #31





Group: Members
Posts: 22
Joined: 29-March 04
Member No.: 13109



RESULT

QUOTE
3.96b2 -V 5 > 3.90.3 --ap 128 :: fatboy :: FatBoyFin :: 1x verified by High Fidelity using 3.96 final


Ive also comfirmed this for 3.96 now with ABX results atleast 7/7.
Go to the top of the page
+Quote Post
tigre
post Apr 16 2004, 13:22
Post #32


Moderator


Group: Members
Posts: 1434
Joined: 26-November 02
Member No.: 3890



RESULTS

Tested settings:
Lame 3.90.3 --alt-preset 128
Lame 3.96 --preset 128
Lame 3.96 -V 5

Tested samples from ff123's 64kbps test:

LifeShatters
LisztBMinor
MidnightVoyage

All encoded files were ABXed successuflly (p < 0.01) against the original and against each other. For details (abx logs, descriptions of the audible differeces) see attatched .zip file.

Results (ABC/HR ratings):

LifeShatters
4.2 Lame 3.90.3 --alt-preset 128
4.0 Lame 3.96 --preset 128
4.5 Lame 3.96 -V 5

LisztBMinor
2.5 Lame 3.90.3 --alt-preset 128
1.5 Lame 3.96 --preset 128
1.0 Lame 3.96 -V 5

MidnightVoyage
3.0 Lame 3.90.3 --alt-preset 128
3.6 Lame 3.96 --preset 128
4.0 Lame 3.96 -V 5
Attached File(s)
Attached File  ff123s_samples_part3.zip ( 2.29K ) Number of downloads: 318
 


--------------------
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello
Go to the top of the page
+Quote Post
[proxima]
post Apr 16 2004, 15:48
Post #33





Group: Members
Posts: 197
Joined: 12-October 02
From: Italy
Member No.: 3537



RESULTS

Tested settings:
Lame 3.90.2 --alt-preset 128 --scale 1
Lame 3.96 --preset 128 --scale 1
Lame 3.96 -V 5

Layla is a strange case, 3.96 abr has a metallic artifact in the first two seconds. Preecho in all the three but (of course) well handled with VBR encoded sample.
3.90.2 : 3.0
3.96abr: 2.5
3.96_V5: 3.8

LisztBMinor is a low volume sample and VBR is not the best anymore. With all the three samples there are problems with background noise and ringing/chirping. VBR can offer better preecho handling but maybe with such samples is nothing special.
3.90.2 : 3.0
3.96abr: 2.1
3.96_V5: 2.0

Blackwater is very indicative for me. The well known chirping/ringing artifact, even if reduced with the new 3.96 are still present in the two 3.96 encodings, 3.90.2 doesn't ring at all. Again, preecho is better with -V 5.
3.90.2 : 3.2
3.96abr: 2.8
3.96_V5: 2.9

With Waiting critical sample VBR is advantaged. 3.96 --ap 128 manifested a bad ringing in some parts.
3.90.2 : 2.2
3.96abr: 1.5
3.96_V5: 3.3

ABC/HR and ABX results here: http://xoomer.virgilio.it/fofobella/396_1.zip

However 3.96 is not always worse than 3.90.2: with some samples the progress is real. It's a pity for this slight ringing artifact that penalize 3.96 too much for me.
I plan to complete all the 12 samples during this weekend wink.gif

@tigre: In my previous post i linked a test done with lamex9 slightly in favour of 3.96. I cheched 3.96 vs lamex9 --p 128 --sfscale and they produce the same file, i think that this test could be added to the results even if greyed out.

This post has been edited by [proxima]: Apr 16 2004, 15:57


--------------------
WavPack 4.3 -mfx5
LAME 3.97 -V5 --vbr-new --athaa-sensitivity 1
Go to the top of the page
+Quote Post
ff123
post Apr 16 2004, 16:08
Post #34


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



QUOTE ([proxima)
,Apr 16 2004, 06:48 AM] Blackwater is very indicative for me. The well known chirping/ringing artifact, even if reduced with the new 3.96 are still present in the two 3.96 encodings, 3.90.2 doesn't ring at all. Again, preecho is better with -V 5.
3.90.2 : 3.2
3.96abr: 2.8
3.96_V5: 2.9

Did lame switch back to gpsycho or something?

The ringing artifacting that people complained about in gpsycho was always something that I couldn't hear, because it occurs out of my high frequency range. This was also something that nspsytune seemed to handle better.

I will have a tendency to hear lower frequency artifacting, such as preecho, warbling or fluttering. So this might be a case where there could be an interesting split of opinions.

ff123
Go to the top of the page
+Quote Post
tigre
post Apr 16 2004, 16:30
Post #35


Moderator


Group: Members
Posts: 1434
Joined: 26-November 02
Member No.: 3890



RESULTS

Tested settings:
Lame 3.90.3 --alt-preset 128
Lame 3.96 --preset 128
Lame 3.96 -V 5

Tested samples from ff123's 64kbps test:

Thear1
TheSource
Waiting

All encoded files were ABXed successuflly (p < 0.01) against the original and against each other (or got equal ratings). For details (abx logs, descriptions of the audible differeces) see attatched .zip file.

Results (ABC/HR ratings):

Thear1
4.5 Lame 3.90.3 --alt-preset 128
4.5 Lame 3.96 --preset 128
4.5 Lame 3.96 -V 5

TheSource
4.0 Lame 3.90.3 --alt-preset 128
4.0 Lame 3.96 --preset 128
3.8 Lame 3.96 -V 5

Waiting
2.0 Lame 3.90.3 --alt-preset 128
2.2 Lame 3.96 --preset 128
3.0 Lame 3.96 -V 5
Attached File(s)
Attached File  ff123s_samples_part4.zip ( 1.63K ) Number of downloads: 328
 


--------------------
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello
Go to the top of the page
+Quote Post
[proxima]
post Apr 16 2004, 16:37
Post #36





Group: Members
Posts: 197
Joined: 12-October 02
From: Italy
Member No.: 3537



QUOTE (ff123 @ Apr 16 2004, 04:08 PM)
QUOTE ([proxima)
,Apr 16 2004, 06:48 AM] Blackwater is very indicative for me. The well known chirping/ringing artifact, even if reduced with the new 3.96 are still present in the two 3.96 encodings, 3.90.2 doesn't ring at all. Again, preecho is better with -V 5.
3.90.2 : 3.2
3.96abr: 2.8
3.96_V5: 2.9

Did lame switch back to gpsycho or something?

I will have a tendency to hear lower frequency artifacting, such as preecho, warbling or fluttering. So this might be a case where there could be an interesting split of opinions.

ff123

Ringing problems, at least for some samples are partially resolved with the changed noise shaping type for --preset 128. 3.96 that defaults to ns 2 is much more better with ringing problems (rebel sample is a very strong proof, even with spectral analisys there are less droputs above 10 kHz). Small problems still remain and i prefer 3.90.2 for some cases.
3.96 seems to be much more "aggresive" with background noise (see blackwater, LisztBMinor) and this could "unmask" artifacts with mid-high frequencies.
Ringing artifact noticeable with blackwater is not audible with 3.90.2 maybe because there is the original background noise.


--------------------
WavPack 4.3 -mfx5
LAME 3.97 -V5 --vbr-new --athaa-sensitivity 1
Go to the top of the page
+Quote Post
tigre
post Apr 16 2004, 16:57
Post #37


Moderator


Group: Members
Posts: 1434
Joined: 26-November 02
Member No.: 3890



I haven't compared directly 3.96 beta vs. 3.96 final yet, but I share your observation that there are less ringing problems.

LisztBMinor is a very interesting sample indeed. I just did a quick test with 3.96 --preset standard. There ringing/warbeling added to background noise when the piano starts playing is quite obvious (of course much better then with --preset 128 or -V 5). This seems to be the next case where 3.90.3 outperforms 3.96 with --(alt-)preset standard. (and this is one of the few samples where the bitrate of 3.96 is higher).

I think this sample could be very useful to further improve 3.96 (mainly VBR modes).


--------------------
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello
Go to the top of the page
+Quote Post
ff123
post Apr 16 2004, 17:00
Post #38


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



For tigre, the results of the 12 samples do not indicate any preference:

CODE
FRIEDMAN version 1.24 (Jan 17, 2002) http://ff123.net/
Blocked ANOVA analysis

Number of listeners: 12
Critical significance:  0.05
Significance of data: 1.69E-01 (not significant)
---------------------------------------------------------------
ANOVA Table for Randomized Block Designs Using Ratings

Source of         Degrees     Sum of    Mean
variation         of Freedom  squares   Square    F      p

Total               35          28.68
Testers (blocks)    11          21.23
Codecs eval'd        2           1.11    0.56    1.93  1.69E-01
Error               22           6.34    0.29
---------------------------------------------------------------


Also, when averaged with my results, there is no significant preference (although it is just on the cusp of doing so):

CODE
FRIEDMAN version 1.24 (Jan 17, 2002) http://ff123.net/
Blocked ANOVA analysis

Number of listeners: 12
Critical significance:  0.05
Significance of data: 5.45E-02 (not significant)
---------------------------------------------------------------
ANOVA Table for Randomized Block Designs Using Ratings

Source of         Degrees     Sum of    Mean
variation         of Freedom  squares   Square    F      p

Total               35          10.92
Testers (blocks)    11           7.08
Codecs eval'd        2           0.89    0.45    3.33  5.45E-02
Error               22           2.95    0.13
---------------------------------------------------------------


ff123
Go to the top of the page
+Quote Post
tigre
post Apr 16 2004, 17:01
Post #39


Moderator


Group: Members
Posts: 1434
Joined: 26-November 02
Member No.: 3890



QUOTE ([proxima)
,Apr 16 2004, 04:48 PM] @tigre: In my previous post i linked a test done with lamex9 slightly in favour of 3.96. I cheched 3.96 vs lamex9 --p 128 --sfscale and they produce the same file, i think that this test could be added to the results even if greyed out.

Edit: I found the post you're talking about here

Could you please provide links to the samples used (if I haven't found them already)? I'll add the results to the upload thread once I have links to all samples (editing the post is not very amusing so I'd like to do it in an efficient way). The samples are:

applaud :: from ff123's sample page? If yes, which one?
campestre :: Here?
ct_reference :: I have no idea where to find that one
fall :: Here?
fatboy :: ff123's version or another (longer) one?
preecho1 :: Tell me, please
rebel :: Here?
wating :: Probably the well know one from ff123's page...

This post has been edited by tigre: Apr 16 2004, 22:53


--------------------
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello
Go to the top of the page
+Quote Post
tigre
post Apr 16 2004, 17:10
Post #40


Moderator


Group: Members
Posts: 1434
Joined: 26-November 02
Member No.: 3890



QUOTE (ff123 @ Apr 16 2004, 06:00 PM)
For tigre, the results of the 12 samples do not indicate any preference:
<snip>
Also, when averaged with my results, there is no significant preference (although it is just on the cusp of doing so):
<snip>

Hehe - I've just tried the same. The only thing that gives a significant result is calculating a p-value from our total results for 3.96 -V 5 > 3.90.3 --alt-preset 128

(11+8)/(12+12) = 19/24 -> p = 0.0033

but I doubt that this calculation is correct since every sample is used twice. wink.gif


--------------------
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello
Go to the top of the page
+Quote Post
Hoosierdaddy
post Apr 16 2004, 19:19
Post #41





Group: Members
Posts: 49
Joined: 29-January 04
Member No.: 11592



"How it sounds like: mostly clicks and pops...like a sudden displacement of the waveform - as if the two waves(of the 2 tracks) aren't aligned correctly at the trackchange."

So this is a 'glitch' which the encoder encounters when trying to encode the interstill BETWEEN tracks on a CD? Hmmm. I thought the 'gap' on a cd was actually 'digital silence'. As I understand it, there isn't any actual audio data there to encode. Just two embedded 'flags' which the signal end of one track and the beginning of another to a CD player. I wonder if the blooper occurs because there really isn't any audio data to encode?
Go to the top of the page
+Quote Post
Shingetsu
post Apr 16 2004, 20:08
Post #42





Group: Members
Posts: 22
Joined: 15-April 04
Member No.: 13527



and is there some quality loss betwen 3.90.3 and 3.96 which could be audibly in normal music ? this sounds all so complicated.
Go to the top of the page
+Quote Post
ff123
post Apr 16 2004, 20:13
Post #43


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



CODE
Means:

3.96V5   3.96p128 3.90.3  
 4.03     3.74     3.65  

---------------------------- p-value Matrix ---------------------------

        3.96p128 3.90.3  
3.96V5   0.061    0.017*  
3.96p128          0.547    
-----------------------------------------------------------------------

3.96V5 is better than 3.90.3


Averaging in proxima's current results, it looks like 3.96V5 will be the winner out of these 3 competitors.

QUOTE
and is there some quality loss betwen 3.90.3 and 3.96 which could be audibly in normal music ? this sounds all so complicated.


For the abr mode, it looks like you win some and lose some, and it depends on your ears. If you've got older ears like mine, you might prefer 3.96 abr at 128 kbs.
Go to the top of the page
+Quote Post
ff123
post Apr 16 2004, 20:18
Post #44


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



BTW, have there been bitrate tests of 3.96 -V5?

ff123
Go to the top of the page
+Quote Post
tigre
post Apr 16 2004, 20:54
Post #45


Moderator


Group: Members
Posts: 1434
Joined: 26-November 02
Member No.: 3890



QUOTE (ff123 @ Apr 16 2004, 09:18 PM)
BTW, have there been bitrate tests of 3.96 -V5?

ff123

Only with beta 1 AFAIK.

Edit: for -V 5 3.96 beta 1 and final give bit-identical outputs. (Forget 3.90, was a typo.) So no further testing is necessary. (Well, doing some more bitrate tests can't do any harm of course wink.gif ).

This post has been edited by tigre: Apr 16 2004, 22:20


--------------------
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello
Go to the top of the page
+Quote Post
Jack Comics
post Apr 16 2004, 21:24
Post #46





Group: Members (Donating)
Posts: 76
Joined: 16-December 03
From: Pennsylvania, U.S.A.
Member No.: 10488



QUOTE (tigre @ Apr 16 2004, 02:54 PM)
Only with beta 1 AFAIK.

So @ anyone who wants to help: Could you do some encodingings for 3.90 beta1 AND final with -V 5 and post the results here, please? Thanks in advance.

I'd imagine finding 3.90 beta 1 would be pretty difficult. Why would you want to test it to begin with though?
Go to the top of the page
+Quote Post
tigre
post Apr 16 2004, 22:17
Post #47


Moderator


Group: Members
Posts: 1434
Joined: 26-November 02
Member No.: 3890



Jack Comics - you're right. I've already found out and reported earlier in this thread:
QUOTE
VBR (-V 5)
final == beta2 == beta1 (bit-identical)

Hehe - thanks. I'll edit my previous post.

So ff123, -V 5 was tested and the results are "close enough" to 128kbps.


--------------------
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello
Go to the top of the page
+Quote Post
detokaal
post Apr 16 2004, 22:19
Post #48





Group: Members
Posts: 191
Joined: 9-November 03
Member No.: 9748



16 files 779,114 kb wav

Total Time is 75:22

Lame 3.96 -V 5 compresses to total 72,983 kb

Average bitrate from viewing the kbs for these 16 tracks in Winamp is 131.25

Lowest was 117 and highest was 144.
Go to the top of the page
+Quote Post
[proxima]
post Apr 17 2004, 00:51
Post #49





Group: Members
Posts: 197
Joined: 12-October 02
From: Italy
Member No.: 3537



QUOTE
Could you please provide links to the samples used (if I haven't found them already)?

Sure. Applaud and fatboy samples from here:
http://lame.sourceforge.net/download/samples
ct_reference and preecho1 uploaded in the apposite thread. You've already found the others wink.gif


--------------------
WavPack 4.3 -mfx5
LAME 3.97 -V5 --vbr-new --athaa-sensitivity 1
Go to the top of the page
+Quote Post
freakngoat
post Apr 17 2004, 01:18
Post #50





Group: Members
Posts: 170
Joined: 10-January 02
From: Manhattan Beach
Member No.: 979



RESULTS

Drone_short, aps, 0-1.2 sec

3.90.3 vs. original - 8/8
Easy to ABX.

3.96 vs. original - 8/8
Equally easy.

3.96 vs 3.90.3 - 8/8
3.96 sounds better than 3.90.3 now!

3.96 vs. 3.96b1 - 8/8
Air sounds are quieter in 3.96

Edit: added link to sample and encoding command line used

This post has been edited by freakngoat: Apr 17 2004, 01:59
Go to the top of the page
+Quote Post

6 Pages V  < 1 2 3 4 > » 
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 19th September 2014 - 15:05