IPB

Welcome Guest ( Log In | Register )

3 Pages V  < 1 2 3 >  
Reply to this topicStart new topic
Lame 3.95 vs 3.90.3, ABC/HR test results here only
indybrett
post Jan 15 2004, 22:11
Post #26





Group: Members (Donating)
Posts: 1350
Joined: 4-March 02
From: Indianapolis, IN
Member No.: 1440



QUOTE (adlai @ Jan 15 2004, 03:15 PM)
so, from skimming over this thread, I'm guessing that 3.95 is a "meh" release since quality is more or less the same while file sizes have increased

On my music, filesizes decreased when using APS.


--------------------
flac>fb2k>kernel streaming>audiophile 2496>magni>dt990 pro
Go to the top of the page
+Quote Post
Digga
post Jan 15 2004, 22:28
Post #27





Group: Members
Posts: 1047
Joined: 28-June 03
From: on the dock of the bay
Member No.: 7423



QUOTE (indybrett @ Jan 15 2004, 10:11 PM)
QUOTE (adlai @ Jan 15 2004, 03:15 PM)
so, from skimming over this thread, I'm guessing that 3.95 is a "meh" release since quality is more or less the same while file sizes have increased

On my music, filesizes decreased when using APS.

on the samples I tried (mostly hip-hop), the size gets smaller too. no wonder though I guess, with bitrates on aps going now down to 96 kbps.
but I think we should respect the title of this trhead, shouldn't we?!
"ABC/HR test results here only", and no meh-stuff.


--------------------
Nothing but a Heartache - Since I found my Baby ;)
Go to the top of the page
+Quote Post
fairyliquidizer
post Jan 15 2004, 22:43
Post #28





Group: Members
Posts: 318
Joined: 11-November 03
Member No.: 9786



QUOTE (PlazzTT @ Jan 12 2004, 04:15 PM)
QUOTE (AstralStorm @ Jan 12 2004, 11:10 PM)
Bug report - Lame 3.95.x can't decode at all. (jon33 compile)
Even can't decode its own files!

Verified.

The decoder is only there for the development frontend to allow analysis during testing. It's not intended for wider use.

Cheers,
Fairy


--------------------
http://www.glop.org/starforce/
Go to the top of the page
+Quote Post
phwip
post Jan 15 2004, 23:58
Post #29





Group: Members (Donating)
Posts: 345
Joined: 26-February 03
From: Norwich, UK
Member No.: 5189



QUOTE (Digga @ Jan 15 2004, 09:28 PM)
QUOTE (indybrett @ Jan 15 2004, 10:11 PM)
QUOTE (adlai @ Jan 15 2004, 03:15 PM)
so, from skimming over this thread, I'm guessing that 3.95 is a "meh" release since quality is more or less the same while file sizes have increased

On my music, filesizes decreased when using APS.

on the samples I tried (mostly hip-hop), the size gets smaller too. no wonder though I guess, with bitrates on aps going now down to 96 kbps.

I have just encoded a random selection of 30 tracks from my hard drive all from different albums with both 3.90.3 and 3.95.1, both --preset standard. The songs are in the main rock and pop but there are a couple of classical and a couple of jazz tracks in there and the rest is a whole mixture of styles.

The total size of the wav files is 1.17 GB. With 3.90.3 this compresses to 171 MB and with 3.95.1 to 164 MB. 21 of the 30 tracks were smaller when using 3.95.1.

QUOTE
but I think we should respect the title of this trhead, shouldn't we?!
"ABC/HR test results here only", and no meh-stuff.

You are right. Having done the test I thought I might as well post the result. But perhaps a moderator could move this whole offtopic section to a different thread?
Go to the top of the page
+Quote Post
indybrett
post Jan 16 2004, 02:44
Post #30





Group: Members (Donating)
Posts: 1350
Joined: 4-March 02
From: Indianapolis, IN
Member No.: 1440



It seems that in the lower bitrates that 3.90.3 is winning most of the ABX tests. If the two are found to be of equal sound quality with APS, then the resulting filesize would probably be the determining factor.

There is probably no point in discussing filesize any further until there is a clear winner (or tie) with APS.


--------------------
flac>fb2k>kernel streaming>audiophile 2496>magni>dt990 pro
Go to the top of the page
+Quote Post
Vietwoojagig
post Jan 16 2004, 10:58
Post #31





Group: Members
Posts: 247
Joined: 28-November 02
From: Germany, Trier
Member No.: 3916



Hi,

some people mentioned this before, but:

Don't you think that comparing 3.90.3 and 3.95.1 needs a developer-moderated and coordinated testing? I don't know how many tests they have made on which samples, but a little hint, which of the test-cases in the test-case-archive have been checked and which not, would make it more easy for the people around not to waste their time on already proved samples.

And: Only with a moderated and coordinated tesiting, everybody in this forum will accespt the results of the tests and (in the end) will accept, that 3.95.1 is better or worser than 3.90.3.

Thanks for your attention.
Go to the top of the page
+Quote Post
sld
post Jan 16 2004, 11:38
Post #32





Group: Members
Posts: 1016
Joined: 4-March 03
From: Singapore
Member No.: 5312



QUOTE (AtaqueEG @ Jan 16 2004, 04:28 AM)
OT:
That Harpsichord is fake! Is made of LEGO pieces!

It is real, and playable.
OT, but had to clarify to all.
Go to the top of the page
+Quote Post
AstralStorm
post Jan 16 2004, 17:10
Post #33





Group: Members
Posts: 745
Joined: 22-April 03
From: /dev/null
Member No.: 6130



QUOTE (adlai @ Jan 15 2004, 10:15 PM)
so, from skimming over this thread, I'm guessing that 3.95 is a "meh" release since quality is more or less the same while file sizes have increased

Not necessarily true, the filesizes of the samples I've tested (bytes, preset standard):

275 565 main_theme.3-90-3.mp3
286 648 main_theme.3-95-1.mp3

345 709 velvet.3-90-3.mp3
281 130 velvet.3-95-1.mp3

And the quality isn't same.

QUOTE (Vietwoojagig)
Don't you think that comparing 3.90.3 and 3.95.1 needs a developer-moderated and coordinated testing? I don't know how many tests they have made on which samples, but a little hint, which of the test-cases in the test-case-archive have been checked and which not, would make it more easy for the people around not to waste their time on already proved samples.


It isn't a waste of time, different people have different hearing...

EDIT: I'll edit the subject to say "(and filesizes too)"
EDIT2: Blah, it isn't possible, now you can flame Invision Board devs.

This post has been edited by AstralStorm: Jan 16 2004, 17:17


--------------------
ruxvilti'a
Go to the top of the page
+Quote Post
dev0
post Jan 16 2004, 17:23
Post #34





Group: Developer
Posts: 1679
Joined: 23-December 01
From: Germany
Member No.: 731



QUOTE (Vietwoojagig @ Jan 16 2004, 10:58 AM)
Don't you think that comparing 3.90.3 and 3.95.1 needs a developer-moderated and coordinated testing? I don't know how many tests they have made on which samples, but a little hint, which of the test-cases in the test-case-archive have been checked and which not, would make it more easy for the people around not to waste their time on already proved samples.

I asked about this on lame-dev and tried to contact Gabriel, but have always been given some kind of "Well... whatever... test it." answer.
Of course a developer-moderated and coordinated testing would be more effective, but it seems like the lame-dev team in not really interested in quality and obviously nobody can force them to be.
Go to the top of the page
+Quote Post
Gabriel
post Jan 16 2004, 18:56
Post #35


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



QUOTE
but it seems like the lame-dev team in not really interested in quality

On this board you have to back up your claims with facts. Can you provide facts?

QUOTE
It isn't a waste of time, different people have different hearing...

This is very true. If I say "this and this samples are better when using those settings", will you believe me? I am not sure about it.
Go to the top of the page
+Quote Post
kennedyb4
post Jan 16 2004, 19:07
Post #36





Group: Members
Posts: 772
Joined: 3-October 01
Member No.: 180



QUOTE (dev0 @ Jan 16 2004, 11:23 AM)
I asked about this on lame-dev and tried to contact Gabriel, but have always been given some kind of "Well... whatever... test it." answer.
Of course a developer-moderated and coordinated testing would be more effective, but it seems like the lame-dev team in not really interested in quality and obviously nobody can force them to be.

As an avid observer over the last several years, I have to say that the opposite appears to be true, at least to me.
Go to the top of the page
+Quote Post
indybrett
post Jan 16 2004, 19:08
Post #37





Group: Members (Donating)
Posts: 1350
Joined: 4-March 02
From: Indianapolis, IN
Member No.: 1440



Time to split the thread off...maybe?


--------------------
flac>fb2k>kernel streaming>audiophile 2496>magni>dt990 pro
Go to the top of the page
+Quote Post
Atlantis
post Jan 16 2004, 20:07
Post #38





Group: Members
Posts: 250
Joined: 27-December 02
From: ROMA, Italy
Member No.: 4269



QUOTE (Gabriel @ Jan 16 2004, 07:56 PM)
If I say "this and this samples are better when using those settings", will you believe me? I am not sure about it.

Hi Gabriel,
I agree with dev0 that "a developer-moderated and coordinated testing would be more effective".

I know that your time (and other developer's one) dedicated to developing LAME can be limited (everybody needs a life smile.gif ) but you (LAME developers) should really consider to coordinate the testing process.
I think that many people (me included) will gladly help you if asked for (let's say) a test about a sample using two different lame version or something like that.
I follow the lame-dev mailing list, but still I don't exactly know what is the impact of the changes added by 3.95x encoder: so maybe it could be good to have more "inverse feedback" (please smile at this definition) by you on test done by HA users.



Last but not least, thank-you for working on LAME: it is of great help for me.
Regards


--------------------
Vital papers will demonstrate their vitality by spontaneously moving from where you left them to where you can't find them.
Go to the top of the page
+Quote Post
Gabriel
post Jan 16 2004, 20:16
Post #39


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



QUOTE
I think that many people (me included) will gladly help you if asked for (let's say) a test about a sample using two different lame version or something like that.

When there is a specific change that needs to be tested, I agree. But for 3.95 against 3.90, there are many changes, and so nearly everything could be impacted.
Go to the top of the page
+Quote Post
[proxima]
post Jan 16 2004, 20:55
Post #40





Group: Members
Posts: 197
Joined: 12-October 02
From: Italy
Member No.: 3537



QUOTE (dev0 @ Jan 16 2004, 05:23 PM)
Of course a developer-moderated and coordinated testing would be more effective

I perfectly agree with you and i've pointed to this point many times in my posts. I like doing listening tests but sometimes, i feel something like i'm wasting my time because lack of organization.
In this community there are for sure a lot of people interested in the future of LAME but what miss is a good organization: listening tests should be "guided" by someone with artifacts trained ears and with a little coding skills so that he can directly tweak the code. This role was covered very well by Dibrom in the past but now he lost interest and has very little time. Dibrom himself stated many times that the alt-presets are a sort of "hack" of LAME 3.90 and there is always the danger of modified behaviour with newer versions, even a different compilation switches seem to have altered slightly the behaviour of alt-presets !! So we have "our stable version" that is 3.90.2/3.90.3 but (even if very good in quality) now is quite old and we need to unblock this situation if we want to get advantages from future LAME progresses.
Alt-presets as they are now seem to have a too high sensitivity to slight encoder changes, IMHO what we need is to "stabilize" the alt-presets in the sense that they must be considered no more as "hacks" but as an integral part of the LAME code.
Of course, at this time, i don't see any people that could do this and i don't want to accuse developers, i perfectly understand the priorities and the limited time.
QUOTE
but it seems like the lame-dev team in not really interested in quality

Sure, there was some hurried releases marked as "stable" (3.93 for example). Maybe the alt-presets behavior of some "post 3.90" stable releases is not perfect but i think that what you stated is not literally true.


--------------------
WavPack 4.3 -mfx5
LAME 3.97 -V5 --vbr-new --athaa-sensitivity 1
Go to the top of the page
+Quote Post
Atlantis
post Jan 16 2004, 21:03
Post #41





Group: Members
Posts: 250
Joined: 27-December 02
From: ROMA, Italy
Member No.: 4269



QUOTE (Gabriel @ Jan 16 2004, 09:16 PM)
But for 3.95 against 3.90, there are many changes, and so nearly everything could be impacted.

Agreed.

In my first post I was just talking about (like you correctly understood) the "usual" development process (alpha => beta => release).

Regards,
Alberto

This post has been edited by Atlantis: Jan 16 2004, 21:09


--------------------
Vital papers will demonstrate their vitality by spontaneously moving from where you left them to where you can't find them.
Go to the top of the page
+Quote Post
dev0
post Jan 16 2004, 21:47
Post #42





Group: Developer
Posts: 1679
Joined: 23-December 01
From: Germany
Member No.: 731



QUOTE ([proxima)
,Jan 16 2004, 08:55 PM] Alt-presets as they are now seem to have a too high sensitivity to slight encoder changes, IMHO what we need is to "stabilize" the alt-presets in the sense that they must be considered no more as "hacks" but as an integral part of the LAME code.
Of course, at this time, i don't see any people that could do this and i don't want to accuse developers, i perfectly understand the priorities and the limited time.

From what I've gathered from the changelogs that's exactly what Gabriel is trying to do, there's only some more testing needed to verify that his changes and the new version provide an overall higher quality than the currently recommended version with Dibrom's original tunings.

A lot of fine tuning will have to be redone when LAME4 is approaching completition and will hopefully introduce us to a whole new level of MP3 quality.
Go to the top of the page
+Quote Post
Wombat
post Jan 16 2004, 22:25
Post #43





Group: Members
Posts: 1011
Joined: 7-October 01
Member No.: 235



At first, i donīt abx much samples at this time.

One of the old samples i always hear a problem with aps is sophia2.
As i already mentioned in another thread the sophia2 sample has an artifact plop with 3.95.1 and has a sandpaper noise with 3.90.3 but no plop.
I prefer the sandpaper over the artifact but the best out of the mix would even be perfect smile.gif
So someone beeing able to change the preset directly to test some circumstances will be the key like proxima already wrote.
The potential of lame curing this sample is there obviously.

Atm Gabriel seems to be the only one being able to this.
So i have to thank Gabriel, the developer, for his great work till here and wish him good luck finding the final tweak!


Wombat
Go to the top of the page
+Quote Post
Wombat
post Jan 17 2004, 01:56
Post #44





Group: Members
Posts: 1011
Joined: 7-October 01
Member No.: 235



One thing to think about:
Even if lame 3.95.1 is behaving equal in the end with the testsamples it wouldnīt tell much!
It took about 2 years now to find all these problem music. It would take another 2 years to
find problems with 3.95 introduced with the modifications. There is no proove at all to set it
equal to 3.90.3.

There is no sense for using aps with any newer build than 3.90.3 until many testsamples "are really cured"

Isnīt it?

Wombat
Go to the top of the page
+Quote Post
AstralStorm
post Jan 17 2004, 03:14
Post #45





Group: Members
Posts: 745
Joined: 22-April 03
From: /dev/null
Member No.: 6130



Actually, there is a reason: it is about 2x faster with similar quality as 3.90.3 most of the time.
It also doesn't produce larger files on (my) average, but it really depends on the type of music.
It's only up to the admins to determine if it's good enough to replace 3.90.3 as recommended version,
but to do this they need test results, hence this thread.
It seems like the recomended version needs a refresh - but when is it going to happen, we shall see.
Maybe just not now.

Anyway, there's no perfect psychoacoustic compression yet and there won't be any method soon.
MusePack is much closer to this than LAME. For True Quality ™, use the lossless compression.
We're trying here to just save some space with acceptable (possibly artifact-free) quality.
It's just the difference in tradeoff (perceived quality vs space vs speed).
One may even find 128kbps MP3s produced by l3enc good enough.
(shame on him, but we don't need to listen to that)

Enough discussion, please test further.

This post has been edited by AstralStorm: Jan 17 2004, 03:30


--------------------
ruxvilti'a
Go to the top of the page
+Quote Post
LoFiYo
post Jan 17 2004, 04:52
Post #46





Group: Members
Posts: 133
Joined: 2-January 04
Member No.: 10896



Here is my test result using my plastic (as opposed to golden) ears wink.gif .

CODE
ABC/HR Version 0.9b, 30 August 2002
Testname: Polonaise 3.90.3 v 3.95.1 - ABR128

1L = C:\My Music\lab\Polonaise\Polonaise-ABR128-3.95.1.wav
2L = C:\My Music\lab\Polonaise\Polonaise-ABR128-3.90.3.wav

---------------------------------------
General Comments:
3.90.3 vs 3.95.1 (ABR 128kbps mode = alt-preset 128)
---------------------------------------
1L File: C:\My Music\lab\Polonaise\Polonaise-ABR128-3.95.1.wav
1L Rating: 3.7
1L Comment: After around 15.5 sec, noticeable distortion is detected.
---------------------------------------
2L File: C:\My Music\lab\Polonaise\Polonaise-ABR128-3.90.3.wav
2L Rating: 4.0
2L Comment: Distortion is still noticeable, but clearly better than the other encode.
---------------------------------------
ABX Results:
Original vs C:\My Music\lab\Polonaise\Polonaise-ABR128-3.95.1.wav
   15 out of 17, pval = 0.001
Original vs C:\My Music\lab\Polonaise\Polonaise-ABR128-3.90.3.wav
   13 out of 14, pval < 0.001
Go to the top of the page
+Quote Post
Continuum
post Jan 17 2004, 09:16
Post #47





Group: Members
Posts: 473
Joined: 7-June 02
Member No.: 2244



LoFiYo, try an ABX-test between the two encoded samples as well.
Go to the top of the page
+Quote Post
LoFiYo
post Jan 17 2004, 19:22
Post #48





Group: Members
Posts: 133
Joined: 2-January 04
Member No.: 10896



QUOTE (Continuum @ Jan 17 2004, 12:16 AM)
LoFiYo, try an ABX-test between the two encoded samples as well.

I gave it a try just now, but failed miserably. Actually I started out really well (up to the 7th or 8th trial), but ended up something like 9/13. It seemed that toward the end, my ears (and brain) got so tired that they stopped caring about the difference. I guess to me with this sample, the difference wasn't that big after all...
Go to the top of the page
+Quote Post
LoFiYo
post Jan 17 2004, 20:57
Post #49





Group: Members
Posts: 133
Joined: 2-January 04
Member No.: 10896



I tried a personal sample file (the first 30 seconds of It's a Sin by Pet Shop Boys). I remember reading somewhere that newer versions of Lame should be a little better at lower bitrates than 3.90.3, so I tried a very low bitrate this time (ABR80).

CODE
ABC/HR Version 0.9b, 30 August 2002
Testname: ABR 80 - 3.90.3 v 3.95.1

1L = C:\My Music\lab\sin\ABR80-3.90.3.wav
2L = C:\My Music\lab\sin\ABR80-3.95.1.wav

---------------------------------------
General Comments:
The fist thiry seconds of "It's a Sin" by Pet Shop Boys
---------------------------------------
1L File: C:\My Music\lab\sin\ABR80-3.90.3.wav
1L Rating: 3.3
1L Comment: Though the distortion is still noticeable, overall this sounds much better than the other encode.
---------------------------------------
2L File: C:\My Music\lab\sin\ABR80-3.95.1.wav
2L Rating: 2.3
2L Comment: Distortion/chirpiness is a little bit more noticeable throughout the sample.
---------------------------------------
ABX Results:
Original vs C:\My Music\lab\sin\ABR80-3.90.3.wav
   8 out of 8, pval = 0.004
Original vs C:\My Music\lab\sin\ABR80-3.95.1.wav
   8 out of 8, pval = 0.004


I couldn't ABX between the two encodes, but when ABXing the original vs 3.95.1, it was much easier to notice the difference than the original vs 3.90.3.
Go to the top of the page
+Quote Post
ViPER1313
post Jan 17 2004, 21:49
Post #50





Group: Members
Posts: 271
Joined: 19-August 02
From: Maryland
Member No.: 3109



ABC/HR Version 0.9b, 30 August 2002
Testname: BeautySlept --preset standard

1R = Z:\Music\Test Samples\testTemp\3.95.1 BeautySlept --preset standard.wav
2R = Z:\Music\Test Samples\testTemp\3.90.3 BeautySlept --preset standard.wav

---------------------------------------
General Comments:
I cant ABX this sample with either codec..... I think that sample 1 has somthing wrong with it but i cant ABX it with 100% certainty
---------------------------------------
ABX Results:
Original vs Z:\Music\Test Samples\testTemp\3.95.1 BeautySlept --preset standard.wav
17 out of 26, pval = 0.084
Original vs Z:\Music\Test Samples\testTemp\3.90.3 BeautySlept --preset standard.wav
4 out of 10, pval = 0.828
Z:\Music\Test Samples\testTemp\3.95.1 BeautySlept --preset standard.wav vs Z:\Music\Test Samples\testTemp\3.90.3 BeautySlept --preset standard.wav
4 out of 7, pval = 0.500
Go to the top of the page
+Quote Post

3 Pages V  < 1 2 3 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 22nd August 2014 - 22:17