IPB

Welcome Guest ( Log In | Register )

2 Pages V   1 2 >  
Reply to this topicStart new topic
LAME compiles test: Dibrom's & Mitiok's, A speed / size comparison
Megaman
post Nov 23 2002, 10:19
Post #1





Group: Members
Posts: 148
Joined: 27-October 02
From: Buenos Aires
Member No.: 3626



I've made a short test with two LAME binaries: Dibrom's 3.90.2 ICL latest compile and Mitiok's 3.92 latest compile.

I've uploaded a report here: LAME test.

Read at your own risk (!) , I'm not much more than a newbie so my conclusions might be completely wrong , however I tried to do my best.


--------------------
No se porqué
imaginé
que estabamos unidos
y me sentí mejor
pero aquí estoy
tan solo en la vida
que mejor me voy
Go to the top of the page
+Quote Post
Dibrom
post Nov 23 2002, 10:57
Post #2


Founder


Group: Admin
Posts: 2958
Joined: 26-August 02
From: Nottingham, UK
Member No.: 1



The only thing I'd be a little wary about is the conclusion that they don't sound different. I've mentioned this before, but one of the changes made after 3.90.2 was that /qrcd was no longer used in the Makefile. Granted, it is "bad form" (or whatever you want to refer to it as) because it changes the rounding behavior of the code slightly, but even so, I have noticed on at least one occassion that there can be slight degredations in quality without it (I noticed this on fatboy). I don't believe the LAME developers did any significant testing after removing this command from the makefile to make sure it did not impact quality. This is one of the reasons I don't recommend using 3.92 right now.. since there is more room for quality degredation because of this slight change in the compile time options. It's probably true that this won't affect most situations significantly, and that most people won't hear the difference anyway, but I still wouldn't make the conclusion that there is no detectable difference at all.
Go to the top of the page
+Quote Post
Megaman
post Nov 23 2002, 11:14
Post #3





Group: Members
Posts: 148
Joined: 27-October 02
From: Buenos Aires
Member No.: 3626



QUOTE (Dibrom @ Nov 23 2002 - 01:57 AM)
The only thing I'd be a little wary about is the conclusion that they don't sound different.......It's probably true that this won't affect most situations significantly, and that most people won't hear the difference anyway, but I still wouldn't make the conclusion that there is no detectable difference at all.

Right....maybe the difference would be noticed only on very critical samples but of course if you want the best possible quality under any possible circumstances no matter if highly critical or not , it would be reasonable to choose your compile , assuming that you heard that slight quality degradation with Mitiok's compile.
I will check out the fatboy sample myself although I doubt that I have the adequate gear to do any quality ABX testing at all , so I guess in the end it will be a matter of trust smile.gif.

This post has been edited by Megaman: Nov 23 2002, 11:21


--------------------
No se porqué
imaginé
que estabamos unidos
y me sentí mejor
pero aquí estoy
tan solo en la vida
que mejor me voy
Go to the top of the page
+Quote Post
robert
post Nov 23 2002, 12:11
Post #4


LAME developer


Group: Developer
Posts: 788
Joined: 22-September 01
Member No.: 5



I cannot recommend the use of 3.90.2 if that binary was compiled with the /qrcd switch turned on. It simply uses the wrong rounding mode.
Go to the top of the page
+Quote Post
Dibrom
post Nov 23 2002, 12:39
Post #5


Founder


Group: Admin
Posts: 2958
Joined: 26-August 02
From: Nottingham, UK
Member No.: 1



QUOTE (robert @ Nov 23 2002 - 04:11 AM)
I cannot recommend the use of 3.90.2 if that binary was compiled with the /qrcd switch turned on. It simply uses the wrong rounding mode.

This is what I was talking about.

Ironically, this statement is probably not made with any basis in quality testing.

It's just "wrong", so it doesn't matter if it sounds better or not. From a development point of view, this might be interesting, but from an end user point of view (something which LAME does not address very well.. tons of unnecessary experimental switches are an example of this), it is not. Most of the people on this site who are looking for recommended compiles and who looking at the --alt-presets are doing so because they want the best quality. They want something which has been verified via listening tests. I doubt they would really care whether or not it uses the "intended" rounding modes if the actual end result is better.

EDIT: I do understand the need for code to behave a certain way, but with the way LAME is right now (basically no quality testing or control), I don't think we can take anything for granted without testing its effects on quality -- this includes rounding behavior (in that, the behavior of the ICL compiles was changed after 3.90.2, but I don't think any quality testing was done by the people making this change). The 3.93 release should be a lesson in this...

So basically what I'm trying to say is that regardless of how 3.90.2 works at a code level, regardless of rounding modes or anything which should work a certain way, I'm only making a judgement on how it actually sounds in practice -- I'm not going to make any assumptions. I think this is, at least right now, the only true way to determine the actual quality of the output. 3.90.2 has been verified to be good.. 3.92 hasn't, and 3.93 has been verified to not be good.

This post has been edited by Dibrom: Nov 23 2002, 12:52
Go to the top of the page
+Quote Post
ErikS
post Nov 23 2002, 13:20
Post #6





Group: Members
Posts: 757
Joined: 8-October 01
Member No.: 247



QUOTE (Dibrom @ Nov 23 2002 - 10:57 AM)
The only thing I'd be a little wary about is the conclusion that they don't sound different.  I've mentioned this before, but one of the changes made  after 3.90.2 was that /qrcd was no longer used in the Makefile.  Granted, it is "bad form" (or whatever you want to refer to it as) because it changes the rounding behavior of the code slightly, but even so, I have noticed on at least one occassion that there can be slight degredations in quality without it (I noticed this on fatboy).  I don't believe the LAME developers did any significant testing after removing this command from the makefile to make sure it did not impact quality.  This is one of the reasons I don't recommend using 3.92 right now.. since there is more room for quality degredation because of this slight change in the compile time options.  It's probably true that this won't affect most situations significantly, and that most people won't hear the difference anyway, but I still wouldn't make the conclusion that there is no detectable difference at all.

I assume you made an ABX test... I can't hear this myself, but are there other persons who can pass an ABX on this sample? Can you repeat the test yourself and pass it again? What I mean: both Takehiro and Robert have now discouraged the use of this switch and it seems you are the only one who claims the opposite with one(?) abx test made some time ago. It would be good if your results on the ABX test was repeatable.

This post has been edited by ErikS: Nov 23 2002, 13:21
Go to the top of the page
+Quote Post
JohnV
post Nov 23 2002, 14:31
Post #7





Group: Developer
Posts: 2797
Joined: 22-September 01
Member No.: 6



QUOTE (ErikS @ Nov 23 2002 - 02:20 PM)
What I mean: both Takehiro and Robert have now discouraged the use of this switch and it seems you are the only one who claims the opposite with one(?) abx test made some time ago. It would be good if your results on the ABX test was repeatable.

Hmm, you seem to forget that the whole development of --aps was made using Dib's compiles. It was all based on listening tests. Granted, I think 3.92 is pretty much the same "quality level" though.

It's quite ironic that the Lame devs are so concerned about some rounding issue, and even recommend something based on it, but then the real problems seem to go unnoticed..
http://www.geocrawler.com/lists/3/SourceFo...323/0/10244994/


--------------------
Juha Laaksonheimo
Go to the top of the page
+Quote Post
Megaman
post Nov 23 2002, 18:49
Post #8





Group: Members
Posts: 148
Joined: 27-October 02
From: Buenos Aires
Member No.: 3626



I agree with Dibrom's statement: what the end user wants is the highest possible perceptual quality no matter the rounding mode.Of course , if the wrong rounding mode results in a binary which delivers files of perceptually superior quality (even if only on critical samples) is something that needs to be backed up with several , seriously made listening tests.

Edit: I'm absolutely paranoid about my english wacko.gif

This post has been edited by Megaman: Nov 24 2002, 05:48


--------------------
No se porqué
imaginé
que estabamos unidos
y me sentí mejor
pero aquí estoy
tan solo en la vida
que mejor me voy
Go to the top of the page
+Quote Post
ErikS
post Nov 24 2002, 10:21
Post #9





Group: Members
Posts: 757
Joined: 8-October 01
Member No.: 247



QUOTE (JohnV @ Nov 23 2002 - 02:31 PM)
Hmm, you seem to forget that the whole development of --aps was made using Dib's compiles. It was all based on listening tests. Granted, I think 3.92 is pretty much the same "quality level" though.

So?

1. Listening tests don't mean anything if they are not reproducable.... Can you hear any improvement with this compiler switch?

2. If the switch was used throughout the development of the alt-preset:s then it is very possible that something else was tuned to compensate for that "wrong" rounding mode, and when you switch back to the "correct" rounding mode it overcompensates and throws something else off. If this is the case the right approach is to fix lame so it can use the correct rounding mode. Many people don't use icl compiles - why should they suffer?
Go to the top of the page
+Quote Post
Megaman
post Nov 24 2002, 10:31
Post #10





Group: Members
Posts: 148
Joined: 27-October 02
From: Buenos Aires
Member No.: 3626



I think that 99% out there use ICL compiles.
Mitiok's official compile is compiled with ICL (am I right?).I guess at least 98% of non-HA members use Mitiok's compile.


--------------------
No se porqué
imaginé
que estabamos unidos
y me sentí mejor
pero aquí estoy
tan solo en la vida
que mejor me voy
Go to the top of the page
+Quote Post
ErikS
post Nov 24 2002, 10:54
Post #11





Group: Members
Posts: 757
Joined: 8-October 01
Member No.: 247



So 99% of the people use windows? I doubt it.
Go to the top of the page
+Quote Post
Dibrom
post Nov 24 2002, 11:09
Post #12


Founder


Group: Admin
Posts: 2958
Joined: 26-August 02
From: Nottingham, UK
Member No.: 1



QUOTE (ErikS @ Nov 24 2002 - 02:21 AM)
QUOTE (JohnV @ Nov 23 2002 - 02:31 PM)
Hmm, you seem to forget that the whole development of --aps was made using Dib's compiles. It was all based on listening tests. Granted, I think 3.92 is pretty much the same "quality level" though.

So?

1. Listening tests don't mean anything if they are not reproducable.... Can you hear any improvement with this compiler switch?

2. If the switch was used throughout the development of the alt-preset:s then it is very possible that something else was tuned to compensate for that "wrong" rounding mode, and when you switch back to the "correct" rounding mode it overcompensates and throws something else off. If this is the case the right approach is to fix lame so it can use the correct rounding mode. Many people don't use icl compiles - why should they suffer?

I'm not really going to spend much effort on this for the following reasons.

1. I don't have time.
2. I'm tired of arguing pointlessly about LAME.
3. I simply don't care enough anymore.

To address your first point, I agree with you that listening tests are pointless if they are not reproducable. I do not plan on reproducing this particular listening test anytime soon though. You can ask the other LAME developers if they would like to do so though.

As to whether or not my statement means anything aside from that in regards to this matter, well, I've done more reproducable listening tests for LAME than any of the other developers have ever done. Most of them were blind, and nearly all of my results have been verified by other parties. The fruits of my labor in regard to past listening tests can be heard through the --alt-presets.

Additionally, Robert and Takehiro are not authorities on quality. It was commits from both of them to CVS which have resulted in 3.93's broken quality. Given that, it's understandable for me to question their statements about what is quality and what isn't. Furthermore, I've never seen an effort from either of them to do comprehensive blind testing with results verified by any sort of community. This is not a reflection on their character, so please don't take this out of context. I get along with these guys fine on a personal level, I'm only attempting to show why I don't believe their comments on this matter hold a lot of weight.

As for your second point... if they feel that there is a problem with the rounding mode in /qrcd, and that the --alt-presets have been tuned with this in mind (which is probably true, except for the possibility of a "problem" with the rounding mode.. this is unclear.. especially if the rounding mode led to higher quality), then they are more than welcome to correct the problem themselves. Unfortunately, I don't think this will be easy without some extensive listening tests, which I don't think either of them has the time or desire to do -- it's easier to just claim that it's "wrong" and be done with it.

Nobody is made to "suffer" with the --alt-presets. I am not forcing anyone to use anything, I am only making a recommendation. I tuned the --alt-presets according to the code that was already in LAME (the /qrcd switch was put there by the LAME developers, not I. Only after the --alt-presets were implemented was it removed), so it is not my responsibility to make sure that they still work after someone else changes their functionality. It should be the responsibility of those who have made the change to put forth the effort to verify that quality has not taken a hit. If that means asking me to help, that's fine.. but I was not asked to help. Some may say that it is my responsibility, because I'm a LAME developer. I would disagree with this. LAME is, accordingly, driven by consensus. My input into this consensus has always been ignored. Because of this, and because of that fact that I don't think the other developers take me seriously (and thus, don't really consider me a peer), I don't consider myself to be a LAME developer. To be honest, I'm not even sure why the --alt-presets ever were included in the first place, other than it was just another thing that could be added to LAME.

Ultimately I agree with your assessment that if /qrcd improves quality, then changes in LAME itself should be made. Who is going to do this though? I don't think the people complaining about /qrcd are going to do it. Complaining to me about it isn't going to accomplish anything, because I have no control over LAME. Furthermore, there is very little initiative or motivation in the LAME project to improve quality at such fine levels. In addition, even if we could prove that there was a beneficial change that could be incorporated into LAME, you'd likely meet with much resistance to implementing your idea (basically, it'd never happen).

I recently had a discussion with Takehiro about the future of LAME and about large scale changes that should be made to the code. He basically told me that LAME is too mature and too much of the code is interdepedant for there to be any more large scale modifications at this point. He mentioned that he didn't believe LAME would ever improve or change significantly at this point. Although I don't agree with the ideas that change can't happen, I do agree with his assessment of this particular situation, given all that has happened (and that which hasn't) over the past year.
Go to the top of the page
+Quote Post
Dibrom
post Nov 24 2002, 11:11
Post #13


Founder


Group: Admin
Posts: 2958
Joined: 26-August 02
From: Nottingham, UK
Member No.: 1



QUOTE (ErikS @ Nov 24 2002 - 02:54 AM)
So 99% of the people use windows? I doubt it.

I guess you haven't done much research. ICL is available for free on Linux, so that thows the "only windows" bit out. I don't know if it compiles LAME, but the possibility it certainly there.

And, FWIW, gcc is not a very good compiler speed wise. I get much better results running the 3.90.2 compile under WINE then I do compiling it with gcc.
Go to the top of the page
+Quote Post
dev0
post Nov 24 2002, 11:16
Post #14





Group: Developer
Posts: 1679
Joined: 23-December 01
From: Germany
Member No.: 731



Did you try GCC 3.2? From what I've heard it's almost as fast as ICL.
Go to the top of the page
+Quote Post
Benjamin Lebsanf...
post Nov 24 2002, 11:39
Post #15





Group: Members
Posts: 761
Joined: 29-September 01
Member No.: 40



are there win32 builds ?
Go to the top of the page
+Quote Post
Dibrom
post Nov 24 2002, 11:40
Post #16


Founder


Group: Admin
Posts: 2958
Joined: 26-August 02
From: Nottingham, UK
Member No.: 1



QUOTE (dev0 @ Nov 24 2002 - 03:16 AM)
Did you try GCC 3.2? From what I've heard it's almost as fast as ICL.

No, I've tried GCC 3 or 3.1, forget which.

I'd be very surprised if GCC 3.2 was as fast as ICL though.
Go to the top of the page
+Quote Post
john33
post Nov 24 2002, 13:46
Post #17


xcLame and OggDropXPd Developer


Group: Developer
Posts: 3760
Joined: 30-September 01
From: Bracknell, UK
Member No.: 111



QUOTE (Dibrom @ Nov 24 2002 - 10:40 AM)
QUOTE (dev0 @ Nov 24 2002 - 03:16 AM)
Did you try GCC 3.2? From what I've heard it's almost as fast as ICL.

No, I've tried GCC 3 or 3.1, forget which.

I'd be very surprised if GCC 3.2 was as fast as ICL though.

It seems to depend rather on the application. The GCC3.2 compiles I made of the OggVorbis apps ran as fast, if not marginally faster in some cases, than the ICL compiles. However, I did produce 3.2 compiles of LAME for my own testing and they were significantly slower than the ICL compiles. So I guess the only conclusion to be drawn is that it depends on how the code is written. wink.gif


--------------------
John
----------------------------------------------------------------
My compiles and utilities are at http://www.rarewares.org/
Go to the top of the page
+Quote Post
ErikS
post Nov 24 2002, 20:43
Post #18





Group: Members
Posts: 757
Joined: 8-October 01
Member No.: 247



QUOTE (Dibrom @ Nov 24 2002 - 11:11 AM)
I guess you haven't done much research.  ICL is available for free on Linux, so that thows the "only windows" bit out.  I don't know if it compiles LAME, but the possibility it certainly there.

My bad. I assumed you needed MSDev to use ICL, plus I never heard that neither MSDev nor ICL exitsted for any other OS. But that is because I didn't look.
Go to the top of the page
+Quote Post
Gabriel
post Nov 25 2002, 08:36
Post #19


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



There is ICL for linux, but I'm not sure that it's free. Under win32, at least, it's not free at all...
Go to the top of the page
+Quote Post
Dibrom
post Nov 25 2002, 09:31
Post #20


Founder


Group: Admin
Posts: 2958
Joined: 26-August 02
From: Nottingham, UK
Member No.: 1



QUOTE (Gabriel @ Nov 25 2002 - 12:36 AM)
There is ICL for linux, but I'm not sure that it's free. Under win32, at least, it's not free at all...

It's free and "unsupported", but I think you do have to register at their developer website to download it. Gentoo, which is a source based distro, has been working on ICL support (to compile the entire distro) for quite some time.. and I'm pretty sure they wouldn't be doing this if it wasn't available freely.

And I'm aware that the win32 version is non-free.
Go to the top of the page
+Quote Post
Gabriel
post Nov 25 2002, 10:20
Post #21


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



Free under linux...That's interesting...
Go to the top of the page
+Quote Post
john33
post Nov 25 2002, 13:14
Post #22


xcLame and OggDropXPd Developer


Group: Developer
Posts: 3760
Joined: 30-September 01
From: Bracknell, UK
Member No.: 111



QUOTE (Gabriel @ Nov 25 2002 - 09:20 AM)
Free under linux...That's interesting...

And they only want your name, email address and country to download! wink.gif


--------------------
John
----------------------------------------------------------------
My compiles and utilities are at http://www.rarewares.org/
Go to the top of the page
+Quote Post
joro_09
post Nov 25 2002, 13:26
Post #23





Group: Members
Posts: 3
Joined: 21-November 02
From: Bulgaria
Member No.: 3835



Hi,
I've been reading different forums and recommendations about quality MP3 encoding for about 2 months up to now, but I still hesitate what switches to use. To be clear, my intention is to encode all my music in MP3 archive with best possible quality, but to be able (if I decide for some reason to do so) to decompress it again and write it to Audio Discs, without furthermore loss of quality.
So here are some questions for you:
1. Some people say --alt-preset insane is the best quality you can get from MP3 format (at least for LAME 3.92). Other people say extreme is better because of its intelligence during coding simple and harder parts of a song (e.g. extreme is better than insane in passing on the stereo panorama). What is the truth?!
2. Is there any official place where developers classify, in order of their quality, the LATEST, TESTED and STABLE switches (presets) for command line? It is very confusing to me to see different developers recommending their own switches and to choose between them… Which is best!?
3. These days I understood LAME 3.93 have been released. Is it safe to use it, or should I wait for further tuning? Or is the lame3.90.2 better?
Thanks in advance for helping me.

Sincerely, George
Go to the top of the page
+Quote Post
KikeG
post Nov 25 2002, 13:54
Post #24


WinABX developer


Group: Developer
Posts: 1578
Joined: 1-October 01
Member No.: 137



- According to my experience and others, --alt-preset insane is the best quality you can achieve, better than with alt-preset extreme.

- 3.93 is not advised, better use 3.90.2
Go to the top of the page
+Quote Post
Chun-Yu
post Nov 25 2002, 14:30
Post #25





Group: Developer
Posts: 359
Joined: 29-October 02
Member No.: 3652



BTW: The Intel C++ Compiler 7.0 has recently been released (but I haven't had time to try it yet). The best thing about it is that they've added a "student license" that's only $30 - "Student pricing is for non-commercial licenses only. The licenses are individual, unsupported user licenses without a timeout on the license."
Go to the top of the page
+Quote Post

2 Pages V   1 2 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 30th July 2014 - 12:59