IPB

Welcome Guest ( Log In | Register )

8 Pages V  < 1 2 3 4 > »   
Reply to this topicStart new topic
MP3 at 128kbps public listening test, Discussion thread
danchr
post Dec 10 2003, 10:09
Post #26





Group: Members
Posts: 487
Joined: 6-April 03
From: Århus, Denmark
Member No.: 5861



QUOTE (rjamorim @ Dec 10 2003, 05:09 AM)
Opinions? Maybe make the test all CBR?

Considering a lot of people still use the fact that the first AAC test was CBR only as an argument against it, I'd say that's a bad idea. IMO you should just let every codec perform the best it possibly can - as long as ordinary people can be expected to make use those settings.

QUOTE (rjamorim @ Dec 10 2003, 05:57 AM)
Yes, I'm quite sure it is SoundJam.

If SoundJam indeed used an alternative MP3 encoder, then it's quite likely that's what iTunes uses since the encoding dialog pretty much hasn't changed since SoundJam. Plus, iTunes 1.0 was basically a scaled down version of SoundJam with a different GUI. It'd be very interesting to know how it performs - I've never seen any tests of it smile.gif

How about using HE-AAC @ 64kbps for lower anchor? It just might be it lives up to it's "same quality, half bitrate" claim! laugh.gif

This post has been edited by danchr: Dec 10 2003, 10:12
Go to the top of the page
+Quote Post
bond
post Dec 10 2003, 10:22
Post #27





Group: Members
Posts: 881
Joined: 11-October 02
Member No.: 3523



what about using mp3pro@64 as anchor for the mp3 test?
i think it would fit as an anchor qualitywise and the results would also be really interesting...


for the aac test i wouldnt drop sorenson (it wasnt that bad at all)


and i would vote for a clip with sound and human voice (as from movies)


--------------------
I know, that I know nothing (Socrates)
Go to the top of the page
+Quote Post
Gabriel
post Dec 10 2003, 10:59
Post #28


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



My opinion:

*using Xing as low anchor seems to be a good idea, and more usefull than dist10.
*comparing the 3 main encoding engines used nowadays is a good thing (Fhg, ITunes, Lame)
*Including the old (but still widely used because of the Radium hack) AudioActive is a good thing.

Now, we could ask why only using current FhG engines in vbr only. The same (reversed) question could be asked about Lame, although I also think that it is very likely that cbr/abr would be better than vbr in those bitrates.

In order not to increase too much the number of candidates, I'd suggest an all cbr test. I know that this would penalize Lame (compared to abr), but otherwise there is the risk of endless debates.

Btw, I'd also be interested by a cbr/abr/vbr test for FhG and Lame, but this should probably belong to another test.

Last question: If a new Lame version is out by the time of the test, which one will you use? 3.90.3, 3.93.1 or new one?
Go to the top of the page
+Quote Post
SacRat
post Dec 10 2003, 11:52
Post #29





Group: Members
Posts: 222
Joined: 8-November 02
From: Sarov, Russia
Member No.: 3719



Agree with Gabriel on MP3 encoders. IMHO we should use all codecs in CBR mode as the most popular one. At least, this way no one would be able to complain on "cheating" smile.gif
@Gabriel... I think , your question about LAME versions is interesting. At least there must be a difference between 3.90.3 and latest stable release (otherwise I don'tknow what you did all this time)... maybe, we could include both? And if so, what settings would you suggest? alt-preset CBR?

Incluging Vorbis into AAC test might also be very interesting. At least there were numerous talks about "AAC vs OGG" here and direct comparison could clear some things.


--------------------
http://sacrat.blogspot.com
Go to the top of the page
+Quote Post
bond
post Dec 10 2003, 12:01
Post #30





Group: Members
Posts: 881
Joined: 11-October 02
Member No.: 3523



i would test all codecs with abr, if a codec doesnt offer this option, we need to use the next best (ie vbr or cbr)

imho the bad codecs should be punished, not the good ones tongue.gif


--------------------
I know, that I know nothing (Socrates)
Go to the top of the page
+Quote Post
Gabriel
post Dec 10 2003, 12:02
Post #31


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



QUOTE
And if so, what settings would you suggest? alt-preset CBR?

For a cbr encoding that would be

*3.90.3: --alt-preset cbr 128
*3.93.1: --preset cbr 128
*3.94: -b 128
Go to the top of the page
+Quote Post
magic75
post Dec 10 2003, 12:09
Post #32





Group: Members
Posts: 511
Joined: 2-December 02
Member No.: 3959



I can agree on skipping VBR since it is difficult to use that in a fair comparison with other encoders, but I don't like the idea of skipping ABR for Lame. I don't that would implicate an unfair comparison.

If Lame 3.94 makes it out on time I vote for skipping one of the FhG encoders and including that one as well.

BTW, thanks rjamorim for doing all these tests. Really great work. I am really looking forward to the march test, to see how MP3 compares to WMA std.
Go to the top of the page
+Quote Post
rjamorim
post Dec 10 2003, 14:46
Post #33


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (magic75 @ Dec 10 2003, 06:32 AM)
I am not quite sure I understand you correctly, but if you are talking about the fact that you can't find any Lame, FhG or dist10 on any warez sites, the explanation is quite straightforward. Warez sites contain cracked and illegal software, but both Lame and FhG can be obtained legally for free, so why would anyone publish them on a warez site?

He was talking about the three official WMP codecs: CyberLink, InterVideo and (whatever was the third one).

QUOTE
How about using HE-AAC @ 64kbps for lower anchor? It just might be it lives up to it's "same quality, half bitrate" claim! laugh.gif


If Ivan managed to fix EnolaGay, it might well surpass some of the MP3 encoders wink.gif

Still, I'm pretty fond of the idea of ruining Xing's reputation once and for all

QUOTE
for the aac test i wouldnt drop sorenson (it wasnt that bad at all)


Indeed, but it didn't change at all since last test. So, people can already have an idea of it's quality by looking at previous test.

QUOTE
and i would vote for a clip with sound and human voice (as from movies)


OK. But first recommend which clip should be used and which clip should be replaced wink.gif

QUOTE
Btw, I'd also be interested by a cbr/abr/vbr test for FhG and Lame, but this should probably belong to another test.


Well, if most people agree on making this test CBR only, I might well later conduce a VBR FhG vs. ABR Lame test. I'm working for you guys, so it's your decision the way the test will go.

QUOTE
Last question: If a new Lame version is out by the time of the test, which one will you use? 3.90.3, 3.93.1 or new one?


To be very sincere, I didn't make up my mind yet on that subject. There are reasons to use the HA version - the very same reasons why HA recommends that fork instead of latest official release.

And then again, there are several reasons to test latest official release: check how quality is progressing, help the developers, etc.

So, which one you recommend, as a lame developer? wink.gif

QUOTE
Agree with Gabriel on MP3 encoders. IMHO we should use all codecs in CBR mode as the most popular one. At least, this way no one would be able to complain on "cheating"


I gotta admit that appeals a lot to me, particularly :B

QUOTE
maybe, we could include both?


We are at a point where, if you suggest another encoder, please suggest what should be replaced. 6 codecs is the maximum, period. Even at 64kbps testing 8 codecs was a mistake and I don't want it to happen again.

QUOTE
Incluging Vorbis into AAC test might also be very interesting. At least there were numerous talks about "AAC vs OGG" here and direct comparison could clear some things.


I agree that would be interesting. But I'm not fond of conducing this test myself. The reason? Everybody know I'm biased towards AAC and against Vorbis. Do you have an idea of what would happen if an AAC encoder won?

That said, it would be my pleasure to help and guide anyone interested in conducing such test.

QUOTE
If Lame 3.94 makes it out on time I vote for skipping one of the FhG encoders and including that one as well.


OK. What FhG encoder, specifically?

Thanks a HUGE bunch for all your comments smile.gif

Best regards;

Roberto.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
magic75
post Dec 10 2003, 15:27
Post #34





Group: Members
Posts: 511
Joined: 2-December 02
Member No.: 3959



QUOTE
OK. What FhG encoder, specifically?

To be honest I didn't consider that, really. I just thought it would be more interesting to see the possible progress of Lame rather than three FhG. But maybe Gabriel can say something about what kind of improvements he expects for the 128 kbps settings in 3.94, before considering having both any further?

OK, here is my very personal wishlist for what I would like to see compared:
- Lame 3.90.3 preset 128 (yes, ABR...)
- FhG 128 CBR "standard newbie encoding", something you would get from Musicmatch etc. typically with default settings
- FhG 128 CBR HQ, but I really don't know what software&setting to produce these, it should be the best possible you can get from FhG
- iTunes 128 CBR
- Xing 128 CBR "standard newbie encoding" as "defined" above...

If I were to add something to this list it would be:
1. Gogo
2. Lame 3.94

These are all just suggestions...
Go to the top of the page
+Quote Post
kwanbis
post Dec 10 2003, 15:41
Post #35





Group: Developer (Donating)
Posts: 2390
Joined: 28-June 02
From: Argentina
Member No.: 2425



QUOTE (rjamorim @ Dec 10 2003, 01:17 AM)
There would be 5 codecs and an anchor:

-Lame - --alt-preset 128
-FhG Audition Legacy Slow - VBR 60~70
-FhG Audition Current - VBR 60~70
-FhG Audioactive - 128kbps high quality
-iTunes - 128kbps
-Xing 128kbps as anchor

i think you should include LAME -APS, cause it is really the standard here, and it would be very nice to see how it compares to -ap 128 and the others.


--------------------
MAREO: http://www.webearce.com.ar
Go to the top of the page
+Quote Post
Gabriel
post Dec 10 2003, 15:47
Post #36


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



QUOTE
So, which one you recommend, as a lame developer?

My own choice would be to test the latest beta or release (of course alpha are excluded).
In my mind, this is the more usefull, as it is representative of the current state of the codec, and could be helpfull in order to fix potential issues.

However, I can already hear those numerous people that will argue that the "HA recommended version" should be used, and nothing else.
(please note that in order to enforce this argument, it means that the "recommended version" has to be tested against latest one)

The problem is that including both 3.94 (if released in time) and 3.90.3 would mean dropping another candidate, and I like every other candidate.
Go to the top of the page
+Quote Post
Gabriel
post Dec 10 2003, 15:49
Post #37


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



QUOTE
i think you should include LAME -APS, cause it is really the standard here, and it would be very nice to see how it compares to -ap 128 and the others.


This is a test of 128kbps mp3 encodings. I do not think that preset standard would fit in a 128kbps test.
Go to the top of the page
+Quote Post
kwanbis
post Dec 10 2003, 16:03
Post #38





Group: Developer (Donating)
Posts: 2390
Joined: 28-June 02
From: Argentina
Member No.: 2425



QUOTE (Gabriel @ Dec 10 2003, 02:49 PM)
This is a test of 128kbps mp3 encodings. I do not think that preset standard would fit in a 128kbps test.

well, this is a test of quality, and *I* would really love to know how all this encoders compare to -APS, i mean, is it really APS that good? ore have we all been talking bs all this time?


--------------------
MAREO: http://www.webearce.com.ar
Go to the top of the page
+Quote Post
Yaztromo
post Dec 10 2003, 16:13
Post #39





Group: Members
Posts: 236
Joined: 28-July 03
From: England, UK
Member No.: 8031



Since we're targeting how well MP3 codecs perform at 128kbps then it makes sense to use CBR for all encoding tests. Since CBR is what most average non-HA users encode with. Giving LAME use of ABR and forcing the rest to use CBR seems an unfair test.
Go to the top of the page
+Quote Post
sthayashi
post Dec 10 2003, 16:13
Post #40





Group: Members
Posts: 494
Joined: 16-April 03
From: Pittsburgh, PA
Member No.: 5997



QUOTE (kwanbis @ Dec 10 2003, 10:03 AM)
QUOTE (Gabriel @ Dec 10 2003, 02:49 PM)
This is a test of 128kbps mp3 encodings. I do not think that preset standard would fit in a 128kbps test.

well, this is a test of quality, and *I* would really love to know how all this encoders compare to -APS, i mean, is it really APS that good? ore have we all been talking bs all this time?

The problem with APS testing is that 1) you would have fewer people testing (and then rjamorim would run into the same problem that I did with MY listening test) and 2) you would have to have highly suggested/optimized settings for all the other codecs, which is simply impossible. Like what's the APS equivalent on the FhG encoders?

Now if you'd like, I can arrange a personal test for you so that you see if you can hear the differences between APS and anything else, but it's not really good for a public test Even at 128, there will be people that aren't able to ABX it from the original.
Go to the top of the page
+Quote Post
KristianT
post Dec 10 2003, 16:27
Post #41





Group: Members
Posts: 9
Joined: 6-June 03
Member No.: 7035



QUOTE
The problem is that including both 3.94 (if released in time) and 3.90.3 would mean dropping another candidate, and I like every other candidate.


Gabriel is right on. Personally, I think the best way to conduct this test would be to use 3.90.3 since it's the recommended version. THEN... when all three tests are done, and we (hopefully) have LAME 3.94 it would be the right time for testing it against 3.93.1 and 3.90.3. This test has been needed for a long time and if I knew how to conduct it, I would make it a top priority.

QUOTE
IMO you should just let every codec perform the best it possibly can - as long as ordinary people can be expected to make use those settings.


Indeed! Let's not cripple good codecs.

Finally, I can see why XING would make an interesting anchor but considering the ones who will actually take time examining the results, I feel HE-AAC would be more interesting. Sure, it would be somewhat misplaced but then again it would provide useful information. In my opinion XING wouldn't.
Go to the top of the page
+Quote Post
sthayashi
post Dec 10 2003, 16:28
Post #42





Group: Members
Posts: 494
Joined: 16-April 03
From: Pittsburgh, PA
Member No.: 5997



QUOTE (rjamorim @ Dec 10 2003, 08:46 AM)
QUOTE
Incluging Vorbis into AAC test might also be very interesting. At least there were numerous talks about "AAC vs OGG" here and direct comparison could clear some things.

I agree that would be interesting. But I'm not fond of conducing this test myself. The reason? Everybody know I'm biased towards AAC and against Vorbis. Do you have an idea of what would happen if an AAC encoder won?

That said, it would be my pleasure to help and guide anyone interested in conducing such test.

I'm still a bit sore from the results (or lack thereof) from my last listening test, otherwise, I'd be happy to volunteer. Besides, you'll get that accusation in your 3rd test anyways, when either Nero or Quicktime (the expected winners of the AAC test, IMO) does nominally better than Vorbis (my expectations for your 128kbps extension).

It's a test that ought to be done, dammit. laugh.gif
Go to the top of the page
+Quote Post
schnofler
post Dec 10 2003, 16:30
Post #43


Java ABC/HR developer


Group: Developer
Posts: 175
Joined: 17-September 03
Member No.: 8879



QUOTE (magic75)
- Lame 3.90.3 preset 128 (yes, ABR...)
- FhG 128 CBR "standard newbie encoding", something you would get from Musicmatch etc. typically with default settings
- FhG 128 CBR HQ, but I really don't know what software&setting to produce these, it should be the best possible you can get from FhG
- iTunes 128 CBR
- Xing 128 CBR "standard newbie encoding" as "defined" above...

I mostly agree with this list, except for the "FhG 128 CBR HQ" bit. I think the test should at least include the optimal configurations of the two "top competitors", Lame and FhG, and that would also mean allowing FhG to use VBR. Using only CBR doesn't make a lot of sense in my opinion. Even if it is the most common newbie setting, so what? We already have Xing in the test, which should suffice to prove that the typical "newbie settings" are not the optimum. Limiting all the other codecs to CBR would make the test very one-sided.
I think one of the main goals of an MP3 128kbit/s test should be to find the best codec at this bitrate. We will never find this out if we limit the codecs artificially. Additionally, using CBR would penalize the MP3 format in the upcoming multiformat test (if the winner of the MP3 test is used) because it doesn't give an accurate image of what the format is capable of at that bitrate.
As for the sixth codec, I would suggest Lame 3.94 (or 3.93.1 if 3.94 is not ready by then). I guess a good result of a current Lame version would be a solid basis for a new HA recommendation (as "lack of proper testing" seems to be the main argument against recommending the newer Lame versions).

This post has been edited by schnofler: Dec 10 2003, 17:26
Go to the top of the page
+Quote Post
ff123
post Dec 10 2003, 16:33
Post #44


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



QUOTE (rjamorim @ Dec 9 2003, 08:57 PM)
I was under the impression that FhG Current was the slow codec. In this case, I'll do current and legacy slow. Thanks for the tip. I'll update my first post now.

You would know for sure which codec is the slow one. It's super slow.

QUOTE
QUOTE
I agree that the iTunes mp3 encoder should be tested if at all possible (I don't think it's FhG based).


Yes, I'm quite sure it is SoundJam.


Yep. It is apparently quite good.

QUOTE
QUOTE
There's also the FhG slow codec (high quality codec within fastencc.exe)


Huh? So fastencc isn't the fast codec?


It contains both the fast and slow codecs. The fast codec within fastencc.exe is buggy (loss of stereo separation). The slow one (-hq) works fine.

ff123
Go to the top of the page
+Quote Post
danchr
post Dec 10 2003, 16:40
Post #45





Group: Members
Posts: 487
Joined: 6-April 03
From: Århus, Denmark
Member No.: 5861



QUOTE (kwanbis @ Dec 10 2003, 04:03 PM)
well, this is a test of quality, and *I* would really love to know how all this encoders compare to -APS, i mean, is it really APS that good? ore have we all been talking bs all this time?

People aren't talking bullshit. Judging from most ABX tests I've read about, LAME --alt-preset standard is virtually transperant, but with a couple of known problem samples. The flaws of --alt-preset standard seem to be due to either limitations in the MP3 format, or problems with LAME's psychoacoustic model. The same thing can be said about almost every other encoder...
Go to the top of the page
+Quote Post
Gabriel
post Dec 10 2003, 16:40
Post #46


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



QUOTE
Gabriel is right on. Personally, I think the best way to conduct this test would be to use 3.90.3 since it's the recommended version.

I hope that I wasn't misunderstood. My own choice would be to use 3.94, but I pointed that other people might have different opinions.
Go to the top of the page
+Quote Post
kwanbis
post Dec 10 2003, 16:45
Post #47





Group: Developer (Donating)
Posts: 2390
Joined: 28-June 02
From: Argentina
Member No.: 2425



QUOTE (danchr @ Dec 10 2003, 03:40 PM)
People aren't talking bullshit. Judging from most ABX tests I've read about, LAME --alt-preset standard is virtually transperant, but with a couple of known problem samples. The flaws of --alt-preset standard seem to be due to either limitations in the MP3 format, or problems with LAME's psychoacoustic model. The same thing can be said about almost every other encoder...

yeah, but is it really that better? i use it all the time, but, is it really worth it?


--------------------
MAREO: http://www.webearce.com.ar
Go to the top of the page
+Quote Post
rpop
post Dec 10 2003, 16:47
Post #48





Group: Super Moderator
Posts: 332
Joined: 20-May 03
From: Pittsburgh, USA
Member No.: 6718



QUOTE (rjamorim @ Dec 10 2003, 12:09 AM)
QUOTE (CosmoKramer @ Dec 10 2003, 01:47 AM)
What about --alt-preset cbr 128? I for one would like to see that one.

No problem, but that would replace Lame ABR.

Opinions? Maybe make the test all CBR?

I'm in favor of this. Making some VBR or ABR is rather unfair, IMO, because in my experience, most VBR and ABR modes tend to have, on average, a bitrate that is higher or lower than 128kbps.

From what I learned at school (in science biggrin.gif), for best results in an experiment, only one variable should be modified. In this case, that would be the encoder. By using VBR and ABR, the bitrate is also modified, as well as the encoder's specific implementation of VBR/ABR. For example, in my experience, LAME ABR 128 consistently produces ABRs below 128.


--------------------
[url=http://noveo.net/ph34r.htm]Happiness[/url] - The agreeable sensation of contemplating the misery of others.
Go to the top of the page
+Quote Post
DigitalDictator
post Dec 10 2003, 16:50
Post #49





Group: Members
Posts: 313
Joined: 9-August 02
From: SoFo
Member No.: 3002



QUOTE
i would test all codecs with abr, if a codec doesnt offer this option, we need to use the next best (ie vbr or cbr) imho the bad codecs should be punished, not the good ones

I TOTALLY AGREE!

This post has been edited by DigitalDictator: Dec 10 2003, 16:51
Go to the top of the page
+Quote Post
Continuum
post Dec 10 2003, 17:07
Post #50





Group: Members
Posts: 473
Joined: 7-June 02
Member No.: 2244



QUOTE (schnofler @ Dec 10 2003, 04:30 PM)
QUOTE (magic75)
- Lame 3.90.3 preset 128 (yes, ABR...)
- FhG 128 CBR "standard newbie encoding", something you would get from Musicmatch etc. typically with default settings
- FhG 128 CBR HQ, but I really don't know what software&setting to produce these, it should be the best possible you can get from FhG
- iTunes 128 CBR
- Xing 128 CBR "standard newbie encoding" as "defined" above...

I mostly agree with this list, except for the "FhG 128 CBR HQ" bit. I think the test should at least include the optimal configurations of the two "top competitors" Lame and FhG and that would also mean allowing FhG to use VBR.

Add another vote for this selection! Two FhGs, one "default" the other "best quality", seem enough.


QUOTE (kwanbis @ Dec 10 2003, 04:45 PM)
QUOTE (danchr @ Dec 10 2003, 03:40 PM)
People aren't talking bullshit. Judging from most ABX tests I've read about, LAME --alt-preset standard is virtually transperant, but with a couple of known problem samples. The flaws of --alt-preset standard seem to be due to either limitations in the MP3 format, or problems with LAME's psychoacoustic model. The same thing can be said about almost every other encoder...

yeah, but is it really that better? i use it all the time, but, is it really worth it?

You have to find out yourself, if the difference is worth the effort to you, but the increase in quality is really undisputable. Even 160-abr beats 128 hands down.
Go to the top of the page
+Quote Post

8 Pages V  < 1 2 3 4 > » 
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 26th December 2014 - 04:28