IPB

Welcome Guest ( Log In | Register )

8 Pages V  « < 4 5 6 7 8 >  
Reply to this topicStart new topic
MP3 at 128kbps public listening test, Discussion thread
rjamorim
post Dec 22 2003, 16:55
Post #126


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



Hello.

Some last points before I go on a trip (I'll only visit the boards seldomly until Jan 10th).

About VBR vs. CBR:

There are good reasons for either method. They are:

CBR is what newbies use once they start playing with MP3. Only later some of them come to know about VBR and ABR. Also, it's the default mode in most MP3 encoders: Lame, FhG, Xing... (or the only mode in some cases: Radium, Audioactive, iTunes)

Also, CBR is best to avoid criticism on test fairness, since there is no bitrate deviation.

VBR/ABR is what should be used to take the most out of each codec at the chosen bitrate. Lame is known for performing quite better at ABR than CBR, and I believe the same applies to FhG. In this case, Lame, FhG and Xing would use VBR encoding, and Audioactive/Radium and iTunes would use CBR.

No-CBR for Lame only is definitely out of the question. The test would look scandalously biased if Lame was tested at ABR and everything else at CBR.

IMO, the bottom point is if we want the test results to appeal to newbies or enlightened HA users. If we go for newbies, the test should be CBR. If we go for veterans, the test should be ABR.

Any thoughts?


About Lame version:

I think it's closely related to the newbies vs. veterans issue. If the test is done for veterans, Lame 3.90.3 should be featured. If it is for newbies, Lame 3.93.1, since this is the version you'll find in CDex, Winamp, and at Mitiok's site. I think that actually the only popular Lame binary distribution point featuring 3.90.3 is RareWares.


About Audioactive vs. Radium

It is said that both use similar encoding routines, so I would definitely test only one of them. The appeal of Radium is that it is very widely used in DivX rips. The appeal of Audioactive is that it seems to be based on more recent - and therefore more tweaked - libraries, so quality is probably better.


About FhG version:

I'm inclined to test FhG Current and Legacy Fast from Adobe Audition, both at some VBR mode that comes close to 128kbps (in case we decide to go for VBR). Legacy Slow is - as the name implies - too slow and, from what it is said, fast is actually better (!).

If only one FhG codec is featured (that would be Audition Current), I might as well replace the other with Gogo 3.x


So, the codec list would be as follows:

Lame 3.90.3 or 3.93.1
FhG Current in Audition
FhG Legacy Fast or Gogo 3.x
iTunes 4.2
Audioactive 2.04j or Radium
RealOne MP3 encoder (Xing)


And the settings would depend on the choice for VBR or CBR.


Please post your comments and suggestions.

Best regards;

Roberto.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
Latexxx
post Dec 22 2003, 17:45
Post #127


A/V Moderator


Group: Members
Posts: 858
Joined: 12-May 03
From: Finland
Member No.: 6557



I'd like to see lame -br 128 as an anchor because all lamers just set their cdex to rip at that bitrate without touching the other setteings. The quality difference (???) between the br-switch and preset might even force some of those nasty pirate gruops to switch to presets. (But on the other hand lame 3.94 will force them to switch to presets but they'll still use stereo instead of joint stereo because they are stupid tongue.gif )
Go to the top of the page
+Quote Post
rjamorim
post Dec 22 2003, 17:59
Post #128


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



Good point. But the low anchor is meant to sound bad, and I'm afraid Lame CBR could sound better than iTunes and Radium/Audioactive...

That's why I plan to use Xing as anchor.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
Gabriel
post Dec 22 2003, 18:33
Post #129


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



Please, leave Xing as anchor.

Regarding Lame, it seems to me that your choice is between 3 versions: 3.90.3, 3.93.1, 3.94
Go to the top of the page
+Quote Post
rjamorim
post Dec 22 2003, 19:01
Post #130


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (Gabriel @ Dec 22 2003, 03:33 PM)
Please, leave Xing as anchor.

I will, most probably.

QUOTE
Regarding Lame, it seems to me that your choice is between 3 versions: 3.90.3, 3.93.1, 3.94


But 3.94 is still beta. It's not a good idea to use beta stuff in listening tests, IMO, because people can easily claim the codec performed badly (if that's the case) because if some beta flaw. I personally would prefer to use a release version, either 3.90.3 or 3.93.1

Unless you plan to make Lame 3.94 stable until ~Jan 11th wink.gif


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
schnofler
post Dec 22 2003, 20:48
Post #131


Java ABC/HR developer


Group: Developer
Posts: 175
Joined: 17-September 03
Member No.: 8879



I'm still in favor of an ABR/VBR test.
QUOTE (rjamorim)
IMO, the bottom point is if we want the test results to appeal to newbies or enlightened HA users. If we go for newbies, the test should be CBR. If we go for veterans, the test should be ABR.

I don't agree with you on this point. I think if you're a newbie and you're reading listening test results, that's probably because you want to find out what to use to make your encodings sound as good as possible (with MP3 at that specific bitrate). And if you want your encodings to sound good, you should use ABR/VBR. So I don't think CBR would be interesting for newbies. There is one reason I can think of for using CBR, though. The results of such a test might be useful for people who want to find out the quality of MP3s they didn't encode themselves (I probably don't have to mention where those MP3s come from, usually).

As for the question what Lame version to use, I'd go with 3.90.3, although I don't think it makes much of a difference, because both versions will likely be superceded by 3.94 in the near future.
Go to the top of the page
+Quote Post
ff123
post Dec 22 2003, 22:41
Post #132


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



QUOTE (rjamorim @ Dec 22 2003, 07:55 AM)
About FhG version:

I'm inclined to test FhG Current and Legacy Fast from Adobe Audition, both at some VBR mode that comes close to 128kbps (in case we decide to go for VBR). Legacy Slow is - as the name implies - too slow and, from what it is said, fast is actually better (!).

If both Legacy Fast and Legacy Slow actually encode with VBR, then they are probably the same codec -- the FhG fast codec, because that's the only one which does VBR.

In Cool Edit Pro 2.1, they had actually removed the FhG slow codec from the library, and made quality settings based on only the fast codec.

If that is really the case, then I would choose legacy slow for the FhG codec. Remember that you can try blackbird.wav to find out if it has glitches in it or not. That would indicate whether Legacy Slow really is the FhG slow codec or not.

ff123
Go to the top of the page
+Quote Post
Jojo
post Dec 22 2003, 23:28
Post #133





Group: Members
Posts: 1361
Joined: 25-November 02
Member No.: 3873



QUOTE (Gabriel @ Dec 22 2003, 09:33 AM)
Regarding Lame, it seems to me that your choice is between 3 versions: 3.90.3, 3.93.1, 3.94

hmm, what about LAME 3.92 :confused:


--------------------
--alt-presets are there for a reason! These other switches DO NOT work better than it, trust me on this.
LAME + Joint Stereo doesn't destroy 'Stereo'
Go to the top of the page
+Quote Post
bidz
post Dec 22 2003, 23:37
Post #134





Group: Members
Posts: 351
Joined: 27-December 02
From: Norway
Member No.: 4258



QUOTE (rjamorim @ Dec 22 2003, 07:55 AM)
CBR is what newbies use once they start playing with MP3. Only later some of them come to know about VBR and ABR. Also, it's the default mode in most MP3 encoders: Lame, FhG, Xing... (or the only mode in some cases: Radium, Audioactive, iTunes)

FYI, iTunes supports MP3 VBR encoding. Press "Custom" and it'll pop-up a configuration dialog, and you can check "Use Variable Bitrate Encoding (VBR)"

When in VBR mode, the bitrate you choose is the lowest bitrate the encoder will use. And you can select between these presets:

Lowest
Low
Medium Low
Medium
Medium High
High
Highest

and also select stereo/joint stereo, sample rate and lowpass filter.

Also EncSpot detects the iTunes files as: FhG (fastenc or mp3enc)

This post has been edited by bidz: Dec 22 2003, 23:44


--------------------
myspace.com/borgei - last.fm/user/borgei
Go to the top of the page
+Quote Post
Steve999
post Dec 23 2003, 00:11
Post #135





Group: Members
Posts: 88
Joined: 16-December 03
Member No.: 10473



As an actual real-life newbie, I'd most want to see comparisons of VBR recordings that average or approximate 128 kbps. That's what I could best use to make practical decisions. cool.gif

This post has been edited by Steve999: Dec 23 2003, 00:12
Go to the top of the page
+Quote Post
rjamorim
post Dec 23 2003, 00:40
Post #136


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (schnofler @ Dec 22 2003, 05:48 PM)
I don't agree with you on this point. I think if you're a newbie and you're reading listening test results, that's probably because you want to find out what to use to make your encodings sound as good as possible (with MP3 at that specific bitrate).

Very good point. Maybe I'll go with ABR/VBR after all.

QUOTE
If both Legacy Fast and Legacy Slow actually encode with VBR, then they are probably the same codec -- the FhG fast codec, because that's the only one which does VBR.


oh, I see. I thought Legacy Slow was mp3enc. Thanks for this information. I'll test blackbird.wav when I return home, and if it isn't the FhG Slow codec, I'll replace Legacy Fast with it.

QUOTE
hmm, what about LAME 3.92 :confused:


I chose Lame 3.90.3 because it's the HA branch, and 3.93.1 because it's the latest stable release. Since 3.93.1 came after 3.92, I see no point in using a deprecated version.

QUOTE
FYI, iTunes supports MP3 VBR encoding. Press "Custom" and it'll pop-up a configuration dialog, and you can check "Use Variable Bitrate Encoding (VBR)"


Ah, that's useful information. Thank-you for mentioning it.

QUOTE
Also EncSpot detects the iTunes files as: FhG (fastenc or mp3enc)


EncSpot is wrong. It has already been proven earlier in this thread (look for a link to the lame mailing list) that iTunes is based on SoundJam, that on it's turn is an independent tweak line from dist10.

Thank-you for your replies so far.

Regards;

Roberto.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
rjamorim
post Dec 31 2003, 18:51
Post #137


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



OK, I'm getting close to freezing the encoders and settings.

My ideas:

-FhG Audition Current VBR
-FhG Audition Legacy Fast or Slow VBR (more on that later)
-Lame 3.90.3 --alt-preset 128
-iTunes VBR
-Audioactive 2.04 high quality 128kbps CBR
-RealOne (Xing) VBR

I'll test the VBR settings of Audition, Xing and iTunes to find out what comes closer to 128kbps overall.

About FhG Legacy Fast vs. Legacy Slow: I'll use slow if it's not the MP3enc branch. But I need someone to test it using Blackbird.wav (like ff123 suggested). I can't test now because I'm on a trip.
More info on identifying MP3enc: http://ff123.net/identify.html

About Lame version: I plan to test Lame 3.90.3, unless 3.94 is released until the test date (since 3.94 seems to feature major improvements compared to 3.90.X)

If everything goes as planned, the test will start on January 14th.

Please comment.

Best regards;

Roberto.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
ff123
post Dec 31 2003, 20:58
Post #138


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



Ok,

I acquired Adobe Audition 1.0 so I could take a look at the various mp3 codecs in it.

1.
Preset: VBR - Good Quality Stereo
VBR, MP3
70 - (125-175 Kbps), High Quality
Max Bandwidth 14112 Hz
VBR Quality 70
Changed Codec to Legacy - High Quality (Slow)
Allow Mid-Side Joint Stereo is unchecked
Allow Intensity Joint Stereo is unchecked
Allow Narrowing of Stereo Image is unchecked

Result: not the super slow codec, as found within fastencc.exe 1.02 using the -hq switch.

2.
Preset: 128 Kbps Stereo (Internet)
CBR, MP3
128 Kbps, 44100 Hz, Stereo (11.0:1)
Max Bandwidth 22050
CBR Bitrate 128 Kbps
Sample Rate 44100 Hz
Changed Codec to Legacy - High Quality (Slow)
Allow Mid-Side Joint Stereo is checked
Allow Intensity Joint Stereo is checked
Allow Narrowing of Stereo Image is unchecked

Result: not the super slow codec

3.
Preset: 128 Kbps Stereo (Internet)
CBR, MP3
128 Kbps, 44100 Hz, Stereo (11.0:1)
Max Bandwidth 22050
CBR Bitrate 128 Kbps
Sample Rate 44100 Hz
Codec: Current - Best Quality
Allow Mid-Side Joint Stereo is checked
Allow Intensity Joint Stereo is checked
Allow Narrowing of Stereo Image is unchecked

Result: not the super slow codec

4.
Preset: VBR - High Quality Stereo
VBR, MP3
Changed setting to 70 - (125-175 Kbps), High Quality
Max Bandwidth 17660 Hz
VBR Quality 70
Changed Codec to Legacy - High Quality (Slow)
Allow Mid-Side Joint Stereo is unchecked
Allow Intensity Joint Stereo is unchecked
Allow Narrowing of Stereo Image is unchecked

Result: not the super slow codec

So, as I suspected, the super slow code no longer exists in the FhG library. I would choose either "128 Kpbs Stereo (Internet)" or "VBR - High Quality Stereo" as the base presets.

Using "VBR - Good Quality Stereo" will set the Max Bandwidth too low, in my opinion. If using High Quality VBR, the default VBR setting is 100 and the default codec is Legacy - Medium Quality (Fast). You'll probably have to at least change the VBR setting. Probably either vbr 50 or 60 is closest to 128 kbps.

On main_theme.wav, at least, vbr 50 is an improvement over cbr 128. It didn't matter whether I used legacy fast or current for the vbr encode. Either one sounded better than cbr 128 (current).

So my suggestion would be to use the "VBR - High Quality Stereo" preset, and change the VBR setting to either 50 or 60, and change the codec to "Current - Best Quality." I think that all the codecs within Audition are variations on the Basic FastEnc, so you should just choose one.

ff123

Edit: Possibly the reason why VBR sounds better on main_theme.wav is because joint-stereo is disallowed (although I didn't verify with EncSpot that this is true for VBR 50).

This post has been edited by ff123: Dec 31 2003, 21:04
Go to the top of the page
+Quote Post
rjamorim
post Jan 2 2004, 17:14
Post #139


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



Thanks for the help, ff123. As usual, I owe a lot to you smile.gif


So, the updated codec list:

-FhG Audition Current VBR 50-60
-Lame 3.90.3 --alt-preset 128
-iTunes VBR
-Audioactive 2.04 high quality 128kbps CBR
-RealOne (Xing) VBR - anchor


Now, the usual request for opinions:

Do you guys prefer that I leave the codec list at that (only 5 encoders featured) or that I replace Audition Legacy with Gogo, or some other encoder?

Also, since this is a mid-bitrate test, it might be a good idea to replace one or two of the samples with problem cases. Do you agree?

Thanks a lot.

Best regards;

Roberto.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
guruboolez
post Jan 2 2004, 18:31
Post #140





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



I like the idea of a killer-sample. This will show to everybody a good exemple of failure of mp3 at this bitrate. Why not the good old friend of mp3: fatboy?

This post has been edited by guruboolez: Jan 2 2004, 18:31
Go to the top of the page
+Quote Post
DigitalDictator
post Jan 2 2004, 20:19
Post #141





Group: Members
Posts: 313
Joined: 9-August 02
From: SoFo
Member No.: 3002



I'd like to see how much quality was sacrificed for speed with GOGO. IMO it's always been left out in tests, not among the best encoders but not really bad either. Kind of in the middle, almost boring:) The quality is basically (again IMO) estimated (i.e. it SHOULD perform this... and IS LIKELY to sound like that...)
Go to the top of the page
+Quote Post
Continuum
post Jan 2 2004, 21:26
Post #142





Group: Members
Posts: 473
Joined: 7-June 02
Member No.: 2244



QUOTE (rjamorim @ Jan 2 2004, 05:14 PM)
-FhG Audition Current VBR 50-60
-Lame 3.90.3 --alt-preset 128
-iTunes VBR
-Audioactive 2.04 high quality 128kbps CBR
-RealOne (Xing) VBR - anchor
I'd love to have a Lame ABR-CBR comparison, this way we would know, how large the difference (which I think is undisputed) is compared to codec differences. And there are still scenarios where the CBR compatibility is needed.

QUOTE
Also, since this is a mid-bitrate test, it might be a good idea to replace one or two of the samples with problem cases. Do you agree?
I'm not sure. Could a few problem cases skew the total ratings? Problem sample rating differences will probably be larger (as more of the scale is used), and hence have more influence on the overall scores. I am not sure if this is a desired effect.
Go to the top of the page
+Quote Post
guruboolez
post Jan 2 2004, 21:42
Post #143





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



QUOTE (Continuum @ Jan 2 2004, 09:26 PM)
Could a few problem cases skew the total ratings? Problem sample rating differences will probably be larger (as more of the scale is used), and hence have more influence on the overall scores. I am not sure if this is a desired effect.

As long as the test is comparing various encoders of one single format, I think that there are no problems to incude a killer-sample. Fatboy for exemple is destroyed by each encoders at this bitrate. It's not a lame-killer only, or a typical fhg-problem sample.

Of course, bad notations will affect the overall scores at the end of the test. On the other side, bad surprises may happen in real life with mp3 at this bitrate; so, I think that introducing a killer in the arena isn't a bad thing.

This post has been edited by guruboolez: Jan 2 2004, 21:43
Go to the top of the page
+Quote Post
LoFiYo
post Jan 2 2004, 23:36
Post #144





Group: Members
Posts: 133
Joined: 2-January 04
Member No.: 10896



QUOTE (DigitalDictator @ Jan 2 2004, 11:19 AM)
I'd like to see how much quality was sacrificed for speed with GOGO. IMO it's always been left out in tests, not among the best encoders but not really bad either. Kind of in the middle, almost boring:) The quality is basically (again IMO) estimated (i.e. it SHOULD perform this... and IS LIKELY to sound like that...)

Why don't we test GOGO 3.12 once and for all, and see if it's really not that good in quality? If it's no good, then we can stop talking about it (until an improved version is released).
Go to the top of the page
+Quote Post
guruboolez
post Jan 2 2004, 23:55
Post #145





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



QUOTE (LoFiYo @ Jan 2 2004, 11:36 PM)
Why don't we test GOGO 3.12 once and for all, and see if it's really not that good in quality? If it's no good, then we can stop talking about it (until an improved version is released).

It's not a bad idea, but if we include gogo, there will be 6 encoders to rate. Five encoders are enough for a lot of people, maybe too much!

Nevertheless, if Roberto launch a mp3 general test, why not imagine a complete one. After all, will there be enough progress with MP3 to expect any similar test in the next years? I'm not sure.
With 6 challengers, we'll have :

- the good : lame
- the bad : Audioactive Pro
- the ugly : Xing
- the fast : gogo
- the alternative : iTunes (MAC users)
- the expensive : Audition

Good script, no? As sample, why not Morricone?
Go to the top of the page
+Quote Post
bond
post Jan 2 2004, 23:59
Post #146





Group: Members
Posts: 881
Joined: 11-October 02
Member No.: 3523



and mp4pro at 64kbps biggrin.gif


--------------------
I know, that I know nothing (Socrates)
Go to the top of the page
+Quote Post
Continuum
post Jan 3 2004, 09:16
Post #147





Group: Members
Posts: 473
Joined: 7-June 02
Member No.: 2244



QUOTE (bond @ Jan 2 2004, 11:59 PM)
and mp4pro at 64kbps biggrin.gif

Hmm. Doesn't the 64kbit Test tell you everything you want to know about it? (HE-AAC 64 against Lame 128)
Go to the top of the page
+Quote Post
tycho
post Jan 3 2004, 11:27
Post #148





Group: Members
Posts: 345
Joined: 5-August 03
Member No.: 8183



Just wondering what settings you planned for iTunes VBR. Obviously you should set Quality to "Highest" and leave "Joint Stereo". But the bitrate settings is for the lower bound. I.e. 128 will give an avg. about 140 or so. and 112, produces an avg. around 120 (strangely this is independent of Quality settings). So which one to choose?

guruboolez: iTunes is not for MAC users only - its for PCs, too.
Go to the top of the page
+Quote Post
AstralStorm
post Jan 3 2004, 11:48
Post #149





Group: Members
Posts: 745
Joined: 22-April 03
From: /dev/null
Member No.: 6130



It'd be fun to test Lame 3.90.3/3.93.1 vs 3.94b, but I think this isn't the scope of this test...

(mabe with just plain -b 128 to test the effectiveness of automatic presets)

This post has been edited by AstralStorm: Jan 3 2004, 11:52


--------------------
ruxvilti'a
Go to the top of the page
+Quote Post
danchr
post Jan 3 2004, 13:44
Post #150





Group: Members
Posts: 487
Joined: 6-April 03
From: Århus, Denmark
Member No.: 5861



QUOTE (tycho @ Jan 3 2004, 11:27 AM)
guruboolez: iTunes is not for MAC users only - its for PCs, too.

<nitpick mode>
It's Mac, not MAC. MAC is something your ethernet card has, while Mac is short for Macintosh.
</nitpick mode>
Go to the top of the page
+Quote Post

8 Pages V  « < 4 5 6 7 8 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 1st August 2014 - 07:13