Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: MP3 Listening Test at 128 kbps (Read 207088 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

MP3 Listening Test at 128 kbps

Reply #25 –
Quote from: ezra2323 on
dBPoweramp uses the Fhg IIS 4.0.3 encoder as well. Is this the surround sound encoder? Does Nero use 4.0.3 as well? What versions of Nero offer this encoder? WMP, as someone showed, uses Fhg 3.x.

What do you mean by IIS 4.0.3 ? Is it available somewhere?

The current command line encoder reports this version info:

Code: [Select]
********************************************************************
*                                                                  *
*       Fraunhofer IIS MP3 Surround Commandline Encoder V1.4       *
*                                                                  *
*                                                                  *
*           Encoder-Library V04.01.00 (build 2007-05-18)           *
*                                                                  *
*                                                                  *
*                 (c) 1996 - 2007 Fraunhofer IIS                   *
*         (c) 2004 Fraunhofer IIS and Agere Systems Inc.           *
*                     All Rights Reserved.                         *
*  This software and/or program is protected by copyright law and  *
*   international treaties. Any reproduction or distribution of    *
*  this software and/or program, or any portion of it, may result  *
*  in severe civil and criminal penalties, and will be prosecuted  *
*           to the maximum extent possible under law.              *
*                                                                  *
********************************************************************


EDIT

I found that Illustrate advertises the encoder as 4.0.3:
http://www.dbpoweramp.com/codec-central-mp3-fraunhofer.htm

It looks like Fraunhofer does not provide their latest codec version here:
http://www.all4mp3.com/tools/sw_fhg_cl.html

MP3 Listening Test at 128 kbps

Reply #26 –
In general, I dont think we have any reliable information about the FhG v.3.4.0 vs. v.4.0.x quality differences. Has Fraunhofer actually done any development work with the standard stereo mode since the IIS 3.4 version?

I guess we can't expect Fraunhofer to answer.


Perhaps Spoon could say something. He must have some insider information.

MP3 Listening Test at 128 kbps

Reply #27 –
I don't know the exact version numbering, but we have something that starts with 4 and ends with 10  and VBR was only supported since the version that starts with 4 and ends with 4... IIRC

MP3 Listening Test at 128 kbps

Reply #28 –
I just noticed that the command line encoder is 4.01.00 not 4.0.1. So perhaps FhG means 4.1.0. 

I'd suggest that we disqualify Fraunhofer because of misleading version numbering. 

MP3 Listening Test at 128 kbps

Reply #29 –
Quote from: Alex B on
Here's my two cents' worth..

I pretty much agree here. I'd like to see FhG (no particular version preference), LAME (3.98 preferable over 3.97) and iTunes. Shine seems like it would do well as the low anchor (though perhaps at a lower bitrate than 128kbps -- I'm not familiar enough with Shine to have an idea how it sounds at 128kbps), and FhG at 192kbps sounds like a fitting high anchor. I think it might be more fitting to use both VBR and CBR formats in this test, when appropriate. Naturally, opinion may vary on this to a high degree, but I agree with others that the hardware issues are being eliminated at a fairly brisk pace, and each encoder should use whatever method it's stronger with and/or whatever method is more widely used.

MP3 Listening Test at 128 kbps

Reply #30 –
Quote from: Ron Jones on
... Shine seems like it would do well as the low anchor (though perhaps at a lower bitrate than 128kbps -- I'm not familiar enough with Shine to have an idea how it sounds at 128kbps) ...


This is what the creator of the Shine encoder says:

Quote from: Gabriel on
*Low anchor:
AAC-LC @96 is not a suitable low anchor for such a test, IMHO. (anchor should not be used as a contender)
Regarding Shine, well, now that everyone seen how lame is Shine I think that it's perhaps a too low anchor. I would rather suggest either Blade or L3enc 2.72...

According to Gabriel Shine is intended to be the most basic MP3 encoder that can produce standard conforming MP3 files. It is just an educational tool.

I must check how it was ranked in the 128 kbps multi-format test. I don't recall it being too bad, but perhaps I remember wrong.

In any case, I am afraid that even the oldest release version of FhG is not suitable at 128 kbps. It wasn't that bad. I think it already included advanced things like bitreservoir, joint/intensity stereo and short/long block switching. It kind of created the myth of CD quality MP3 at 128 kbps.

Perhaps Blade would be a better low anchor, but even Blade may be surprisingly good with tonal samples that don't have sharp attacks.

MP3 Listening Test at 128 kbps

Reply #31 –
In that case, I'll try to do some evaluations with both Shine and Blade tonight versus LAME before forming a real opinion.

Given my understanding of the purpose of the low anchor, so long as it's is easily identifiable, it doesn't truly matter how poor it sounds. According to Sebastian, way back in 2006, in fact, "..a low anchor is supposed to sound really bad, it should be the worst", so Shine would seem suitable (i.e., "really bad") for that if quality at 128kbps is as poor as you say.

In any case, I'll give 'em both a listen and see how I feel after doing so.

MP3 Listening Test at 128 kbps

Reply #32 –
Quote from: Ron Jones on
In that case, I'll try to do some evaluations with both Shine and Blade tonight versus LAME before forming a real opinion.

Given my understanding of the purpose of the low anchor, so long as it's is easily identifiable, it doesn't truly matter how poor it sounds. According to Sebastian, way back in 2006, in fact, "..a low anchor is supposed to sound really bad, it should be the worst", so Shine would seem suitable (i.e., "really bad") for that if quality at 128kbps is as poor as you say.

In any case, I'll give 'em both a listen and see how I feel after doing so.

Shine was the low anchor in this test:
http://www.listening-tests.info/mf-128-1/results.htm

Instead of wasting your time testing Shine vs. Blade you could try GoGo 3.13 with the following settings and post your findings here.
-b 135 -a -q 2
-b 135 -a
-b 128 -q 2
-b 128

-b 135 -a = an ABR setting for an average bitrate of 133 kbps = about the average bitrate of LAME -V5
-b 128 = CBR 128
-q 2 = a slightly slower and possibly higher quality setting


BTW,

We had a 14-page discussion last year before this 128 kbps MP3 test was postponed:
http://www.hydrogenaudio.org/forums/index....showtopic=47313

During the discussion Sebastian decided to not use a high anchor. Perhaps without a high anchor we could have five contenders (LAME, GoGo, FhG, Helix and iTunes).

MP3 Listening Test at 128 kbps

Reply #34 –
Well, being an anchor, it wouldn't be that much of a nuisance, would it? FhG @192kbps (maybe even higher, like 224/256) seems like a good choice, since it can serve its goal well and probably showcase the drastic leap in quality between it and LAME.

(Then again, I visit HA so rarely nowadays so my opinion doesn't really matter.)
Infrasonic Quartet + Sennheiser HD650 + Microlab Solo 2 mk3. 

MP3 Listening Test at 128 kbps

Reply #35 –
I think an ancient yet reasonably popular historic encoder like Radium or SoloH could serve well as a potential low anchor even at the same bitrates. I am interested in seeing how far along MP3 has come in the past 7-8 years. I see Blade's been suggested, which isn't bad except it was never realistically considered a high quality solution at any time. It was just free and quickly available after ISO's code release. Radium was *the* MP3 codec for some time, for longer than it had any right to be. I suggest that if there's hesitation about its possible competitiveness with respect to being a low anchor, then we aren't convinced whether MP3 has truly made gains since the late 90s. In any case, I think the choice should be something that mattered at some time, not an artificially lousy choice like Shine. I'll take Xing circa 1998 over that.

I agree with eliminating the high anchor. It's pretty clear from recent tests that its value approaches 5 at these bitrates.

Also, I'm for the exclusion of LAME 3.97 if it bumps an otherwise worthy contender. Far more relevant than possible LAME regression is its performance compared to other encoders in general. I doubt minor issues between point releases would make or break comparisons between fundamentally different software. LAME 3.97 vs. 3.98 is important, but I think this test would be too crowded for meaningful comparisons.

MP3 Listening Test at 128 kbps

Reply #36 –
I would agree with foregoing the high anchor.  You could have two low anchors, one worse than the other.

MP3 Listening Test at 128 kbps

Reply #37 –
Quote from: Alex B on
I just noticed that the command line encoder is 4.01.00 not 4.0.1. So perhaps FhG means 4.1.0. 

I'd suggest that we disqualify Fraunhofer because of misleading version numbering. 


Yes, it's 4.1.0 I just checked. This is the version that adds VBR (with VBRI headers) and original file length support.

MP3 Listening Test at 128 kbps

Reply #38 –
Gabriel suggested 3.98 so I am going to include that instead of 3.97. Gabriel, do you recommend -V 5 --vb-new or some other setting for ~128 kbps?
iTunes and Helix are also featured for sure. Should I use both in VBR mode?
Fraunhofer is also an encoder I would like to include - same question now: CBR or VBR?
Is Gogo still used or did people switch to Helix / FhG / iTunes for quick encoding?

ff123 and other pros, what do you think about dropping the high anchor and using the five contenders LAME, FhG, iTunes, Helix and Gogo at 128 kbps?

As for the low anchor, I don't see any point in using Blade. I could imagine using Shine because it's very simple, Radium because it was very popular years ago or l3enc because of its age.

MP3 Listening Test at 128 kbps

Reply #39 –
I just reread the complete last year's thread. The codecs and settings were decided (almost) and we were talking about the sample choices on the last two pages.

We discussed about the need of a high anchor on the page three. The conclusion was to not include one.
The selected low anchor was FhG l3enc v.1, but I was concerned about its possibly too high quality.

Here are a few selected quotes from that thread:

Quote from: Alex B on
I have had an impression that Gogo should be used with ABR at a bitrate like ~130 kbps. It is based on LAME 3.88 and VBR wasn't very matured at that time. Even LAME 3.9x ABR has been used at lower bitrates for example by Guruboolez until the most recent versions. I mentioned earlier that I have used command lines like this: "-a -b 192 -m j -q 2". For this test the command line could be like "-a -b 135 -m j" and the quality option 0, 1 or 2. (I have used -q 2 without any testing just because LAME 3.88 -h was mapped with the -q 2 option and it produced a good balance between the quality and speed).

Quote from: Gabriel on
Quote from: Sebastian Mares on
Gabriel, were you recommending ABR over VBR in the LAME 3.88 days?

Fur such bitrates, definitively ABR. Note: it's likely that abr will undersize its target bitrate.



Quote from: Alex B on
For Helix I would actually recommend using Real Player 10.5. I just tried it (what an awkward experience) and it seems to have only two 128 kbps MP3 options. CBR and VBR. Once again the 128 kbps VBR bitrates seem to be within the test target (~ about 135 kbps).

One would like to assume that Real Networks has tweaked their flagship program to produce best possible MP3 quality. This may not be practically true, but it would be an acceptable and a better reasoning for the used Helix setting than anything else I have seen in this thread so far.

Quote from: Alex B on
... Perhaps it would be good to just use the latest WMP and the options MS has decided to make available. After all they should know what is best for their MP3 encoder.

The test idea would be then like what the big guys offer vs. LAME & Gogo at about 130 kbps. The big guys are naturally MS, Apple and Real. (I don't know how big share Real has anymore, but they were one of the first and Real Player is still very well-known.)



Quote from: Sebastian Mares on
I am testing Gogo because it's pretty fast and I want to see how it competes against LAME and other fast encoders like iTunes, Helix and FhG.

Discussion about encoder decision is really closed now. All additional codec requests or questions regarding why codec A was used over B are going to be ignored. Current list of encoders is:

LAME 3.97: -V 5 --vbr-new

iTunes 7.0.1.8: 128 kbps, VBR, highest quality, automatic sampling rate, automatic channel mode, "Stereo (joint)" stereo mode, intelligent, filter frequencies below 10 Hz

RealPlayer 10.5: 128 kbps, VBR

Gogo 3.13: have to check which ABR settings are suitable. Should I manually specify JS coding? What about the quality - should I even use that switch?

FhG: Windows Media Player 11, 128 kbps CBR

Low Anchor: l3enc 1.0, -br 128000. Should I foce JS coding here, too?



I don't think much has changed since then. Perhaps we could reconsider the FhG version, but for me it would still make sense to include the WMP11 version, which is the version that most Windows users already have. If FhG 4.1.0 really is better in stereo mode it could be considered. It may also be regressed. I really would like to see some personal test results with ABX logs. It would also be useful to compare the v. 3.4 and 4.1.0 encoding speeds before making the choice. After all we were supposed to compare the faster options against LAME.

MP3 Listening Test at 128 kbps

Reply #40 –
Also, wondering if Gabriel changed his mind regarding VBR vs. ABR in 3.98. Using RealPlayer for creating the Helix MP3s is reasonable - saves us from testing the various options given by the CLI.

Does anyone know how quick Microsoft usually is with updating the MP3 libraries?

MP3 Listening Test at 128 kbps

Reply #41 –
Quote from: Sebastian Mares on
Fraunhofer is also an encoder I would like to include - same question now: CBR or VBR?


There seems no mode to produce something around 128kbps using VBR, only much lower (mode 5) and much higher (mode 4).

MP3 Listening Test at 128 kbps

Reply #42 –
Quote from: Sebastian Mares on
Also, wondering if Gabriel changed his mind regarding VBR vs. ABR in 3.98.

Why would Gabriel had changed his mind between the versions 3.97 and 3.98 ? Isn't  -V5 (--vbr-new) de facto standard that is known to be able to compete with the newer encoders. (If you mean the quoted reply, it was for 3.88.)

I just updated my WMP11, iTunes and Real Player to the latest versions.

WMP11 had no recent MP3 codec updates. It uses the version 3.4 as I said earlier.

The current iTunes version is 7.4.0.28 and the installer is 13 MB bigger than the 7.2 installer that I had previously. I have no idea if its MP3 encoding has changed anyhow.

Real Player is still the version 10.5, but the so called product version number has advanced from 6.0.12.1483 to 6.0.12.1662 since last year.

I am going to run my 25-track bitrate test set with these versions and also check the encoding speeds.

I really like to see GoGo included because it has not been tested against Helix & FhG. For GoGo I would suggest to use -a -b 135 and a q-option that would produce at least about three times faster speed when compared with LAME. The default without a q-switch could be possible. On my PC the default setting produces about 60x speed and -q 2 about 40x, but I have not tested LAME yet. (Joint stereo seems to be enabled automatically at this ABR bitrate so a stereo mode switch is not needed)

EDIT

I'll include the FhG IIS v. 4.1.0 command line encoder in my bitrate & speed test. Perhaps I am able to do some quick quality comparisons too (using the encoded files). I'll post the results during the coming weekend.

MP3 Listening Test at 128 kbps

Reply #43 –
Quote from: Sebastian Mares on
ff123 and other pros, what do you think about dropping the high anchor and using the five contenders LAME, FhG, iTunes, Helix and Gogo at 128 kbps?


Previous listening tests at 128 kbps clearely teached us that transparency is almost reached for several people. Including an high anchor is very important but in this situation it may be pointless. I would go for the one of the following configurations:
• 3 competitors + 2 anchors (low & high)
• 4 competitors + 1 anchor (low)
=> 5 encodings to focuse on (and for most people, 4 only - assuming that low anchor is immediatelly detected)


VBR vs ABR/CBR:

I wouldn't drop VBR. From my bitrate table (based on classical music only - thus with limited validity) we should get three competitors with comparable bitrate : LAME -V5 (~130 kbps); Fraunhofer 4.xx CLI (~130 kbps) and of course Helix which is by far the most flexible tool. I don't know anything about iTunes & gogo.

The remaining question is: is VBR the best choice for HELIX and Fraunhofer (and other encoders). There's only one way to get a valid answer: pre-listening tests. I seriously doubt that people would spend their free time to get a solid experience with these rather unused (here on HA) encoders. I tried once and quickly gave up: I don't see the point of spending hours and hours to analyse encoders that are immediately unusable (for my taste).

Using CBR/ABR for each competitors is another possibility but such configuration won't learn anything to the community (assuming that most HA members are following the recommendations/experience of other members and are using LAME -V5 to get small and HQ MP3 encodings). So I would definitely discard this option.


If I had to make this test alone I would go for:
- VBR only encoders
- it implies to discard all encoders that doesn't allow ~130 kbps VBR encodings and keep the other ones
- unless switches are specifically labelled as 'improve quality' I wouldn't tweak the commandline (there are some dubious ones for HELIX) unless some people could provide valid proof of their positive effects on most situations.

MP3 Listening Test at 128 kbps

Reply #44 –
I don't see any reason why every encoder should be used in VBR mode or to exclude competitors with poor VBR quality.
IMO we should use any encoder with settings that are expected to be best from experience though certainly there is no scientific evaluation.
As for additional switches I agree that only when there's strong experience against it we should use other than the default settings.

From Gabriel we know already that gogo should be used with ABR.

From my experience with FhG I can say that VBR mode isn't very robust. So CBR should be used IMO.

With Lame Gabriel and robert can say what setting to use but certainly this will be -V5.

Helix should be used in VBR mode IMO.

I have no idea for the other encoders.
lame3995o -Q1.7 --lowpass 17

MP3 Listening Test at 128 kbps

Reply #45 –
Quote from: guruboolez on
If I had to make this test alone I would go for:
- VBR only encoders
- it implies to discard all encoders that doesn't allow ~130 kbps VBR encodings and keep the other ones
- unless switches are specifically labelled as 'improve quality' I wouldn't tweak the commandline (there are some dubious ones for HELIX) unless some people could provide valid proof of their positive effects on most situations.

As I suggested, there would be no problem If the test idea was MS, Apple & Real vs. LAME & GoGo.

VBR can and should be used in the iTunes and Real Player options as the developers provide the option and the average bitrate seems to be suitable.

WMP11 has no VBR option so MS has made the choice. If the newer FhG CL encoder (or the same encoder through Nero's UI) is selected instead of WMP11 the CBR option may be the only possibility. It has been said that there is no suitable VBR quality setting. In this case Fraunhofer has made the choice.

GoGo is an exception because it is not maintained anymore and we don't have any developers' recommendations. We should use a setting that a sensible and experienced user would select when LAME -V5 (--vbr-new) is too slow for getting the files ready before they are needed. If GoGo is left out of the test we will never know if e.g. Helix would have been better or worse than GoGo. If GoGo is found to be worse than the others it would work as a good "second low anchor". It may also positively surprise with some of the samples.

MP3 Listening Test at 128 kbps

Reply #46 –
I don't see any reason why every encoder should be used in VBR mode or to exclude competitors with poor VBR quality.
To avoid criticism and suspicion... it already happened in the past: if any encoder, set with ABR/CBR, doesn't come on first position, some people (call them zealot or not) are claiming that the whole test is biased. Here on HA and also on other website. It partially ruin the credibility of the whole test (and not necessary for a bad reason).

IMO we should use any encoder with settings that are expected to be best from experience though certainly there is no scientific evaluation.
The lack of "scientific" evaluation lead to the whole internet carnival of opinions. HA.org was born to fight against this whole mess. Listening tests share the same goal. I can't imagine any listening test based on fragile preconceptions. That's precisely why I'm so worried about this idea of starting a MP3 listening test. Unlike multiformat listening tests, we don't know anything solid about most competitors.

As for additional switches I agree that only when there's strong experience against it we should use other than the default settings.
I also said that we shouldn't use the default setting if some specifically "HQ" switches are existing but aren't used by default. It's the case for Fraunhofer 4.xx IIRC.

From Gabriel we know already that gogo should be used with ABR.
As far as I can read he said that LAME 3.88 was likely to be better with ABR. I can't understand the source code of gogo and I can't say if gogo is nothing more than LAME 3.88 with speed optimisation. In this case we would have a valid opinion about gogo. But if GOGO is nothing more than a faster LAME 3.88 what's the point of testing such antiquity?


From my experience with FhG I can say that VBR mode isn't very robust. So CBR should be used IMO.
Perfect. But are there any chance to see detailed results?

MP3 Listening Test at 128 kbps

Reply #47 –
IMHO, this whole test has no point unless the tested LAME rivals are sigificantly faster than LAME. That's why I may change my opinion about using a high quality -q switch with GoGo if it reduces the encoding speed too much.

My very modest proof is this:
Quote from: Alex B on
As a quick rehearsal I tried this sample

http://www.hydrogenaudio.org/forums/index....showtopic=47370

using the following encoders & settings:

LAME 3.97b2, -V5 --vbr-new
WMP10 / FhG, 128 kbps CBR default settings (I used the acmenc command line interface)
Helix (hmp3enc.exe 23.7.2005), -X2 -U2 -V60 -HF2 -F17000
iTunes 6.0.5.20, 128 kbps, VBR, Quality: Highest, Joint Stereo, Smart enabled, Filter below 10 Hz enabled
iTunes 6.0.5.20, 128 kbps, CBR, Joint Stereo, Smart enabled, Filter below 10 Hz enabled
Gogo 3.13, -a -b 133 -m j -q 2

The first few distorded rock guitar chords sent all other encoders except LAME straight to the Hell.

I ABXed them all 8/8, but LAME was difficult and the others were quick and easy.

In general, LAME was quite good, not annoying at all, FhG was barely usable, the others were bad.

I don't know if the encoder versions and settings I used were optimal, but if the actual test samples behave even partially like this I think we may have a good chance to find a clear winner this time.

Personally, after this rehearsal, I would use only LAME at this bitrate. Even the other samples would be transparent with all encoders a standard rock quitar sound like this is too important for me.

I reuploaded the "AC_DC___Highway_to_Hell.wv" sample:
http://www.hydrogenaudio.org/forums/index....st&p=514963

MP3 Listening Test at 128 kbps

Reply #48 –
Quote from: menno on
Quote from: Sebastian Mares on

Fraunhofer is also an encoder I would like to include - same question now: CBR or VBR?


There seems no mode to produce something around 128kbps using VBR, only much lower (mode 5) and much higher (mode 4).


Correction, mode 4 seems to produce 132 kbps on a big set of files, so it seems usable.

MP3 Listening Test at 128 kbps

Reply #49 –
Quote from: guruboolez on
it already happened in the past: if any encoder, set with ABR/CBR, doesn't come on first position, some people (call them zealot or not) are claiming that the whole test is biased. Here on HA and also on other website. It partially ruin the credibility of the whole test (and not necessary for a bad reason).

So should we do anything to avoid people to react strangely? IMO this is not a good target and we won't succeed anyway.

We should do the best according to what we know - well conscious about the imperfections.
You're absolutely right that we don't know a lot about the competitors, and everything is disputable, but that's no reason to me to worry about such a test.
To me any listening test has it's restricted meaning - but still is valuable and adds to experience.
lame3995o -Q1.7 --lowpass 17