Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Multiformat 48 kbps Listening Test (Read 83735 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Multiformat 48 kbps Listening Test

Although we had a pre-test discussion already, I wanted a fresh start. This doesn't mean it should take ages before a decision is made like it was the case before.

So - now that the new WMA codec is out and ready for testing, I decided to delay the MP3 listening test and organize this long-awaited multiformat test. The candidates so far are:

Nero HE-AAC
AoTuV Vorbis Beta 5
Windows Media Audio 10

Possible anchors are:

LC-AAC at 48 kbps as low anchor
LC-AAC at 128 kbps or 96 kbps as high anchor

Ivan / Nero Crew and Aoyumi, can you recommend settings for your encoders?

What other contenders would you like to see (preferably other formats)?

In the meanwhile, I am going to re-read the old thread and see if I can find anything useful there.

Multiformat 48 kbps Listening Test

Reply #1 –
I would restrict the test to :
• the most serious competitors for low bitrate (it excludes MP3, MPC, AAC-LC... one of them could nonethless be introduced as anchor)
• living formats (no MP3Pro even if this format should still be competitive)
• one implementation per format

The purpose of those restrictions is to drastically limit the number of competitors. As a consequence I only see the following formats:
• AAC (High Efficiency profile)
• Vorbis
• WMA Standard
• WMA Professional
(+ AAC-LC or MP3 as anchor and also to please people interested by these hardware-friendly formats).

The only true problem so far is the choice of HE-AAC implementation: Nero Digital or Coding Technologies?


The choice of settings is not easy. Vorbis should compete as VBR (the CBR mode of Vorbis isn't recommended and is painfully slow; I believe that nobody uses it). HE-AAC VBR is possible, thanks to Nero. And if we go for VBR, then there's no problem with the choice of HE-AAC implementation.
WMA & WMAPro may cause problems. They're both working with VBR but we would be terribly lucky if one VBR preset gives us the expected bitrate. VBR 2-pass can't be an option (it was discussed last year) unless you're (Sebastian) ready to accept full tracks in order to build the WMA samples (it means: encode each full track in 2-pass mode, decode it and then precisely extract the 20...30sec desired sample - but it also imply that people have the complete track of all samples).

To sum-up:
Code: [Select]
FORMAT    |  MODE             | IMPLEMENTATION  |
__________|___________________|_________________|
          |                   |                 |
AAC-HE    |  VBR              | Nero Digital    |
Vorbis    |  VBR              | aoTuV beta 5    |
WMA [std] |  VBR if possible  | Microsoft WMP11 |
WMAPro    |  VBR if possible  | Microsoft WMP11 |
__________|___________________|__________________

Multiformat 48 kbps Listening Test

Reply #2 –
Quote from: Sebastian Mares on
Although we had a pre-test discussion already, I wanted a fresh start. This doesn't mean it should take ages before a decision is made like it was the case before.

So - now that the new WMA codec is out and ready for testing, I decided to delay the MP3 listening test and organize this long-awaited multiformat test. The candidates so far are:

Nero HE-AAC
AoTuV Vorbis Beta 5
Windows Media Audio 10

Possible anchors are:

LC-AAC at 48 kbps as low anchor
LC-AAC at 128 kbps or 96 kbps as high anchor

Ivan / Nero Crew and Aoyumi, can you recommend settings for your encoders?

What other contenders would you like to see (preferably other formats)?

In the meanwhile, I am going to re-read the old thread and see if I can find anything useful there.


As 48k is almost exclusively used for streaming audio, RealAudio should be there. Also Coding Technology's HE-AAC (the Winamp dll), as it is by far the dominant codec for HE-AAC Shoutcast radio.

mp3pro is largely abandoned so I see little point in testing it. Maybe ATRAC3, but I guess that's quietly being phased out by Sony as well - do their Walkman phones even support it? WMA-standard is not very interesting I think, as WMA Pro is quickly taking over.

Anchors look fine.

Multiformat 48 kbps Listening Test

Reply #3 –
CT  (Winamp) HE-AAC at 48 kbits gives approx 46-46.5 kbits.  While Nero VBR/CBR, Vorbis and WMA CBR have not problem with bitrate precision.

Multiformat 48 kbps Listening Test

Reply #4 –
According to the WME, for WMA/WMAPro @48kbps, we have an overhead of 6 kbps. So, it's around to 54 kbps.

Multiformat 48 kbps Listening Test

Reply #5 –
Quote from: kurtnoise on
According to the WME, for WMA/WMAPro @48kbps, we have an overhead of 6 kbps. So, it's around to 54 kbps.

Did you verify it?
I also noticed in the past an overhead with WMA, but it seemed to be a fixed one (e.g. +50kb per file). The impact is completely null for full tracks but distort the average bitrate of short encodings.

EDIT: I just looked on some freshly WMA encoded tracks and the final bitrate is correct (expected bitrate + 1 kbps).

Multiformat 48 kbps Listening Test

Reply #6 –
No real verification sorry. I just read these infos in extra settings.

Multiformat 48 kbps Listening Test

Reply #7 –
I would like to test WMA Professional and WMA Standard and see how they allocate bits (=> build a bitrate table). Should I encode each track of my albums individually, sum up the bitrates and divide by the number of samples (roughly 600) or can I encode full CD images (roughly 50)? I would say encoding each track is near to real-world usage.

BTW guys, do you know any tool other than WMP 11 that can encode using the new WMA codec?

Multiformat 48 kbps Listening Test

Reply #8 –
I'd go for separate tracks.
I'm starting on my side a bitrate table for both WMA products. But it will take some time and it only concerns classical music.

Multiformat 48 kbps Listening Test

Reply #9 –
I will let my PC encode this night if I find the right tool. My small collection:

Code: [Select]
Alphaville - Forever Young
Chris Rea - Auberge
Chris Rea - Espresso Logic
Chris Rea - The Road To Hell
Dire Straits - Alchemy
Dire Straits - Brothers In Arms
Dire Straits - Money For Nothing
Eminem - The Eminem Show
Eric Clapton - Unplugged
Evanescence - Fallen
Foreigner - Agent Provocateur
Foreigner - Inside Information
Jethro Tull - Living With The Past
Kiss - Revenge
Kitaro - Best Of Kitaro (Volume 1)
Led Zeppelin - Led Zeppelin I
Led Zeppelin - Led Zeppelin II
Mike Oldfield - The Orchestral Tubular Bells
Peter Gabriel - Shaking The Tree: Sixteen Golden Greats
Peter Gabriel - So
Peter Gabriel - Up
Peter Gabriel - Us
Pink Floyd - A Momentary Lapse Of Reason
Pink Floyd - A Saucerful Of Secrets
Pink Floyd - Animals
Pink Floyd - Delicate Sound Of Thunder
Pink Floyd - Echoes: The Best Of Pink Floyd
Pink Floyd - Meddle
Pink Floyd - The Dark Side Of The Moon
Pink Floyd - The Division Bell
Pink Floyd - The Wall
Pink Floyd - Wish You Were Here
Prince And The Revolution - Purple Rain
Roger Waters - The Wall: Live In Berlin
Santana - Freedom
The London Symphony Orchestra - The Power Of Classic Rock
Toto - Past To Present 1977 - 1990
Uriah Heep - The Ultimate Collection
Verschiedene Künstler - Hits Der 70er Jahre
Verschiedene Künstler - Hits Der 80er Jahre

Multiformat 48 kbps Listening Test

Reply #10 –
Some tips for WMA conversion:
• foobar2000 using WMCmd.vbs script: it works, but I'd like to know if it's normal that each encoding needs so much time to begin... Unusable on my side.

• Winamp 5.31: works great (WMA Std appears as v.9.2 and WMAPro as v.10) but the transcoder needs compatible components (converting from flac files works but not from wavpack).

• dBPowerAmp : should work with v.11 (it worked well with beta versions of WMP) but I don't think that WMA components are available for dBPowerAmp v.12

• VUPlayer : should work, but IIRC the converter doesn't have many encoding options. At least transcoding from WavPack should work with this one (contrary to winamp).

Multiformat 48 kbps Listening Test

Reply #11 –
Me again. In Windows Media Encoder I have four options (translated from German):

CBR
Bitrate VBR (Highest value)
Quality VBR
Bitrate VBR

CBR and Bitrate VBR (Highest value) can be ignored. Do you know if "Bitrate VBR" is the 2-pass VBR mode?

Multiformat 48 kbps Listening Test

Reply #12 –
Guru, were you successful? I just set up fb2k for encoding and with WMA9PRO (which is 10, but the VBScript file is older than the codec) Q10_44_2_24 which is the smallest quality setting, I get around 68 kbps per song.

Multiformat 48 kbps Listening Test

Reply #13 –
I didn't try WMAPro yet.
Before encoding a complete library of full tracks, I'm doing preliminary experimentation on short samples (faster processing). What I can say yet is:
• MrQuestionMan 0.8 doesn't report correctly the bitrate of VBR encoded files (WMA only). I'm currently using foobar2000 but I didn't checked the accuracy yet.
• WMAstd q10 gives me 42 kbps on average [foobar2000] with classical short samples. Bitrate varies from 27 kbps to 93 kbps. But I have several files at 27 kbps, that's why I start to suspect foobar2000 to not give correct values on all files.

Multiformat 48 kbps Listening Test

Reply #14 –
Nobody said anything about mp3PRO. Bleh, you edited your post.

I am experimenting with Alphaville's "Forever Young" album. WMA 10 Professional at Q10 produces around 64 kbps. Bitrate based VBR (2-pass) produces around 48 kbps. Didn't try WMA Standard, yet. However, WMA Professional is not looking very good since plain quality settings won't work for the target bitrate of this test. This brings three possibilities:
  • Use 2-pass VBR and try to obtain full tracks. However, this might be problematic because the people who are sharing the files would do something illegal and me purchasing CDs is not an option. As for cutting out the samples precisely - wouldn't ABC/HR compensate for inaccurately cropped samples?
  • Use CBR for WMA (Professional and possibly Standard depending on the outcome).
  • Inrease testing bitrate to 64 kbps.
Edit: Re-checked bitrates with Mp3tag and WMP instead of Windows Explorer and the figures I posted before were wrong. Seems that Explorer is off by ~5 kbps: the files actually have ~5 kbps less. 64 kbps for Q10 is still too far away from 48 kbps.

Multiformat 48 kbps Listening Test

Reply #16 –
WMA standard -q10 SHORT SAMPLES:
Code: [Select]
                    foobar2000    real bitrate
150 classical         42,83          46,69 kbps
50 non-classical      50,72          51,33 kbps


The difference between foobar2000 & 'real bitrate' (=computed by 'hand') is expected: it must correspond to the impact of the WMA overhead which is significant when compared to the small size of the files. As proof, the difference is much smaller with non-classical samples, which are on average 3 times longer.

A bitrate table based on full tracks should give more reliable values. It will be performed overnight. But I guess that WMA Std -q10 is fine for a 48 kbps comparison.

Multiformat 48 kbps Listening Test

Reply #18 –
WMAPRO -q10 SHORT SAMPLES:
Code: [Select]
                    foobar2000    real bitrate
150 classical         52,03          56,37 kbps
50 non-classical      60,66          61,54 kbps


While VBR10 might be usable with classical music the bitrate is too far from 48 kbps with non-classical music (~61 kbps). My values are again close to yours, Sebastian.
I guess that we could forget WMAPro@VBR for this (48 kbps) test. If I'm correct we still have the choice between CBR and the 2-pass VBR mode, right? Each mode lead to different issue:
-CBR: 'unfair' comparison, or "apples and oranges", or "anti-WMA trolling", or... {choose yours}
-2Pass: legal issue (i.e. sharing full tracks over the net) or changes in organisation (i.e. either allowing other people to prepare the WMA samples or providing yourself all future samples).

Multiformat 48 kbps Listening Test

Reply #19 –
Quote from: guruboolez on
WMAPRO -q10 SHORT SAMPLES:
Code: [Select]
                    foobar2000    real bitrate
150 classical         52,03          56,37 kbps
50 non-classical      60,66          61,54 kbps


While VBR10 might be usable with classical music the bitrate is too far from 48 kbps with non-classical music (~61 kbps). My values are again close to yours, Sebastian.
I guess that we could forget WMAPro@VBR for this (48 kbps) test. If I'm correct we still have the choice between CBR and the 2-pass VBR mode, right? Each mode lead to different issue:
-CBR: 'unfair' comparison
-2Pass: legal issue (i.e. sharing full tracks over the net) or changes in organisation (i.e. either allowing other people to prepare the WMA samples or providing yourself all future samples).


I'm not 100% sure that it will work, but I can provide a WMA encoder for Winamp that let's you have more fine-grained control over the quality value (e.g. 8 or 9).  I'll go ahead and try it, and let me know if it would help you out.

Multiformat 48 kbps Listening Test

Reply #20 –
It would be nice!

Multiformat 48 kbps Listening Test

Reply #22 –
Well, using about the same bitrates WMA Pro's 24 bits files are incomparable to the other contenders' 16 bits anyway, since the latter naturally consume less space, leaving WMA Pro in a clear disadvantage. To improve its chances a little it should be encoded in bitrate-based VBR mode instead, which allows 16 bits stereo at 48 kbps.

Alternatively, since 16 bits files are 2/3 the size of 24 bits ones, you could also allow the WMA Pro codec to participate with 48 kbps*1.5=72 kbps.

Multiformat 48 kbps Listening Test

Reply #23 –
Vorbis, AAC, MP3... aren't 16 bits.

Multiformat 48 kbps Listening Test

Reply #24 –
Quote from: guruboolez on
Vorbis, AAC, MP3... aren't 16 bits.


You forgot WMA Standard, which is also part of the test. Would be new to me if it encoded to anything besides 16 bits.

Edit: Nope, it doesn't, here's part of its specification: "This codec samples audio at 44.1 or 48 kilohertz (kHz) using 16 bits, similar to the current CD standard, offering CD quality at data rates from 64 to 192 kilobits per second (Kbps)."