IPB

Welcome Guest ( Log In | Register )

48 kbps listening test, pretest discussion thread, Pretest thread / Encoder settings
Shade[ST]
post Dec 12 2005, 17:06
Post #1





Group: Members
Posts: 1189
Joined: 19-May 05
From: Montreal, Canada
Member No.: 22144



Hello!

After assisting like many of you on the controversy of guruboolez' tests concerning the Nero AAC encoders and having seen many suggestions towards preparing a listening test around 48kbps - in particular with the different HE encoders, I decided to take matters into my own hands.
I currently have 8 samples testing for tonality, transients and stereo separation. These are generally classical music, but also include electronic music, 80s pop, and I might add a jazz recording for ambience reproduction.
As for encoders, I'm stumped.

Should I also include an old version of nero (old being relative) as to see the progress in a general evaluation ? At this bitrate, many encoders is not really a problem, is it? For now, I was thinking :

-Nero AAC Tape / PNS Enabled / HE / High (Should I enable hint track?)
-Ogg Vorbis, latest AoTuV, q -2 to -4 (whatever's approximately the same bitrates ~42-53)
-Lame ABR 48 (any tunings suggested?)
-mp3pro, probably, at any close bitrate I can achieve (adobe audition's encoder)
-Itunes AAC (HE if available)
-CT encoder (winamp 5.12) 48kbps, HE.

If I should use an older nero as well, where can I find it?

Also, could anyone help me prepare the ABC/HR logs? (eg, Sebastian Mares)
Thanks in advance,
Tristan.
Go to the top of the page
+Quote Post
 
Start new topic
Replies
Ivan Dimkovic
post Dec 13 2005, 13:13
Post #2


Nero MPEG4 developer


Group: Developer
Posts: 1466
Joined: 22-September 01
Member No.: 8



But @Gabriel, do you think that the 32kbps resutls would extrapolate well to 48 kbps to be useful for the selection of the encoder? I have a fear that some codecs >might< show completely different behavior in terms of ranking - not sure about that, but just maybe.

Also, there is another problem @48 kbps - while @32 kbps it is pretty clear that HE-AAC v2 will outperform HE-AAC v1 in all implementations, at 48 kbps this is starting to be a bit questionable as it has been shown on few internal tests of HE-AAC v1 vs HE-AAC v2 at 48 kbits/s.

I will definitely organise a separate small listening test of HE-AAC v1 and v2 codecs @48 kbps (and Ogg maybe) - by using of the 10-or-so average worst samples (combined average SDG) from the previous 128 and 64 kbps tests.

My intial plan was to use:

* 5 samples from the previous multiformat 64 kbps test, with worst average SDG
* 5 samples from the previous 128 kbps test, with worst average SDG

This will help chose the proper AAC mode at 48 kbps for the future 48 kbps multiformat listening, because there is an open issue whether Parametric Stereo (PS) or HE-AAC v2 is improving quality enough or the stereo-field regressions are too bad to justify the quality increase on some clips. Testing these two modes side-by-side directly would help us make the choice for the future.

Also I think this will have very big value for the people considering HE-AAC/HE-AAC v2 for their 48 kbps encodings/streams - as it would show whether Parametric Stereo is good for that bitrate or not. Currently there is not yet enough test data to make advisory - this test would definitely help smile.gif

I planned to do this test soon (in 10 or so days maybe)- I will post it on HA, too - as well as methodology and the selection criteria for the samples which will be completely transparent and unbiased toward any codec/solution.

For sure there will be:

- CT HE-AAC v1 (aacPlus)
- CT HE-AAC v2 (aacPlus v2)
- Nero Digital - HE-AAC v1 [new codec]
- Nero Digital - HE-AAC v2 [new codec]

- And probably LAME @128 kbps as high-anchor
- LAME @48 kbps (edit: or, even better, AAC @48 kbps), as low anchor


But I am not sure whether to reveal the codec names in the test resutls (I would anyway do it for the orgnizers of the future tests), as I am sure that there will be doubts about the credibility here (as always smile.gif - and by not promoting any codec I am trying to stay as neutral as possible - this test is about the v1 and v2 HE-AAC technology, not about the particular implementations.

This post has been edited by Ivan Dimkovic: Dec 13 2005, 13:29
Go to the top of the page
+Quote Post
stephanV
post Dec 13 2005, 14:05
Post #3





Group: Members
Posts: 394
Joined: 6-May 04
Member No.: 13932



QUOTE (Ivan Dimkovic @ Dec 13 2005, 01:13 PM)
But I am not sure whether to reveal the codec names in the test resutls (I would anyway do it for the orgnizers of the future tests), as I am sure that there will be doubts about the credibility here (as always smile.gif - and by not promoting any codec  I am trying to stay as neutral as possible - this test is about the v1 and v2 HE-AAC technology, not about the particular implementations.
*

You are kidding right? smile.gif

As long as results are reproducible (if we now, or in the soon future, can use an encoder to create the files that were tested)and the selection of samples is transparent and open for discussion, I can't see any reason why it would hurt the credibility of the test if the names where revealed.

The reason I'm bringing this up is two-fold:

1. The results will be less interesting and people might feel less inclined to participate. (assuming your test is open for us)

2. Technology and implementation are not always very separable. For example, you can't conclude that MP3 is a better technology than Vorbis or AAC by comparing LAME with a very bad AAC or Vorbis encoder. I don't know if this comparison holds also when it comes to SBR and PS, but it would be something to consider I think.


--------------------
"We cannot win against obsession. They care, we don't. They win."
Go to the top of the page
+Quote Post

Posts in this topic
- Shade[ST]   48 kbps listening test, pretest discussion thread   Dec 12 2005, 17:06
- - guruboolez   Is it a collective listening test or a personal ex...   Dec 12 2005, 17:10
|- - Shade[ST]   QUOTE (guruboolez @ Dec 12 2005, 10:10 AM)Is ...   Dec 12 2005, 17:14
- - guruboolez   Isn't it a bit premature to start a new pretes...   Dec 12 2005, 17:16
- - Ivan Dimkovic   If it is a collective test, may I ask for delaying...   Dec 12 2005, 17:16
|- - Shade[ST]   QUOTE (Ivan Dimkovic @ Dec 12 2005, 10:16 AM)...   Dec 12 2005, 17:20
|- - naylor83   QUOTE (Shade[ST] @ Dec 12 2005, 06:2...   Dec 12 2005, 17:46
- - Sebastian Mares   I think the whole thing is too rushed. Personally,...   Dec 12 2005, 17:52
|- - naylor83   QUOTE (Sebastian Mares @ Dec 12 2005, 06:52 P...   Dec 12 2005, 18:22
||- - Sebastian Mares   QUOTE (naylor83 @ Dec 12 2005, 06:22 PM)QUOTE...   Dec 12 2005, 18:24
||- - naylor83   QUOTE (Sebastian Mares @ Dec 12 2005, 07:24 P...   Dec 12 2005, 18:34
||- - Shade[ST]   I was thinking maybe about making LAME the low anc...   Dec 12 2005, 21:20
|- - user   QUOTE (Sebastian Mares @ Dec 12 2005, 05:52 P...   Dec 14 2005, 12:00
- - Sebastian Mares   Regarding the encoders again - personally, I would...   Dec 12 2005, 18:32
|- - Latexxx   I'd personally drop mp3pro because its margina...   Dec 12 2005, 21:35
|- - dimzon   QUOTE (Latexxx @ Dec 13 2005, 12:35 AM) mp3 (...   Dec 13 2005, 09:38
- - Gambit   With all respect, Shade[ST], this is a really bad ...   Dec 12 2005, 21:17
- - rjamorim   QUOTE (Shade[ST] @ Dec 12 2005, 02:2...   Dec 13 2005, 00:46
|- - Shade[ST]   QUOTE (rjamorim @ Dec 12 2005, 05:46 PM)QUOTE...   Dec 13 2005, 04:07
|- - naylor83   QUOTE (Shade[ST] @ Dec 13 2005, 05:0...   Dec 13 2005, 10:10
- - Ivan Dimkovic   QUOTE don't forget about 3GP reference encoder...   Dec 13 2005, 10:42
|- - dimzon   QUOTE (Ivan Dimkovic @ Dec 13 2005, 01:42 PM)...   Dec 13 2005, 17:20
- - Gabriel   My opinions is that it could be a good idea to fir...   Dec 13 2005, 12:51
- - Ivan Dimkovic   But @Gabriel, do you think that the 32kbps resutls...   Dec 13 2005, 13:13
|- - stephanV   QUOTE (Ivan Dimkovic @ Dec 13 2005, 01:13 PM)...   Dec 13 2005, 14:05
- - Ivan Dimkovic   QUOTE You are kidding right? I wish - but reading...   Dec 13 2005, 14:17
|- - Shade[ST]   Any word of yet as to how any of these tests are g...   Dec 13 2005, 16:10
- - Ivan Dimkovic   QUOTE Any word of yet as to how any of these tests...   Dec 13 2005, 17:34
|- - IgorC   It wil be good to see not only music samples but a...   Dec 13 2005, 19:40
|- - Halcyon   Before you start any test, I suggest you take the ...   Dec 13 2005, 21:24
- - Ivan Dimkovic   Gabriel and I discussed these ideas - and we will ...   Dec 14 2005, 12:43
- - fpi   Before testing aoTuV beta 4.51 would be interestin...   Dec 14 2005, 16:45
- - gameplaya15143   QUOTE (fpi @ Dec 14 2005, 10:45 AM)Before tes...   Jan 18 2006, 04:15
- - pepoluan   Ahhh... the thread has been resurrected So. Is...   Jan 19 2006, 17:51


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 29th November 2014 - 11:20