IPB

Welcome Guest ( Log In | Register )

Public Listening Test [2010], Discussion
IgorC
post Dec 26 2009, 04:06
Post #1





Group: Members
Posts: 1532
Joined: 3-January 05
From: ARG/RUS
Member No.: 18803



Many people will be agree that itís time for public listening test.

Short Agenda:
1. Organization
2. Samples
3. Codecs
4. Bitrates

Detailed Agenda:
1. Organization.
People with experience of personal or public test are welcome here.
Personally I can afford help for conduction of test.
Unfortunately if I understand correctly that Sebastian informed that he donít want to conduct public tests.

2. Samples.
Different styles of music, different levels of difficulty, pointing issues etc....? To be discussed here or in separate topic.

3. Codecs
3.a) Multiformat test.
3.b) AAC test.
I think that itís more appropriate to conduct AAC test because there are at least 3 AAC encoders to test. Nero, Apple and Coding Technologies/Dolby. All these codecs were updated during this year. More information in List of AAC encoders

4.Bitrates.
96-100 kbits/s?
It's possible to perform test at 128 kbits/s with very hard samples. Aemese-like samples will be very easy to spot from original.


Sorry for my English.

All kind of thoughts and suggestions are welcomed here. smile.gif
Go to the top of the page
+Quote Post
 
Start new topic
Replies
Sebastian Mares
post Dec 26 2009, 19:36
Post #2





Group: Members
Posts: 3629
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



I could imagine an AAC test at 80 or 96 kbps. At 80 kbps you could feature both LC and HE encoders and see which one performs best. The winner could be used in a multiformat listening test at the same bitrate.

Regarding the samples, muaddib once had an idea to create a samples DB that should be divided into problem samples and regular samples. When preparing a test, one could pick X samples from that DB based on lottery numbers so that people don't complain that sample Y was selected with the purpose of letting encoder A appear better than encoder B. Additionally, samples collected especially for the respective test could be used like it was done in the past.

Good luck with the test Igor. Are you planing to use existing software (ABC/HR for Java) for the test or something new? ABC/HR's development is dead unfortunately and there were some problems in my last test requiring the installation of JRE 1.5 (which some people with JRE 1.6 found annoying).


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
IgorC
post Dec 26 2009, 20:39
Post #3





Group: Members
Posts: 1532
Joined: 3-January 05
From: ARG/RUS
Member No.: 18803



QUOTE (Sebastian Mares @ Dec 26 2009, 15:36) *
Good luck with the test Igor.

That sounded sarcastic. laugh.gif


QUOTE (Sebastian Mares @ Dec 26 2009, 15:36) *
Are you planing to use existing software (ABC/HR for Java) for the test or something new? ABC/HR's development is dead unfortunately and there were some problems in my last test requiring the installation of JRE 1.5 (which some people with JRE 1.6 found annoying).

Yes, I remember of this issue and speaking of me I could get rid of this without problem however with some workaround. Must see what happens with other people.

QUOTE (antman @ Dec 26 2009, 15:06) *
Definately AAC. Nero v new Nero alone will bring in a lot of interest. Add in Apple's latest, with it's true vbr options. And it'd be interesting to see CT/Dobly in the mix since it's been left out of so many AAC discussions. And I agree with you, it should be in the ~96k range.

True VBR mode isn't available in Windows version. Many people couldn't use it.

QUOTE (C.R.Helmrich @ Dec 26 2009, 16:00) *
An AAC test would be interesting indeed. But at 96 kbps or so, you would have to test both AAC LC and HE-AAC, since they offer about the same average quality. With about 4 encoders under test (Apple, 2x nero, Dolby, ...), this would give 8 codecs under test. That's too many in my opinion (risk of overload and fatigue).

Itunes has HE-AAC only up to 80 kbps. Nero's defaults option goes to HE-AAC up to ~85 kbps.
From my personal test Apple LC-AAC is better than HE-AAC already at 80 kbps.
http://www.hydrogenaudio.org/forums/index....showtopic=74781
But yes it's personal test, not public.
96 kbps will be enough to test only LC-AAC codecs.
Anyways all bitrates should be shifted to 100 kbps as Itunes constrained VBR mode at 96 kbps produces real bitrates ~100 kbps. There is mediacoder which can shift bitrates for CT encoder.


QUOTE (C.R.Helmrich @ Dec 26 2009, 16:00) *
How about 112 kbps or so? There you can be relatively sure that LC is better then HE on average, i.e. you would need to test only LC.
Chris

iTunes hasn't 112 kbps. Only 96 and 128.

As I can see people interesting in AAC test and next codecs:
1. Nero 1.3.3
2. Nero 1.5.1
3. CT
4. Apple
5. Divx (???) or other?

I think 4-5 codecs are already enough. Not more. What do you think?

I found that Divx AAC enocoder has all potential to get into test.
http://www.hydrogenaudio.org/forums/index....st&p=675456

This post has been edited by IgorC: Dec 26 2009, 20:41
Go to the top of the page
+Quote Post

Posts in this topic
- IgorC   Public Listening Test [2010]   Dec 26 2009, 04:06
- - Fandango   Codecs: I would love to see an AAC test including ...   Dec 26 2009, 18:02
- - antman   Definately AAC. Nero v new Nero alone will bring ...   Dec 26 2009, 19:06
|- - C.R.Helmrich   An AAC test would be interesting indeed. But at 96...   Dec 26 2009, 20:00
- - Sebastian Mares   I could imagine an AAC test at 80 or 96 kbps. At 8...   Dec 26 2009, 19:36
|- - IgorC   QUOTE (Sebastian Mares @ Dec 26 2009, 15...   Dec 26 2009, 20:39
||- - guruboolez   ~100 kbps listening test should be fine. Quality a...   Dec 26 2009, 21:06
|||- - IgorC   QUOTE (guruboolez @ Dec 26 2009, 17:06) A...   Dec 27 2009, 21:04
|||- - Polar   QUOTE (IgorC @ Dec 27 2009) QUOTE (guruboolez...   Dec 28 2009, 03:18
||||- - IgorC   QUOTE (Polar @ Dec 27 2009, 23:18) So I g...   Dec 28 2009, 06:13
||||- - Polar   QUOTE (IgorC @ Dec 28 2009) High anchor. Do w...   Dec 28 2009, 09:20
||||- - C.R.Helmrich   QUOTE (Polar @ Dec 28 2009, 10:20) Imho, ...   Dec 28 2009, 12:29
|||||- - guruboolez   I like it (with some reserves for low anchor).   Dec 28 2009, 13:39
|||||- - IgorC   QUOTE (guruboolez @ Dec 28 2009, 05:06) I...   Dec 29 2009, 03:25
||||||- - C.R.Helmrich   QUOTE (IgorC @ Dec 29 2009, 04:25) QUOTE ...   Dec 30 2009, 20:37
|||||- - IgorC   After reading Roberto's (fast ) manual about l...   Dec 30 2009, 22:37
|||||- - Porcus   QUOTE (C.R.Helmrich @ Dec 28 2009, 12:29)...   Jan 3 2010, 14:35
|||||- - C.R.Helmrich   QUOTE (Porcus @ Jan 3 2010, 15:35) I am n...   Jan 3 2010, 17:34
|||||- - jido   QUOTE (C.R.Helmrich @ Jan 3 2010, 08:34) ...   Jan 6 2010, 00:09
||||- - IgorC   QUOTE (Polar @ Dec 28 2009, 05:20) Fwiw, ...   Dec 29 2009, 03:39
|||- - guruboolez   QUOTE (IgorC @ Dec 27 2009, 22:04) There ...   Dec 28 2009, 09:06
||- - C.R.Helmrich   I see. Didn't know that most encoders don...   Dec 26 2009, 21:06
|||- - C.R.Helmrich   QUOTE (C.R.Helmrich @ Dec 26 2009, 22:06)...   Jan 6 2010, 13:23
||- - Sebastian Mares   QUOTE (IgorC @ Dec 26 2009, 20:39) QUOTE ...   Dec 26 2009, 23:12
|- - IgorC   QUOTE (Sebastian Mares @ Dec 26 2009, 15...   Dec 26 2009, 20:55
- - hellokeith   QUOTE (IgorC @ Dec 25 2009, 22:06) 4.Bitr...   Dec 27 2009, 04:28
- - antman   QUOTE (IgorC @ Dec 26 2009, 14:39) True V...   Dec 27 2009, 05:36
- - IgorC   The poll of Nero 1.5.1 vs 1.3.3 is opened. http://...   Dec 28 2009, 00:16
- - /mnt   I would like to see Quicktime's new true VBR e...   Dec 28 2009, 00:25
|- - kornchild2002   QUOTE (/mnt @ Dec 27 2009, 16:25) I would...   Dec 29 2009, 05:25
- - funkyblue   Please tell me if I'm on the wrong track. How...   Dec 28 2009, 01:16
- - roozhou   Why not add ffmpeg aac encoder to this test?   Dec 28 2009, 11:32
|- - kurtnoise   QUOTE (roozhou @ Dec 28 2009, 11:32) Why ...   Dec 28 2009, 13:47
- - IgorC   I propose to include only iTunes constrained VBR m...   Dec 29 2009, 22:12
- - antman   I still say quicktime true vbr. I think if the re...   Dec 29 2009, 23:10
- - IgorC   All right, there is poll (true vs constrained VBR)...   Dec 30 2009, 01:08
|- - kornchild2002   I am not sure how the bitrates of true VBR and VBR...   Dec 30 2009, 02:02
|- - birdy25   QUOTE (IgorC @ Dec 29 2009, 19:08) All ri...   Dec 30 2009, 18:21
|- - IgorC   QUOTE (birdy25 @ Dec 30 2009, 14:21) QUOT...   Dec 30 2009, 23:17
- - jido   This is Hydrogenaudio, why not using LossyWAV for ...   Jan 3 2010, 16:25
- - Sebastian Mares   Regarding the results, I would personally discard ...   Jan 3 2010, 22:05
|- - IgorC   QUOTE (Sebastian Mares @ Jan 3 2010, 18:0...   Jan 3 2010, 22:09
|- - Sebastian Mares   QUOTE (IgorC @ Jan 3 2010, 22:09) QUOTE (...   Jan 3 2010, 22:14
|- - IgorC   QUOTE (Sebastian Mares @ Jan 3 2010, 18:1...   Jan 3 2010, 22:43
- - IgorC   As high anchor will be dropped then we can add one...   Jan 4 2010, 05:56
- - Sebastian Mares   Are you sure 48 kbps LC is not too exaggerated? Do...   Jan 4 2010, 11:42
|- - C.R.Helmrich   IgorC: Yes, Fraunhofer's AAC encoder supports ...   Jan 4 2010, 14:06
|- - IgorC   QUOTE (C.R.Helmrich @ Jan 4 2010, 10:06) ...   Jan 6 2010, 00:52
- - devnul   Wasn't the primary goal to test the latest ner...   Jan 4 2010, 15:40
- - kornchild2002   Though many people want to see how QuickTimes true...   Jan 4 2010, 18:10
- - nao   For Windows users, I made a tiny tool to access th...   Jan 5 2010, 17:14
|- - IgorC   QUOTE (nao @ Jan 5 2010, 13:14) For Windo...   Jan 5 2010, 20:39
|- - lvqcl   QUOTE (nao @ Jan 5 2010, 19:14) For Windo...   Jan 6 2010, 01:21
- - kornchild2002   Performance isn't all that great with with foo...   Jan 5 2010, 17:53
|- - nao   As far as I've tested with fb2k 0.9.6.9 and wi...   Jan 5 2010, 18:38
- - Larson   i've tested your script nao and thank you so m...   Jan 5 2010, 18:42
- - lvqcl   QUOTE (kornchild2002)Lame.exe and Nero.exe consume...   Jan 5 2010, 18:45
- - kornchild2002   I don't see a thread count option under the Co...   Jan 5 2010, 19:40
- - Enig123   I might have missed something, but where to get th...   Jan 6 2010, 02:18
|- - Sylph   QUOTE (Enig123 @ Jan 6 2010, 02:18) I mig...   Jan 6 2010, 22:27
- - nao   Updated qtaacenc. QUOTE (kornchild2002 @ Jan...   Jan 6 2010, 18:37
- - tedgo   @nao Thanks for your qtaacenc . I love it! May...   Jan 6 2010, 19:01
|- - kornchild2002   QUOTE (tedgo @ Jan 6 2010, 11:01) Maybe y...   Jan 6 2010, 22:33
- - rpp3po   I don't think that a <128kbit/s listening t...   Jan 8 2010, 16:57
|- - halb27   QUOTE (rpp3po @ Jan 8 2010, 16:57) ... we...   Jan 8 2010, 18:23
|- - C.R.Helmrich   QUOTE (halb27 @ Jan 8 2010, 19:23) QUOTE ...   Jan 8 2010, 21:19
|- - rpp3po   QUOTE (C.R.Helmrich @ Jan 8 2010, 21:19) ...   Jan 8 2010, 22:14
|- - C.R.Helmrich   QUOTE (rpp3po @ Jan 8 2010, 23:14) And th...   Jan 8 2010, 23:03
|- - IgorC   QUOTE (rpp3po @ Jan 8 2010, 18:14) The en...   Jan 12 2010, 03:29
|- - halb27   QUOTE (IgorC @ Jan 12 2010, 03:29) [...I...   Jan 12 2010, 08:44
|- - /mnt   QUOTE (IgorC @ Jan 12 2010, 03:29) QUOTE ...   Jan 12 2010, 21:56
- - MichaelW   A question from the peanut gallery. Do different...   Jan 8 2010, 23:43
|- - IgorC   QUOTE (MichaelW @ Jan 8 2010, 19:43) A qu...   Jan 12 2010, 03:17
- - IgorC   /mnt Thank you for data. As poll indicates there ...   Jan 14 2010, 18:35
- - halb27   I'd welcome to have winamp's CT AAC encode...   Jan 14 2010, 21:10
|- - IgorC   QUOTE (halb27 @ Jan 14 2010, 17:10) I...   Jan 14 2010, 21:35
- - /mnt   I would like see to DivX's AAC encoder on the ...   Jan 14 2010, 21:22
- - greynol   I thought I'd just pipe in and say I'm opp...   Jan 14 2010, 21:49
|- - IgorC   QUOTE (greynol @ Jan 14 2010, 17:49) I th...   Jan 14 2010, 22:04
- - greynol   Alright then; the masses have spoken. Good luck w...   Jan 14 2010, 22:13
- - kornchild2002   Yeah, I don't like it either. It takes 5 minu...   Jan 14 2010, 23:27
|- - C.R.Helmrich   I agree. Since I have yet to see a blind test comp...   Jan 14 2010, 23:47
- - IgorC   One thing I'm afraid that the difference betwe...   Jan 15 2010, 00:33
|- - rpp3po   The effect of TVBR is mainly space saving at highe...   Jan 15 2010, 01:42
- - IgorC   This Poll will define T&CVBR's comparison...   Jan 16 2010, 17:16
- - IgorC   As now Apple TVBR is available for Windows users s...   Jan 23 2010, 03:28
|- - greynol   QUOTE (IgorC @ Jan 22 2010, 18:28) As now...   Jan 23 2010, 21:41
- - halb27   IMO we should use --highest. Though not many peop...   Jan 23 2010, 21:10
- - IgorC   halb, I think we came back to specific options or...   Jan 23 2010, 23:06
- - halb27   I understand your considerations, and part of them...   Jan 23 2010, 23:36
- - IgorC   iTunes AAC encodes at fixed quality option which i...   Jan 24 2010, 00:06
- - halb27   I'm not quite sure I understand what you'r...   Jan 24 2010, 00:22
- - IgorC   Yes, that's correct. Maybe there are people ...   Jan 24 2010, 01:22
- - IgorC   This open discussion will have enough time but can...   Jan 25 2010, 05:30
- - halb27   As for 3. I'd like to see ABR in the test beca...   Jan 25 2010, 08:59
- - IgorC   I've tried a few samples for ABX test on ABR v...   Jan 27 2010, 03:13
- - halb27   OK, I see the arguments (and your findings), so le...   Jan 27 2010, 09:46
- - IgorC   The rules: CODERemove all listeners from analysis ...   Jan 27 2010, 16:01
- - IgorC   Nero Bitrate settings: -q 0.42 is comparable with...   Jan 27 2010, 16:12
- - Alex B   Regarding the bitrates, I have run some tests usin...   Jan 27 2010, 16:41
4 Pages V   1 2 3 > » 


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 24th July 2014 - 14:45