IPB

Welcome Guest ( Log In | Register )

3 Pages V   1 2 3 >  
Closed TopicStart new topic
Public AAC Listening Test @ ~96 kbps [July 2011], Nero, Apple, Fraunhofer and Coding Technologies (CT)
IgorC
post Jul 17 2011, 22:55
Post #1





Group: Members
Posts: 1581
Joined: 3-January 05
From: ARG/RUS
Member No.: 18803



Today we have a chance to see the performance of well known AAC encoders like Nero, Apple and Coding Technologies (previous versions of Winamp).
Also Fraunhofer have released their totally new AAC encoder.


The following codecs are presented in this test:
Nero 1.5.4
Apple QuickTime 7.6.9 true VBR
Apple QuickTime 7.6.9 constrained VBR
Fraunhofer (Winamp 5.62)
Coding Technologies (Winamp 5.61)
ffmpeg's AAC (low anchor)

The homepage of the test:
Public AAC Listening Test @ ~96 kbps [July 2011]


There are 20 samples. Please, don't hurry. Take your time as there will be enough of it. It's better to do fewer samples per day and avoid a fatigue.
The test will be open during 10 days (until July 27)

As always every single result is helpful.
Have a good time testing it! smile.gif
Go to the top of the page
+Quote Post
benski
post Jul 18 2011, 17:53
Post #2


Winamp Developer


Group: Developer
Posts: 670
Joined: 17-July 05
From: Brooklyn, NY
Member No.: 23375



Thanks so much, Igor.

Hope you don't mind, but I mirrored the large ZIP file with all the samples here:
http://download.nullsoft.com/listening-tes...all_samples.zip

This post has been edited by benski: Jul 18 2011, 17:54
Go to the top of the page
+Quote Post
Garf
post Jul 18 2011, 19:54
Post #3


Server Admin


Group: Admin
Posts: 4886
Joined: 24-September 01
Member No.: 13



I added the code to randomize the sample list, hope that's OK with you.
Go to the top of the page
+Quote Post
IgorC
post Jul 18 2011, 20:01
Post #4





Group: Members
Posts: 1581
Joined: 3-January 05
From: ARG/RUS
Member No.: 18803



QUOTE (benski @ Jul 18 2011, 13:53) *
Hope you don't mind, but I mirrored the large ZIP file with all the samples here:
http://download.nullsoft.com/listening-tes...all_samples.zip

Thank you.

QUOTE (Garf @ Jul 18 2011, 15:54) *
I added the code to randomize the sample list, hope that's OK with you.

Yes, it's better this way

This post has been edited by IgorC: Jul 18 2011, 20:07
Go to the top of the page
+Quote Post
IgorC
post Jul 19 2011, 01:13
Post #5





Group: Members
Posts: 1581
Joined: 3-January 05
From: ARG/RUS
Member No.: 18803



Ok, some listeners have started to send their results.
It will be better if somebody who has already any results send them. It helps to prevent some simple errors related to ABC-HR application or any other at early stage.
The listeners usually will get answer quickly if everything is fine.

Also it is recommended to use headphones during the test.


Go to the top of the page
+Quote Post
kennedyb4
post Jul 20 2011, 12:11
Post #6





Group: Members
Posts: 772
Joined: 3-October 01
Member No.: 180



Hello. I am trying to participate. I downloaded the sample pack above and used the decode all bat file. I can't get abc/hr to play anything yet.

Can someone point me to a tutorial? I see a training mode but I don't really know what I'm doing.

Edit Ok I checked out ff123 page. I have opened the config file for sample 1 after running decode all as well as decode sample 1 bat files. there are 6 wav files in the directory and they all play in winamp OK.

The main abc/hr page is blank with no samples selected or able to be played. I have tried to load the config file over and again with no result. Still can't play anything.

Edit 2 ABC/HR auto selected spdif out on my m-audiophile 24/96 rather than line. I can play and hear now. Still not sure what I am doing though smile.gif

This post has been edited by kennedyb4: Jul 20 2011, 13:07
Go to the top of the page
+Quote Post
xnor
post Jul 20 2011, 15:53
Post #7





Group: Developer
Posts: 1016
Joined: 29-April 11
From: Austria
Member No.: 90198



I've taken the liberty to mention this test over at head-fi (link), hope that's ok IgorC.

This post has been edited by xnor: Jul 20 2011, 15:54


--------------------
"we are having an educated and deep technical discussion"-amirm
Go to the top of the page
+Quote Post
IgorC
post Jul 20 2011, 22:34
Post #8





Group: Members
Posts: 1581
Joined: 3-January 05
From: ARG/RUS
Member No.: 18803



QUOTE (kennedyb4 @ Jul 20 2011, 08:11) *
Edit 2 ABC/HR auto selected spdif out on my m-audiophile 24/96 rather than line. I can play and hear now.

So does it mean you have managed to fix the problems?
ABC-HR application has presented the problem in past. See the link below to fix some problems.
http://www.hydrogenaudio.org/forums/index....st&p=749194


QUOTE (xnor @ Jul 20 2011, 11:53) *
I've taken the liberty to mention this test over at head-fi (link), hope that's ok IgorC.

Thank You, I haven't time to do it yet.

This post has been edited by IgorC: Jul 20 2011, 22:35
Go to the top of the page
+Quote Post
naylor83
post Jul 21 2011, 16:48
Post #9





Group: Members
Posts: 210
Joined: 19-June 05
From: Uppsala, Sweden
Member No.: 22842



Was a long time since I was here ... Just found out about the listening test. Will definitely join in. I'm very curious to see how the best AAC codec fares against AoTuV b6, in a future test.


--------------------
davidnaylor.org
Go to the top of the page
+Quote Post
kennedyb4
post Jul 21 2011, 17:30
Post #10





Group: Members
Posts: 772
Joined: 3-October 01
Member No.: 180



QUOTE (IgorC @ Jul 20 2011, 17:34) *
QUOTE (kennedyb4 @ Jul 20 2011, 08:11) *
Edit 2 ABC/HR auto selected spdif out on my m-audiophile 24/96 rather than line. I can play and hear now.

So does it mean you have managed to fix the problems?
ABC-HR application has presented the problem in past. See the link below to fix some problems.
http://www.hydrogenaudio.org/forums/index....st&p=749194


I can hear the samples now,yes. Finding fault with them is proving much harder in some cases. I can't even hear the lowpass which is really suprising me. Much more difficult than I anticipated. Not like listening to Blade at 96kbps smile.gif

This post has been edited by db1989: Jul 21 2011, 17:46
Reason for edit: removing unrelated part of quote
Go to the top of the page
+Quote Post
naylor83
post Jul 21 2011, 18:06
Post #11





Group: Members
Posts: 210
Joined: 19-June 05
From: Uppsala, Sweden
Member No.: 22842



QUOTE (kennedyb4 @ Jul 21 2011, 17:30) *
Finding fault with them is proving much harder in some cases. I can't even hear the lowpass which is really suprising me. Much more difficult than I anticipated. Not like listening to Blade at 96kbps smile.gif


Don't like the sound of that crying.gif


--------------------
davidnaylor.org
Go to the top of the page
+Quote Post
guruboolez
post Jul 21 2011, 21:57
Post #12





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



My first listening tests since many years smile.gif
I probably lost most of my skill, but it's very interesting for me to see if I'm still able to make a distinction between ~100 kbps encoding and an uncompressed PCM sound. I guess encoders have progressed during the last years. I remember that last time I tested LC-AAC at this bitrate, I was rather disappointed. But is was four or five years ago and I was at the apogee of my skill.

For now, I completed the five first samples. Well, it's interesting! It's nice to see that so many implementations in one test (but it makes the evaluation longer and harder).
Thank you IgorC and other people involved for organizing it!

This post has been edited by guruboolez: Jul 21 2011, 22:00
Go to the top of the page
+Quote Post
xnor
post Jul 21 2011, 23:30
Post #13





Group: Developer
Posts: 1016
Joined: 29-April 11
From: Austria
Member No.: 90198



It's definitely not an easy test.


--------------------
"we are having an educated and deep technical discussion"-amirm
Go to the top of the page
+Quote Post
kennedyb4
post Jul 22 2011, 00:28
Post #14





Group: Members
Posts: 772
Joined: 3-October 01
Member No.: 180



Question. I have used a short selection in the abx mode for a given sample. Once I use test mode,not training mode, I cant erase my results and try again in test mode. I can only use training mode even though I am checking a different time frame of the sample.

Is it supposed to be this way or can I reset back to test mode?
Go to the top of the page
+Quote Post
muaddib
post Jul 22 2011, 09:19
Post #15





Group: Developer
Posts: 401
Joined: 14-October 01
Member No.: 289



QUOTE (kennedyb4 @ Jul 22 2011, 01:28) *
Question. I have used a short selection in the abx mode for a given sample. Once I use test mode,not training mode, I cant erase my results and try again in test mode. I can only use training mode even though I am checking a different time frame of the sample.

Is it supposed to be this way or can I reset back to test mode?

It is supposed to be that way and does exactly what it should. Don't start test mode until you have heard a complete sample and choose your "worst" section before starting the test mode.
Go to the top of the page
+Quote Post
kennedyb4
post Jul 22 2011, 14:02
Post #16





Group: Members
Posts: 772
Joined: 3-October 01
Member No.: 180



QUOTE (muaddib @ Jul 22 2011, 04:19) *
QUOTE (kennedyb4 @ Jul 22 2011, 01:28) *
Question. I have used a short selection in the abx mode for a given sample. Once I use test mode,not training mode, I cant erase my results and try again in test mode. I can only use training mode even though I am checking a different time frame of the sample.

Is it supposed to be this way or can I reset back to test mode?

It is supposed to be that way and does exactly what it should. Don't start test mode until you have heard a complete sample and choose your "worst" section before starting the test mode.


Thanks. I tested a few sections that I had been able to abx in practice mode and failed during the actual test. I was then able to pick another section that I was able to detect in practice mode but could not repeat the testing.

This is the first time I have used abx/hr. I will run more practice tests in the future before making a definitive test.
Go to the top of the page
+Quote Post
kennedyb4
post Jul 22 2011, 15:37
Post #17





Group: Members
Posts: 772
Joined: 3-October 01
Member No.: 180



OK. Mission accomplished and results sent in. Did 20 samples over 3 days. I'm on vacation so what the heck. Very interesting process but a bit grueling too.

I had lots of trouble. Figured I would at least be able to detect a 16K lowpass regularly but I can't anymore. I was at a Rockfest for 4 days last weekend which I hope is part of the problem.

I hope my results are not rubbish because I put in alot of time.

Can't wait to see how it all turns out.smile.gif Thanks to all concerned for an interesting experience.
Go to the top of the page
+Quote Post
Zarggg
post Jul 22 2011, 16:35
Post #18





Group: Members
Posts: 569
Joined: 18-January 04
From: bethlehem.pa.us
Member No.: 11318



I'm going to try finishing this one; I never finished the first test I participated in here, and I didn't participate in the last one at all. (48kbps, which I might have actually had fun with).

One slight issue I've noticed so far: I was not able to identify the low anchor at all on Sample 3. Does this predict a problem for my results? The quality this test focuses is right on the edge of my transparent zone, so I don't want to risk tainting the results.

This post has been edited by Zarggg: Jul 22 2011, 16:37
Go to the top of the page
+Quote Post
kennedyb4
post Jul 22 2011, 17:32
Post #19





Group: Members
Posts: 772
Joined: 3-October 01
Member No.: 180



QUOTE (Zarggg @ Jul 22 2011, 11:35) *
I'm going to try finishing this one; I never finished the first test I participated in here, and I didn't participate in the last one at all. (48kbps, which I might have actually had fun with).

One slight issue I've noticed so far: I was not able to identify the low anchor at all on Sample 3. Does this predict a problem for my results? The quality this test focuses is right on the edge of my transparent zone, so I don't want to risk tainting the results.


You'll get it. Its not mangled like the other samples but its there. Its definitely harder than the others.

If I remember the other listening test discussions, they reject results on certain criteria, like consistently incorrectly identifying the reference or giving it scores too low to be credible. Honest results should not taint but contribute to the experiment.

PS I think I'll get my son to join HA and give this a try. He is a music major and interested in music and music compression. Maybe he could get his friends interested....

This post has been edited by kennedyb4: Jul 22 2011, 17:41
Go to the top of the page
+Quote Post
IgorC
post Jul 22 2011, 19:24
Post #20





Group: Members
Posts: 1581
Joined: 3-January 05
From: ARG/RUS
Member No.: 18803



QUOTE (guruboolez @ Jul 21 2011, 17:57) *
My first listening tests since many years smile.gif

I'm glad to see You back. cool.gif

As first observation, some listeners are expecting to hear the same artifacts as in past. And of course there are new versions of codecs and artifacts are different.
The more listener tries to listen some particular artifacts the less chance to spot any of them. But if the listener just tries to enjoy the music itself then the artifacts appear themselves.

Also it's very important to mention about some practice with headphones. It's known that even good headphones can sound bad because of not optimal position on the head. It's easy to check. While the music is playing the listener should try different positions and distances (different adjustment of the rim) without set the headphones completely on the head.
It might sound crazy but it works.
More here:

http://www.meier-audio.homepage.t-online.de/tipstricks.htm
QUOTE
headphone position

It's so easy, but did you ever test for the optimal position of your headphone? With many headphones the soundstage and the balance of sound are strongly dependent on the position of the driver to our ears.

Scientific research has shown that our perception of depth is increased when the driver is placed more forward and lower with respect to the entrance of the ear channel.

Experimenting doesn't cost you anything and may improve sound considerably.


This post has been edited by IgorC: Jul 22 2011, 19:31
Go to the top of the page
+Quote Post
muaddib
post Jul 23 2011, 08:43
Post #21





Group: Developer
Posts: 401
Joined: 14-October 01
Member No.: 289



QUOTE (IgorC @ Jul 22 2011, 20:24) *
The more listener tries to listen some particular artifacts the less chance to spot any of them. But if the listener just tries to enjoy the music itself then the artifacts appear themselves.

Depends on a listener: how well he can hear, how much concentration he has, his experience, familiarity with the sample,...
Go to the top of the page
+Quote Post
naylor83
post Jul 23 2011, 23:29
Post #22





Group: Members
Posts: 210
Joined: 19-June 05
From: Uppsala, Sweden
Member No.: 22842



Have only had time to do two samples so far. (16 & 20) (I found this listening test just a few days ago.)

Igor, which samples should I focus on if I have to chose? Any with fewer results at the moment?

This post has been edited by naylor83: Jul 23 2011, 23:30


--------------------
davidnaylor.org
Go to the top of the page
+Quote Post
IgorC
post Jul 24 2011, 10:54
Post #23





Group: Members
Posts: 1581
Joined: 3-January 05
From: ARG/RUS
Member No.: 18803



QUOTE (naylor83 @ Jul 23 2011, 19:29) *
Have only had time to do two samples so far. (16 & 20) (I found this listening test just a few days ago.)

You can send a few results already now to be sure that everything is ok.

QUOTE
Igor, which samples should I focus on if I have to chose?

All of them.
It's very unlikely to have enough results during first 10 days so it's more likely (almost sure) that test will be extended for another 10 days.


The rules are the same as from previous test with the small change of possibilities for ranking the references (3 instead of 4) because now there are less samples .

QUOTE
If the listener ranked the reference worse than 4.5 on a sample, the listener's results for that sample were discarded.
If the listener ranked the low anchor at 5.0 on a sample, the listener's results for that sample were discarded.
If the listener ranked the reference below 5.0 on more than 3 samples, all of that listener's results were discarded.

These rules aren't extremely strict in order to allow for simple human
error while still excluding careless participants (*).


A stricter procedure to exclude all ranked references risks a systemic
bias against any codec which are very good on a few samples and thus
subject to more reference confusion by causing those samples to be excluded
and weighing the test towards other samples.


(*)During the previous test some listeners have made the reasonable errors by accident (for example, have forgotten to rank the samples accidentally). But next time the listener wouldn't do such errors again.


Some detailed description of the rules from previous test. http://www.hydrogenaudio.org/forums/index....st&p=749501

This post has been edited by IgorC: Jul 24 2011, 10:58
Go to the top of the page
+Quote Post
naylor83
post Jul 24 2011, 11:35
Post #24





Group: Members
Posts: 210
Joined: 19-June 05
From: Uppsala, Sweden
Member No.: 22842



Ok, thanks.


--------------------
davidnaylor.org
Go to the top of the page
+Quote Post
jukkap
post Jul 25 2011, 18:50
Post #25





Group: Members
Posts: 26
Joined: 14-July 11
Member No.: 92296



This is the 1st listening test that I am trying to do and I found it hard !! I am impressed !! It is extremely difficult to hear differences.
Go to the top of the page
+Quote Post

3 Pages V   1 2 3 >
Closed TopicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 26th December 2014 - 17:18