Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Public AAC Listening Test @ ~96 kbps [July 2011] (Read 55896 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Public AAC Listening Test @ ~96 kbps [July 2011]

Today we have a chance to see the performance of well known AAC encoders like Nero, Apple and Coding Technologies (previous versions of Winamp). 
Also Fraunhofer have released their totally new AAC encoder.


The following codecs are presented in this test:
Nero 1.5.4
Apple QuickTime 7.6.9  true VBR
Apple QuickTime 7.6.9  constrained VBR
Fraunhofer (Winamp 5.62)
Coding Technologies (Winamp 5.61)
ffmpeg's AAC (low anchor)

The homepage of the test:
Public AAC Listening Test @ ~96 kbps [July 2011]


There are 20 samples.  Please, don't hurry. Take your time as there will be enough of it. It's better to do fewer samples per day and avoid a fatigue.
The test will be open during 10 days (until July 27)

As always every single result is helpful.
Have a good time testing it!


Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #2
I added the code to randomize the sample list, hope that's OK with you.

Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #3
Hope you don't mind, but I mirrored the large ZIP file with all the samples here:
http://download.nullsoft.com/listening-tes...all_samples.zip

Thank you.

I added the code to randomize the sample list, hope that's OK with you.

Yes, it's better this way

Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #4
Ok, some listeners have started to send their results.
It will be better if somebody who has already any results send them. It helps to prevent some simple errors related to ABC-HR application or any other at early stage. 
The listeners usually will get answer quickly if everything is fine.

Also it is recommended to use headphones during the test.



Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #5
Hello. I am trying to participate. I downloaded the sample pack above and used the decode all bat file. I can't get abc/hr to play anything yet.

Can someone point me to a tutorial? I see a training mode but I don't really know what I'm doing.

Edit  Ok I checked out ff123 page. I have opened the config file for sample 1 after running decode all as well as decode sample 1 bat files. there are 6 wav files in the directory and they all play in winamp OK.

The main abc/hr page is blank with no samples selected or able to be played. I have tried to load the config file over and again with no result.  Still can't play anything.

Edit 2  ABC/HR auto selected spdif out on my m-audiophile 24/96 rather than line. I can play and hear now. Still not sure what I am doing though

Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #6
I've taken the liberty to mention this test over at head-fi (link), hope that's ok IgorC.
"I hear it when I see it."

Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #7
Edit 2  ABC/HR auto selected spdif out on my m-audiophile 24/96 rather than line. I can play and hear now.

So does it mean you have managed to fix the problems?
ABC-HR application has presented the problem in past.  See the link below to fix some problems.
http://www.hydrogenaudio.org/forums/index....st&p=749194


I've taken the liberty to mention this test over at head-fi (link), hope that's ok IgorC.

Thank You, I haven't time to do it yet.

Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #8
Was a long time since I was here ... Just found out about the listening test. Will definitely join in. I'm very curious to see how the best AAC codec fares against AoTuV b6, in a future test.
davidnaylor.org

Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #9
Edit 2  ABC/HR auto selected spdif out on my m-audiophile 24/96 rather than line. I can play and hear now.

So does it mean you have managed to fix the problems?
ABC-HR application has presented the problem in past.  See the link below to fix some problems.
http://www.hydrogenaudio.org/forums/index....st&p=749194


I can hear the samples now,yes. Finding fault with them is proving much harder in some cases. I can't even hear the lowpass which is really suprising me. Much more difficult than I anticipated. Not like listening to Blade at 96kbps

Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #10
Finding fault with them is proving much harder in some cases. I can't even hear the lowpass which is really suprising me. Much more difficult than I anticipated. Not like listening to Blade at 96kbps


Don't like the sound of that 
davidnaylor.org

Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #11
My first listening tests since many years
I probably lost most of my skill, but it's very interesting for me to see if I'm still able to make a distinction between ~100 kbps encoding and an uncompressed PCM sound. I guess encoders have progressed during the last years. I remember that last time I tested LC-AAC at this bitrate, I was rather disappointed. But is was four or five years ago and I was at the apogee of my skill.

For now, I completed the five first samples. Well, it's interesting! It's nice to see that so many implementations in one test (but it makes the evaluation longer and harder).
Thank you IgorC and other people involved for organizing it!

Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #12
It's definitely not an easy test.
"I hear it when I see it."

Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #13
Question. I have used a short selection in the abx mode for a given sample. Once I use test mode,not training mode, I cant erase my results and try again in test mode. I can only use training mode even though I am checking a different time frame of the sample.

Is it supposed to be this way or can I reset back to test mode?

Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #14
Question. I have used a short selection in the abx mode for a given sample. Once I use test mode,not training mode, I cant erase my results and try again in test mode. I can only use training mode even though I am checking a different time frame of the sample.

Is it supposed to be this way or can I reset back to test mode?

It is supposed to be that way and does exactly what it should. Don't start test mode until you have heard a complete sample and choose your "worst" section before starting the test mode.

Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #15
Question. I have used a short selection in the abx mode for a given sample. Once I use test mode,not training mode, I cant erase my results and try again in test mode. I can only use training mode even though I am checking a different time frame of the sample.

Is it supposed to be this way or can I reset back to test mode?

It is supposed to be that way and does exactly what it should. Don't start test mode until you have heard a complete sample and choose your "worst" section before starting the test mode.


Thanks. I tested a few sections that I had been able to abx in practice mode and failed during the actual test. I was then able to pick another section that I was able to detect in practice mode but could not repeat the testing.

This is the first time I have used abx/hr. I will run more practice tests in the future before making a definitive test.

Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #16
OK. Mission accomplished and results sent in. Did 20 samples over 3 days. I'm on vacation so what the heck. Very interesting process but a bit grueling too.

I had  lots of trouble. Figured I would at least be able to detect a 16K lowpass regularly but I can't anymore. I was at a Rockfest for 4 days last weekend which I hope is part of the problem.

I hope my results are not rubbish because I put in alot of time.

Can't wait to see how it all turns out. Thanks to all concerned for an interesting experience.

Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #17
I'm going to try finishing this one; I never finished the first test I participated in here, and I didn't participate in the last one at all. (48kbps, which I might have actually had fun with).

One slight issue I've noticed so far: I was not able to identify the low anchor at all on Sample 3. Does this predict a problem for my results? The quality this test focuses is right on the edge of my transparent zone, so I don't want to risk tainting the results.

Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #18
I'm going to try finishing this one; I never finished the first test I participated in here, and I didn't participate in the last one at all. (48kbps, which I might have actually had fun with).

One slight issue I've noticed so far: I was not able to identify the low anchor at all on Sample 3. Does this predict a problem for my results? The quality this test focuses is right on the edge of my transparent zone, so I don't want to risk tainting the results.


You'll get it. Its not mangled like the other samples but its there. Its definitely harder than the others.

If I remember the other listening test discussions, they reject results on certain criteria, like consistently incorrectly identifying the reference or giving it scores too low to be credible. Honest results should not taint but contribute to the experiment.

PS I think I'll get my son to join HA and give this a try. He is a music major and interested in music and music compression. Maybe he could get his friends interested....

Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #19
My first listening tests since many years

I'm glad to see You back. 

As first observation,  some listeners are expecting to hear the same artifacts as in past. And of course there are new versions of codecs and artifacts are different.
The more listener tries to listen some particular artifacts the less chance to spot any of them. But if the listener just tries to enjoy the music itself then the artifacts appear themselves.

Also it's very important to mention about some practice with headphones. It's known that even good headphones can sound bad because of not optimal position on the head. It's easy to check. While the music is playing the listener should try different positions and distances (different adjustment of the rim) without set the headphones completely on the head.
It might sound crazy but it works.
More here:

http://www.meier-audio.homepage.t-online.de/tipstricks.htm
Quote
headphone position   

   It's so easy, but did you ever test for the optimal position of your headphone? With many headphones the soundstage and the balance of sound are strongly dependent on the position of the driver to our ears.

Scientific research has shown that our perception of depth is increased when the driver is placed more forward and lower with respect to the entrance of the ear channel.

Experimenting doesn't cost you anything and may improve sound considerably.

Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #20
The more listener tries to listen some particular artifacts the less chance to spot any of them. But if the listener just tries to enjoy the music itself then the artifacts appear themselves.

Depends on a listener: how well he can hear, how much concentration he has, his experience, familiarity with the sample,...

Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #21
Have only had time to do two samples so far. (16 & 20) (I found this listening test just a few days ago.)

Igor, which samples should I focus on if I have to chose? Any with fewer results at the moment?
davidnaylor.org

Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #22
Have only had time to do two samples so far. (16 & 20) (I found this listening test just a few days ago.)

You can send a few results already now to be sure that everything is ok.

Quote
Igor, which samples should I focus on if I have to chose?

All of them.
It's very unlikely to have enough results during first 10 days so it's more likely (almost sure) that test will be extended for another 10 days.


The rules are the same as from previous test with the small change of possibilities for ranking the references (3 instead of 4) because now there are less samples .

Quote
If the listener ranked the reference worse than 4.5 on a sample, the listener's results for that sample were discarded.
If the listener ranked the low anchor at 5.0 on a sample, the listener's results for that sample were discarded.
If the listener ranked the reference below 5.0 on more than 3 samples, all of that listener's results were discarded.

These rules aren't extremely strict in order to allow for simple human
error while still excluding careless participants (*).


A stricter procedure to exclude all ranked references risks a systemic
bias against any codec which are very good on a few samples and thus
subject to more reference confusion by causing those samples to be excluded
and weighing the test towards other samples.


(*)During the previous test some listeners have made the reasonable errors by accident (for example, have forgotten to rank the samples accidentally). But next time the listener wouldn't do such errors again.


Some detailed description of the rules from previous test. http://www.hydrogenaudio.org/forums/index....st&p=749501

Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #23
Ok, thanks.
davidnaylor.org

Public AAC Listening Test @ ~96 kbps [July 2011]

Reply #24
This is the 1st listening test that I am trying to do and I found it hard !! I am impressed !! It is extremely difficult to hear differences.