IPB

Welcome Guest ( Log In | Register )

13 Pages V  < 1 2 3 4 5 > »   
Reply to this topicStart new topic
Public Listening Test [2010], Discussion
Larson
post Jan 5 2010, 18:42
Post #51





Group: Members
Posts: 131
Joined: 27-March 09
Member No.: 68422



i've tested your script nao and thank you so much i have been looking for a solution for windows for such a long time. Conversion time was normal,fast and it did its job at Q127. Having also a macbook and using XLD i've compared a few files and they are identical. I confirm the "bug" that these AACs result in 0 kbps of bitrate in dbpoweramp tab,instead in Windows 7 explorer it's fine.
Go to the top of the page
+Quote Post
lvqcl
post Jan 5 2010, 18:45
Post #52





Group: Developer
Posts: 3399
Joined: 2-December 07
Member No.: 49183



QUOTE (kornchild2002)
Lame.exe and Nero.exe consume about 5% or less

Maybe your HDD is too slow or fragmented?

Added: try to set "Tools > Converter > Thread count" to 1.

This post has been edited by lvqcl: Jan 5 2010, 18:51
Go to the top of the page
+Quote Post
kornchild2002
post Jan 5 2010, 19:40
Post #53





Group: Members
Posts: 2080
Joined: 8-April 05
From: Cincinnati, OH
Member No.: 21277



I don't see a thread count option under the Converter area of the fb2k preferences. My hard drive isn't too slow or fragmented either. In fact, it is an SSD drive so fragmentation has little to no affect and the speeds are above that of a 7200 RPM drive. I do appreciate the work of nao, I don't know if that was clear or not in my first post regarding their tool. In fact, I would use it as a viable encoding option if things weren't so slow.

Edit: my source files are ALAC. I thought that might be an issue so I converted the 5 minute file to FLAC (q 5) and tried converting with foobar2000 again. Still the same results, nearly 4 minutes to convert that 5 minute file. I don't have access to my XP machine so I won't be able to test until tonight. As previously said, Windows 7 explorer is fine displaying the correct bitrate on my end (just not dBpowerAMP).

Edit 2: I went ahead and tried dBpowerAMP's CLI encoder (version R13.2) under Windows 7. Same exact results whether I encode from an ALAC, FLAC, or PCM WAV file. However, dBpowerAMP will display the encoding speed (unlike foobar2000) which hovers at around 1.3x.

This post has been edited by kornchild2002: Jan 5 2010, 19:59
Go to the top of the page
+Quote Post
IgorC
post Jan 5 2010, 20:39
Post #54





Group: Members
Posts: 1576
Joined: 3-January 05
From: ARG/RUS
Member No.: 18803



QUOTE (nao @ Jan 5 2010, 13:14) *
For Windows users, I made a tiny tool to access the QuickTime AAC encoder from the command-line.
qtaacenc

Can you specify also samplerate because all encoded files are 32 khz?

Great tool.
Go to the top of the page
+Quote Post
jido
post Jan 6 2010, 00:09
Post #55





Group: Members
Posts: 246
Joined: 10-February 04
From: London
Member No.: 11923



QUOTE (C.R.Helmrich @ Jan 3 2010, 08:34) *
QUOTE (jido)
This is Hydrogenaudio, why not using LossyWAV for the high anchor? Let's get the word out!

Since people have been almost flooding this thread with proposals for encoders to be tested in a single test, it's time for me to give my 2 cents.

The question is what we want. Of course you could put all codecs of interest (LossyWav, iTunes CVBR and TVBR, nero 1.3.3 and 1.5.3, CT/Winamp, LAME, WMA, etc.) into one test. But trust me, if you do that, you will get mostly inconclusive results, i.e. waste a lot of listening effort, due to listener overload, as I already explained.

LossyWav's objective is to be transparent. AAC at 96 kbps usually is not transparent, so its objective is to be near-transparent, i.e. "as good as possible". If you want to check whether LossyWav is transparent, do a separate ABX test against the unprocessed original (or maybe an ABX-HR test including AAC at 256 kbps or so, if you want.) Then people can focus on whether the codecs under test really are transparent, without being distracted by at the same time having to evaluate the quality of lower-bit-rate codecs.

The idea was not to test LossyWAV transparency, but to download smaller files. BTW that could also apply to the proposed 7 kHz-filtered low anchor.
Go to the top of the page
+Quote Post
IgorC
post Jan 6 2010, 00:52
Post #56





Group: Members
Posts: 1576
Joined: 3-January 05
From: ARG/RUS
Member No.: 18803



QUOTE (C.R.Helmrich @ Jan 4 2010, 10:06) *
A 7-kHz anchor (and 48-kbps LC) should be about mid-way between "very bad quality" and the quality of the 96-kbps LC encoders.

QUOTE (Sebastian Mares @ Jan 4 2010, 07:42) *
Are you sure 48 kbps LC is not too exaggerated? Do you expect any contender to be worse or as bad as 64 kbps?
LC-AAC 56 kbps or 8-khz anchor could be a good compromise. Both looks good for me.


As poll indicates
there are two widely used AAC encoder on HA.
Coding Technologies to be dropped as it's only CBR and not so popular here.

It's clear now at least for me speaking about propsal list:
1. Nero
2. Apple
3. FH
4. Divx

QUOTE (jido @ Jan 5 2010, 20:09) *
The idea was not to test LossyWAV transparency, but to download smaller files. BTW that could also apply to the proposed 7 kHz-filtered low anchor.

I think it's not good idea from point of view of credibility of final test. There will be rumors like "not 100% lossless references"
I'm completely against of lossyWAV here.

This post has been edited by IgorC: Jan 6 2010, 00:52
Go to the top of the page
+Quote Post
lvqcl
post Jan 6 2010, 01:21
Post #57





Group: Developer
Posts: 3399
Joined: 2-December 07
Member No.: 49183



QUOTE (nao @ Jan 5 2010, 19:14) *
For Windows users, I made a tiny tool to access the QuickTime AAC encoder from the command-line.
qtaacenc

Many thanks.
But, it adds samples at the beginning and decoded .m4a file has more samples than original file. Is it possible to make gapless files?
Go to the top of the page
+Quote Post
Enig123
post Jan 6 2010, 02:18
Post #58





Group: Members
Posts: 208
Joined: 11-April 02
Member No.: 1749



I might have missed something, but where to get the Fraunhofer encoder?

This post has been edited by Enig123: Jan 6 2010, 02:19
Go to the top of the page
+Quote Post
C.R.Helmrich
post Jan 6 2010, 13:23
Post #59





Group: Developer
Posts: 688
Joined: 6-December 08
From: Erlangen Germany
Member No.: 64012



QUOTE (C.R.Helmrich @ Dec 26 2009, 22:06) *
...
Agreed. But for completeness sake: Fraunhofer is currently finalizing quality tunings on their encoder which have been going on for about two years. Release is scheduled for end of January. If there is any interest, I can ask if it's possible to provide an evaluation encoder for this test.

Chris

Chris


--------------------
If I don't reply to your reply, it means I agree with you.
Go to the top of the page
+Quote Post
nao
post Jan 6 2010, 18:37
Post #60





Group: Members
Posts: 86
Joined: 16-June 06
Member No.: 31911



Updated qtaacenc.

QUOTE (kornchild2002 @ Jan 6 2010, 01:53) *
dBpowerAMP reports the bitrate as being 0kbps.

Fixed.

QUOTE (IgorC @ Jan 6 2010, 04:39) *
Can you specify also samplerate because all encoded files are 32 khz?

Now possible with the --samplerate option. "--samplerate keep" or "--samplerate 44100" meets your requirement. With the default setting, the optimum samplerate is automatically chosen according to the bitrate and quality.
Go to the top of the page
+Quote Post
tedgo
post Jan 6 2010, 19:01
Post #61





Group: Members
Posts: 1089
Joined: 16-April 04
From: Bavaria, Germany
Member No.: 13548



@nao
Thanks for your qtaacenc smile.gif.
I love it!
Maybe you better should've offered it in an own thread instead of hiding it in this "public listening test" thread.

Unfortunately the encoded files aren't gapless...
So i've to ask the same as Ivqcl: Is it possible to make gapless files?
Go to the top of the page
+Quote Post
Sylph
post Jan 6 2010, 22:27
Post #62





Group: Members
Posts: 259
Joined: 1-February 08
Member No.: 50965



QUOTE (Enig123 @ Jan 6 2010, 02:18) *
I might have missed something, but where to get the Fraunhofer encoder?


Nowhere. smile.gif

Only in MAGIX MP3 Maker and I'd have to check whether it has a VBR mode...
Go to the top of the page
+Quote Post
kornchild2002
post Jan 6 2010, 22:33
Post #63





Group: Members
Posts: 2080
Joined: 8-April 05
From: Cincinnati, OH
Member No.: 21277



QUOTE (tedgo @ Jan 6 2010, 11:01) *
Maybe you better should've offered it in an own thread instead of hiding it in this "public listening test" thread.


Maybe a mod would be nice enough to take the posts dealing with nao's command line tool out of the listening test thread and put them in a new one. That way we can continue to help test qtaacenc for nao without filling up this thread with posts that aren't really about the listening test.
Go to the top of the page
+Quote Post
rpp3po
post Jan 8 2010, 16:57
Post #64





Group: Developer
Posts: 1126
Joined: 11-February 03
From: Germany
Member No.: 4961



I don't think that a <128kbit/s listening test makes any sense nowadays. Nobody is using those bitrates and results cannot be extrapolated into regions of general transparency.

One encoder might be a worse low bitrate performer, but still have much better resilience against killer samples than another, once the bitrate passes a certain barrier. And since we have reached a point were most popular encoders are totally transparent for most music at 128-192kbit/s, we should now focus onto what encoder knows less killer samples. A listening test in its traditional form is not suited to evaluate that. The limited, initial choice of test samples will determine the outcome.

A lengthier version of this...
Go to the top of the page
+Quote Post
halb27
post Jan 8 2010, 18:23
Post #65





Group: Members
Posts: 2436
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



QUOTE (rpp3po @ Jan 8 2010, 16:57) *
... we should now focus onto what encoder knows less killer samples. A listening test in its traditional form is not suited to evaluate that. The limited, initial choice of test samples will determine the outcome. ...

Yes, the value of the outcome of such listening tests is often rated too high IMO. Especially at such a pretty low bitrate. Especially as many people just take the average score of each encoder to get a quality order for the encoders.
But: Despite this listening tests do have a value IMO, though a restricted one.

As for your approach: this would give valuable information, though restricted information as well because the universe of killer samples is not known.


--------------------
lame3100m -V1 --insane-factor 0.75
Go to the top of the page
+Quote Post
C.R.Helmrich
post Jan 8 2010, 21:19
Post #66





Group: Developer
Posts: 688
Joined: 6-December 08
From: Erlangen Germany
Member No.: 64012



QUOTE (halb27 @ Jan 8 2010, 19:23) *
QUOTE (rpp3po @ Jan 8 2010, 16:57) *
... we should now focus onto what encoder knows less killer samples. A listening test in its traditional form is not suited to evaluate that. The limited, initial choice of test samples will determine the outcome. ...

Yes, the value of the outcome of such listening tests is often rated too high IMO. Especially at such a pretty low bitrate. Especially as many people just take the average score of each encoder to get a quality order for the encoders.
But: Despite this listening tests do have a value IMO, though a restricted one.

As for your approach: this would give valuable information, though restricted information as well because the universe of killer samples is not known.

Well, an educated audio codec developer knows a lot about the universe of killer samples for his/her codec. I work as an AAC developer, and I've seen about two dozen high-bitrate killer samples until now. They all fall into very few specific categories of sounds, hence you could look for similar sounds (with high chances that they'll be killer samples as well), or even design your own. Today, one decade after version 1.0 of the encoder I work on, many initial killer samples are not killer samples any more. Of course, some still are, simply because AAC is not perfect, or because appropriate input analysis would make the encoder unacceptably slow. For example, you will never find an AAC encoder which at 128-160 kbps stereo will give you a transparent encoding of the emese sample.

So, rpp3po, I don't understand what you mean by "A listening test in its traditional form is not suited to evaluate that". I think it is. You just need to know the abovementioned categories.

Chris

This post has been edited by C.R.Helmrich: Jan 8 2010, 21:21


--------------------
If I don't reply to your reply, it means I agree with you.
Go to the top of the page
+Quote Post
rpp3po
post Jan 8 2010, 22:14
Post #67





Group: Developer
Posts: 1126
Joined: 11-February 03
From: Germany
Member No.: 4961



QUOTE (C.R.Helmrich @ Jan 8 2010, 21:19) *
So, rpp3po, I don't understand what you mean by "A listening test in its traditional form is not suited to evaluate that". I think it is. You just need to know the abovementioned categories.


With "listening test in its traditional form" in this context I mean public listening tests as the one in discussion. A public listening test like this has usually a limited test of samples. And the more 'emesesk' samples the set includes the worse AAC will look in comparison to MP3 and vice versa. So the outcome is severely influenced by the choice of samples and thus of limited use as a tool to evaluate which is best overall. The encoder with the least problematic samples in the set wins. You can intentionally exclude problematic samples from the test, but what are you going to get then? Good results for all encoders for most music. But we already know that. My point is, isn't it time to move on with testing to compare which encoders have the lowest probability of failing over broad collections of music, even if that would be harder to accomplish than with traditional 8-16 sample shootouts?

For the individual developer, who knows its encoder inside out, a "traditional" listening test with select samples can be of great value, no question.

This post has been edited by rpp3po: Jan 8 2010, 22:48
Go to the top of the page
+Quote Post
C.R.Helmrich
post Jan 8 2010, 23:03
Post #68





Group: Developer
Posts: 688
Joined: 6-December 08
From: Erlangen Germany
Member No.: 64012



QUOTE (rpp3po @ Jan 8 2010, 23:14) *
And the more 'emesesk' samples the set includes the worse AAC will look in comparison to MP3 and vice versa.

Are you sure? Just because a certain encoder has problems with "emesesk" samples doesn't mean that every AAC encoder has problems with them. Btw, to me, MP3 (LAME) doesn't do too well with such samples, either.
QUOTE
You can intentionally exclude problematic samples from the test, but what are you going to get then? Good results for all encoders for most music. But we already know that. My point is, isn't it time to move on with testing to compare which encoders have the lowest probability of failing over broad collections of music, even if that would be harder to accomplish than with traditional 8-16 sample shootouts?

Being a developer, all I care about is problematic samples smile.gif Sure, I agree, "lowest probability of failing over broad collections of music" should be an encoder's ultimate goal. But how else would you test for that than via 8-16 sample shootouts? How about asking our expert listeners (guruboolez, /mnt, IgorC, sauvage78, etc.) for the most critical items they could find in their high-bitrate tests, and put all those items in one public test (bitrate to be defined)? Wouldn't that be a good starting point?

Chris


--------------------
If I don't reply to your reply, it means I agree with you.
Go to the top of the page
+Quote Post
MichaelW
post Jan 8 2010, 23:43
Post #69





Group: Members
Posts: 631
Joined: 15-March 07
Member No.: 41501



A question from the peanut gallery.

Do different codecs have specifiably different types of killer samples?

My thought is that we know that a number of good codecs give excellent results for most music, but it might be possible, perhaps, to give specific recommendations matching codec to genre (codec A is good for harpsichord, for example, but falls over on distorted guitars, on which codec B does better).

IF this were possible, it might be more practically useful for listeners than another very tight general comparison of good codecs over "average" music.

Edit: punctuation

This post has been edited by MichaelW: Jan 8 2010, 23:44
Go to the top of the page
+Quote Post
IgorC
post Jan 12 2010, 03:17
Post #70





Group: Members
Posts: 1576
Joined: 3-January 05
From: ARG/RUS
Member No.: 18803



QUOTE (MichaelW @ Jan 8 2010, 19:43) *
A question from the peanut gallery.

Do different codecs have specifiably different types of killer samples?

Yes, there are many of this kind of sampls.

QUOTE (MichaelW @ Jan 8 2010, 19:43) *
My thought is that we know that a number of good codecs give excellent results for most music, but it might be possible, perhaps, to give specific recommendations matching codec to genre (codec A is good for harpsichord, for example, but falls over on distorted guitars, on which codec B does better).

IF this were possible, it might be more practically useful for listeners than another very tight general comparison of good codecs over "average" music.

Edit: punctuation

The problem of light or killer samples can be eliminated if randomizer will be used.
http://www.hydrogenaudio.org/forums/index....st&p=678087


I'm fine with nao's Apple TVBR encoder.

Availability of encoders
I think any AAC encoder can be tested while it's available at least in one commercial product.
No problem for Apple,Nero and CT(Winamp).
Divx encoder is still beta but every person can download it after registration on Divx home page.

Chris, what about FH encoder?
At least demo version with limited time will be fine. This way we can check that bitrate and quality settings are the same for all samples.
Go to the top of the page
+Quote Post
IgorC
post Jan 12 2010, 03:29
Post #71





Group: Members
Posts: 1576
Joined: 3-January 05
From: ARG/RUS
Member No.: 18803



QUOTE (rpp3po @ Jan 8 2010, 18:14) *
The encoder with the least problematic samples in the set wins. You can intentionally exclude problematic samples from the test, but what are you going to get then? Good results for all encoders for most music. But we already know that. My point is, isn't it time to move on with testing to compare which encoders have the lowest probability of failing over broad collections of music, even if that would be harder to accomplish than with traditional 8-16 sample shootouts?

For the individual developer, who knows its encoder inside out, a "traditional" listening test with select samples can be of great value, no question.

It's just different approach. Good one actually.

Somebody would say that testing killer samples wouldn't be representative but the same way light samples aren't representative neither.

I'm in favor to rise bitrate to 128 kbps and include more difficult samples randomly (and don't include samples which are only difficult for one specific competitor). Emese-like samples are horrible at this bitrate.

This post has been edited by IgorC: Jan 12 2010, 03:31
Go to the top of the page
+Quote Post
halb27
post Jan 12 2010, 08:44
Post #72





Group: Members
Posts: 2436
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



QUOTE (IgorC @ Jan 12 2010, 03:29) *
[...I'm in favor to rise bitrate to 128 kbps and include more difficult samples randomly (and don't include samples which are only difficult for one specific competitor). Emese-like samples are horrible at this bitrate.

I second that. Will get a result which is more relevant for practical purposes. And in case it comes out that there are encoders (may be all of them) which are perfect or near-perfect except for say artificial music this would be a fine result.
As for samples which are difficult for a specific encoders however it would be helpful to have a list of known such samples.

This post has been edited by halb27: Jan 12 2010, 08:45


--------------------
lame3100m -V1 --insane-factor 0.75
Go to the top of the page
+Quote Post
/mnt
post Jan 12 2010, 21:56
Post #73





Group: Members
Posts: 697
Joined: 22-April 06
Member No.: 29877



QUOTE (IgorC @ Jan 12 2010, 03:29) *
QUOTE (rpp3po @ Jan 8 2010, 18:14) *
The encoder with the least problematic samples in the set wins. You can intentionally exclude problematic samples from the test, but what are you going to get then? Good results for all encoders for most music. But we already know that. My point is, isn't it time to move on with testing to compare which encoders have the lowest probability of failing over broad collections of music, even if that would be harder to accomplish than with traditional 8-16 sample shootouts?

For the individual developer, who knows its encoder inside out, a "traditional" listening test with select samples can be of great value, no question.

It's just different approach. Good one actually.

Somebody would say that testing killer samples wouldn't be representative but the same way light samples aren't representative neither.

I'm in favor to rise bitrate to 128 kbps and include more difficult samples randomly (and don't include samples which are only difficult for one specific competitor). Emese-like samples are horrible at this bitrate.


That was the problem with the recent Mp3 test. Since some of the killer samples on test, such as the Final Fantasy sample, seemed to be targeted against LAME 3.97.

I would recommend using a couple of Kraftwerk tracks such as The Robots (0:20 - 0:50 or 4:10 - 4:40) and Musique Non Stop (0:10 - 0:40). Since those tracks produce very obvious artifacts at 128 with every AAC encoder.

This post has been edited by /mnt: Jan 12 2010, 21:58


--------------------
"I never thought I'd see this much candy in one mission!"
Go to the top of the page
+Quote Post
IgorC
post Jan 14 2010, 18:35
Post #74





Group: Members
Posts: 1576
Joined: 3-January 05
From: ARG/RUS
Member No.: 18803



/mnt
Thank you for data.

As poll indicates there is more interest in 128 kbps test.

Bad news most probably FH codec won't be available for public and because of this it won't get into the test.
As we moved to 128 kbps the question of high anchor is opened again.
Also CT encoder can be included into the test despite of its CBR but it's very popular (Winamp,etc.)

Proposal list (3 competitors + 1 high anchor +1 low anchor) :
1. Apple TVBR
2. Nero
3. Winner of pre-test (CT vs Divx)

High anchor (?): LAME 3.98.2 -V2
Low anchor(?): LC-AAC 64 kbps.

This post has been edited by IgorC: Jan 14 2010, 18:37
Go to the top of the page
+Quote Post
halb27
post Jan 14 2010, 21:10
Post #75





Group: Members
Posts: 2436
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



I'd welcome to have winamp's CT AAC encoder included.
As a high anchor I guess Lame -V2 can be worse for some problem samples than a good AAC encoder @128 kbps. From previous discussion I thought lossless was to be the high anchor.
As a low anchor LC-AAC @64 kbps is fine IMO.

This post has been edited by halb27: Jan 14 2010, 21:11


--------------------
lame3100m -V1 --insane-factor 0.75
Go to the top of the page
+Quote Post

13 Pages V  < 1 2 3 4 5 > » 
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 2nd October 2014 - 18:00