Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: New Listening Test (Read 106009 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

New Listening Test

Reply #75 –
Both 96 and 48 kbps would be great for separate reasons:
• 96 kbps seems like a logical step since we are now pretty sure that 128 kbps are already hard enough to discern;
• 48 kbps is a good streaming bitrate, and it'll be nice to test the performance of the new versions of HE-AAC (from any developer) and Vorbis.
Moreover, neither variant was tested recently.
Infrasonic Quartet + Sennheiser HD650 + Microlab Solo 2 mk3. 

New Listening Test

Reply #76 –
96 kbps is not too far from 128 where a test was done earlier this year, I think 80's a better choice.

New Listening Test

Reply #77 –
New test will be at 48 kbps, Stereo.

We should start discussing contenders and anchors now, as well as settings. No doubt, Nero HE-AAC, WMA and Vorbis. What else?
What Vorbis version do you guys recommend? Pre-Betas aren't so great to test, so should use use the latest AoTuV Beta? Also, since I didn't receive any reply from Microsoft staff regarding the new WMA codec, I guess I am forced to test the "old" WMA codec - Pro or Standard?

New Listening Test

Reply #78 –
Quote from: Ivan Dimkovic on
This approach would make sense if and only if all encoders used same bit-allocation methods, but it is not true.

So, therefore - if you compare Vorbis and WMA to, say, AAC - only measure you would have is average bit rate, total - since bit rate allocations and strategies between these algorithms differ too much.

Regarding AAC vs. AAC - I dunno, only implementation capable of doing CBR, VBR and ABR is Nero - so, you either have to cripple Nero's  encoder - or you don't have too much to test anyway.
In multiformat test everything is legal becouse test more different algorithms. Like you see iTunes CBR is better than any lame/vorbis cbr/abr/vbr. But when comparing same tehologies (AAC vs AAC) then make sub testings like CBR vs CBR, if there is not encoders which support same features like test at 48 then no winner, only show results and encoder scores but in different diagrams. So don't make test when not time. Same thing like Video testing, when testing tehologies like ASP,AVC,Theora,VP7 use max setting which encoder have, but testing ASP codecs example XviD/DivX/ND ASP XviD uses Qpel, than DivX and ND aslo use Qpel etc.

Ivan can you provide app for SBR like a dimzon says http://www.hydrogenaudio.org/forums/index....ic=42696&st=25# i wanna see how vorbis or similar codec work with sbr. Thanks

New Listening Test

Reply #79 –
Quote
Nero HE-AAC, WMA and Vorbis. What else?
And maybe mp3PRO. I think to wait WMA 10 becouse the winner without test is Nero Right. vorbis don't have SBR and wait until vorbis got real SBR or similar algorithm.

New Listening Test

Reply #80 –
Purpose of the last test was to determine the best codec for the multiformat test.  I sense no point in forcing all AAC encoders to be "CBR" in that case, neither I would find any useful purpose for CBR when there is VBR/ABR available, unless you want to stream your content over narrowband networks.

If you are interested in evaluation of CBR performance of modern AAC encoders, you could still schedule a listening test

Quote
Ivan can you provide app for SBR like a dimzon says http://www.hydrogenaudio.org/forums/index....ic=42696&st=25# i wanna see how vorbis or similar codec work with sbr. Thanks


Unfortunately I cannot. 

I must point out that SBR is backed up with a set of international patents.  Their inventors agreed to provide these patents under fair and nondiscriminatory terms for the purpose of MPEG and 3GPP/ETSI standardization, as this is one of the requirements of both bodies (ISO and 3GPP)

So, using SBR technology on anything outside of MPEG/3G AAC is a violation of their patents, and it is illegal.  Therefore I am in no legal position to provide anything regarding SBR, which is not in connection with 3GPP and/or MPEG.

Quote
And maybe mp3PRO. I think to wait WMA 10 becouse the winner without test is Nero Right. vorbis don't have SBR and wait until vorbis got real SBR or similar algorithm.


Vorbis certainly will not get SBR in any time soon - even I could recall that Monty himself pointed out few times that using of these technologies is not so effective in Vorbis.

New Listening Test

Reply #81 –
I think that if CBR is going to be tested, it should be alongside the VBR settings.  Shouldn't each codec be given a chance to work in it's best conditions (provided that is how they are used).

New Listening Test

Reply #83 –
Quote
Purpose of the last test was to determine the best codec for the multiformat test
Yes nero is winner and go to the next round he have a higest score. My point is to show in diagram that is 3GPP and CT are CBR and Nero is ABR, so Nero is more effective as ABR then, Nero is the better overall but is one step up than other AAC encoders.

Quote
So, using SBR technology on anything outside of MPEG/3G AAC is a violation of their patents, and it is illegal.
does it mean even proposed simulation (actually not embeding, just emulation in order to evaluate how good/sutable is SBR in Vorbis) is prohibited? In this case can You provide here e-mail of copyright holders, maybe we(community) can write petition to them. Dont get me wrong - I'm just want to know how good will be Vorbis+SBR in theory - nothing more


Quote
Vorbis certainly will not get SBR in any time soon - even I could recall that Monty himself pointed out few times that using of these technologies is not so effective in Vorbis.

There is no sense to work any test now, only will be if WMA 10 came out then Nero HE-AAC vs WMA 10 Pro, it is true that microsoft found some magic to beat sbr

New Listening Test

Reply #84 –
Quote
Do we really need to test mp3PRO?

My mistake you right, but i think now there is not nothing to test until WMA 10 not came out or vorbis make something like sbr. I think is better to do a 5.1 test.

New Listening Test

Reply #85 –
Vorbis is not going to make "something like SBR". AFAIK, Vorbis already has something similar to SBR. And how many more times should I write that 5.1 listening tests are impossible right now because of lack of software...

New Listening Test

Reply #86 –
Quote from: Sebastian Mares on
AFAIK, Vorbis already has something similar to SBR.


Hardly, as explained in one of the other threads.

But the idea is vaporware anyway. Current Vorbis research is going into an entirely different direction.

New Listening Test

Reply #87 –
Quote
Vorbis already has something similar to SBR
Is this work good or is developers only testing, becouse doing tests with broken encoder is not good. Why just wait until encoders come out in full versions and than make big test with multiforamat and multi bitrates, and all of these encoders is multichannel so we then can make and multichannel test but later.

New Listening Test

Reply #88 –
What Sebastian was thinking of has already been deployed for a few years in Vorbis...

New Listening Test

Reply #89 –
lame @48kbps (low anchor)
lc-aac @48kbps (another low anchor, but a good one that can show that it is better/worse than plain mp3, and would show how much sbr actually does)
he-aac " (probably with nero, since it came out slightly on top in the 48kbps he-aac test)
mp3pro (should be included as a reference since it hasn't changed)
wma 9 pro
mp3+V (just for kicks, I'm curious how bad/good it might do against the rest)
vorbis aotuv b5 (that is if b5 is released when the test gets going.. otherwise b4.51)
Vorbis-q0-lowpass99
lame3.93.1-q5-V9-k-nspsytune

New Listening Test

Reply #90 –
Please remember: more codecs = harder on listeners = harder to get useful results.

New Listening Test

Reply #91 –
I'd say MP3 at 48kbps as a low anchor. Then LC-AAC, HE-AAC, and either the aotuv pre-beta 5 or 4.51 at ~48kbps. Maybe WMA std to see how it compares at low bitrates.

New Listening Test

Reply #93 –
Quote from: Ivan Dimkovic on
As far as sample selection goes - I would personally like to avoid using the same samples, as this might lead to assumptions (probably non-justified) that some of the contenders used them to "tune" their encoders.


Quote from: Sebastian Mares on
I see one problem - it is possible that some encoders are now tuned to work well with these samples. Choosing (some) new samples can rule out the possibility of testing samples that were used for tuning.


I didn't thought about this possible issue. Too bad... 


Quote from: Sebastian Mares on
I'd say no more than 4 contenders. Do we need both a high and a low anchor?

I'd say YES without hesitation. They're both important to put results into perspective (see as good example the last listening test).

New Listening Test

Reply #94 –
*I think that once the test goal/conditions (ie 48kbps stereo) is decided, you should start a new discussion thread focusing on it.

*I think that you should have both high an low anchors

*Contenders:
#HE-AAC
#Vorbis
#Wma
#Wma pro
#AAC-LC

*anchors:
mp3?

There is a possibility that mp3 (at least Lame) would be too bad as a low anchor. In this case, you could use AAC-LC as a good low anchor.

I think that inclusing both wma and wmaPro would be interesting. It would clearly establish the quality difference between both.

I see no point in testing mp3Pro, Atrac3 as they are not really used.

New Listening Test

Reply #96 –
Because of its use in portable players?
Right now there are several portable devices able to play AAC-LC: some Nokia phones, iPods, some Expaniums,...
while I know zero portable player able to play HE-AAC (except in pure software mode)

New Listening Test

Reply #98 –
Recent Siemens phones also have HE-AAC support.

New Listening Test

Reply #99 –
Quote
So, using SBR technology on anything outside of MPEG/3G AAC is a violation of their patents, and it is illegal.
does it mean even proposed simulation (actually not embeding, just emulation in order to evaluate how good/sutable is SBR in Vorbis) is prohibited? In this case can You provide here e-mail of copyright holders, maybe we(community) can write petition to them. Dont get me wrong - I'm just want to know how good will be Vorbis+SBR in theory - nothing more

@Sebastian Mares i agree with you not only Sony Ericsson and may others support HE-AAC

@Gabriel why to work with mp3 and LC-AAC when they can be good at that bitrate. And why doing this test when we know the winner. Vorbis or WMA 9 can't beat Nero HE-AAC now. Wait for WMA 10