Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: New Listening Test (Read 106011 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

New Listening Test

Hi!

So now that Gabriel's HE-AAC test at 48 kbps is over, it's time for a new multiformat test which I would love to conduct. I think we all agree that an extension test at 128 kbps isn't going to be very productive (feedback is welcome, though) and therefore I would say that the new test should also target low bitrates (32 or 48 kbps).

If we target low bitrates, which codecs do you think we should test and why? IMO, low bitrates are optimal for streaming, so the codec decision should be based on this. If this is the case, should we therefore test CBR only?

The test is scheduled to begin late April or early May so that the HE-AAC testers can rest for at least one month.

Regards,
Sebastian

New Listening Test

Reply #1 –
My own opinion is also to test codecs in CBR (but "big" buffer is possible), however in the aac test several people disagreed that 48kbps was streamable, so CBR was finally not mandatory.

Regarding bitrate, I'd vote for 48kbps.

Codecs:
*Vorbis
*wma (pro?)
*he-aac
*mp3 ??? (not sure about this one)

Discarded codecs:
*mp3pro: totally proprietary and sparsely supported. Support is unlikely to grow
*Atrac3: really not its target bitrate, no real use at this bitrate

New Listening Test

Reply #2 –
• I'd like to see atrac3plus, but it's so boring to evaluate (Sonic Stage + acquisition [no CLI decoder nor diskwriter AFAIK] + editing and sample-exact cut) that I won't request it.
• I agree with Gabriel: mp3pro is pretty useless to test nowadays.
• WMApro is impossible (VBR 10) at this bitrate.

New Listening Test

Reply #3 –
So, first thing that we should clear up: which is the target bitrate of this test?

In order to answer this question, we should also define the goal for this test, but when comparing codecs at low bitrates, the only things that come into my mind are streaming or use in devices with limited storage capacity and where Hi-Fi is not really required (mobile phones with AAC support for example - I doubt you need Hi-Fi when sitting in the tram or when driving a car).

Since you say that 48 kbps is not streamable all the time, the only thing left is 32 kbps. If we ignore streaming and focus on portability, we can go up to 64 kbps. However, portability would mean using WMA Standard.

New Listening Test

Reply #4 –
I think it would not be too good to change the bit-rates as we did AAC 48-kbps pre-test to pick-up best 48 kbps AAC encoder.  If we do it, people might complain that wrong / suboptimal HE-AAC encoders are used, etc... etc...

I give my vote to 48 kbps because this is the bit-rate where modern state-of-the-art low bitrate codecs should have a "sweet spot"

- 32 kbps was already tested in Roberto's multiformat test
- 64 kbps was tested, too

48 kbps was never tested in multiformat conditions, and it would be very good to verify how codecs rank in this area.

Regarding compatibility - It makes sense to use compatible WMA codec, but I would also vote for inclusion of the latest WMA Pro codec, to see how this technology progressed.

So, my wishlist is:

- HE-AAC
- Ogg Vorbis
- WMA Standard
- Latest WMA Pro (preferably Vista codec that Microsoft claims is found to be better than HE-AAC)
- mp3
- ???  (maybe RealAudio ?)

New Listening Test

Reply #5 –
Hmm

It's time to find encoder for DVD backup. How about 6ch test? Proposed bitrate 128 or 160 kbps

Nero AAC (HE and LC)
CT AAC (HE and LC)
OggVorbis
Aud-X

New Listening Test

Reply #7 –
Quote
What you suggested is almost impossible because you cannot take the test using headphones.

Does it mean headphones is test requerment?

Quote
Also, 128 kbps is too high.
[a href="index.php?act=findpost&pid=373368"][{POST_SNAPBACK}][/a]

It's normal for DVD-backup puposes

New Listening Test

Reply #8 –
Quote
It's time to find encoder for DVD backup. How about 6ch test? Proposed bitrate 128 or 160 kbps
[a href="index.php?act=findpost&pid=373364"][{POST_SNAPBACK}][/a]

The term "backup" always makes me think of "perceptually transparent". Without wanting to start a religious war about this, I gotta say that I'd consider the bitrates you suggest extremely insufficient.

Quote
What you suggested is almost impossible because you cannot take the test using headphones. Also, 128 kbps is too high.
[a href="index.php?act=findpost&pid=373368"][{POST_SNAPBACK}][/a]

...128kbps too high for six-channel audio? 
A riddle is a short sword attached to the next 2000 years.

New Listening Test

Reply #9 –
Quote
Does it mean headphones is test requerment?
[a href="index.php?act=findpost&pid=373369"][{POST_SNAPBACK}][/a]


Headphones is test requirement - sort of. It's much easier to spot artifacts using headphones.

Quote
It's normal for DVD-backup puposes
[a href="index.php?act=findpost&pid=373369"][{POST_SNAPBACK}][/a]


Did you look at my last multiformat test at 128 kbps? All encoders were tied on 1st place, so testing again at 128 kbps or even 160 kbps won't be of any use. Maybe it's more difficult for encoders because they have to share the 128 kbps with 6 channels instead of 2, but still - I doubt I am going to find enough testers in order to have some rankings.

Quote
...128kbps too high for six-channel audio?  
[a href="index.php?act=findpost&pid=373373"][{POST_SNAPBACK}][/a]


Well, to be honest, I don't know for sure how bits get distributed across the channels. Is it like Dual Channel for MP3 where each channel gets 1/6 of the 128 kbps or is it like Stereo or Joint Stereo where some sort of middle channel is created and only the difference between the channels is stored in some side channels?

New Listening Test

Reply #10 –
Supposing that 48kbps is the bitrate for listening test.

As for OggVorbis, aoTuV pre-beta 5 released today.
This snapshot aim at improvement @48kbps and this is almost all final(at this bitrate).

Do we test aoTuV beta5?

New Listening Test

Reply #11 –
I am supportive for testing latest and greatest of what all competitors could offer

New Listening Test

Reply #13 –
Quote
I am supportive for testing latest and greatest of what all competitors could offer
[a href="index.php?act=findpost&pid=373383"][{POST_SNAPBACK}][/a]

Does this mean we need to pre-test OggVorbis too(like AAC),
or blindly use the latest encoder?

As you know, aoTuV has developed since beta 4.0 below -q3.
But no one confirm this AFAIK.

New Listening Test

Reply #14 –
Quote
Without wanting to start a religious war about this, I gotta say that I'd consider the bitrates you suggest extremely insufficient.
[a href="index.php?act=findpost&pid=373373"][{POST_SNAPBACK}][/a]

Actually we need ABX test to say it. That's why I propose such test.
Some peoples from Doom9 is satisfied by CT HE-AAC @ 112kbps! So I really ask for such test - nobody berore made such test but huge community (at least from doom9.org ) really need it!

New Listening Test

Reply #16 –
Quote
But as I said, it's more difficult to spot artifacts when using loudspeakers instead of headphones.
[a href="index.php?act=findpost&pid=373391"][{POST_SNAPBACK}][/a]

I understand it. Anycase it's impossible to listen 5.1 into headphones so if You doesn't spot artefacts it's mean encoder is sutable for backup, isn't it?

New Listening Test

Reply #17 –
I'd like to see RealAudio (as in cook) in this, seen as BBC use .RM as the codec for their listen again feature.

Why would we need MP3 in this? You only have to encode an MP3 at 48kbits to know it's not meant for this bitrate. If it's meant as a low-anchor, do we really need one for this low bitrate?


[span style='font-size:8pt;line-height:100%']EDIT: Spelling[/span]

New Listening Test

Reply #18 –
I'd say we make a full stop here regarding 5.1 because it is going to make things worse. I have no idea how many people have decent surround sound cards that are hooked up to decent surround speakers to allow such testing. A low-bitrate multiformat test also seems natural to me now that we have a finished HE-AAC test.

BTW, does ABC/HR even support playback of 6 channel WAVs?

Edit: We could use MP3 as low anchor. Maybe something like Shine...

New Listening Test

Reply #19 –
Quote
I am supportive for testing latest and greatest of what all competitors could offer

I am supportive for testing something that can really be used by end-users, more than testing prototypal encoders.

New Listening Test

Reply #20 –
Well, if the new version gets more "mature" until the test starts, I have no problem testing it. I have to agree that testing pre-betas or some nightly builds isn't very productive and can be pretty dangerous (look at the Nero problem in my last multi-format test).

New Listening Test

Reply #21 –
96kbps could be also interesting ...

New Listening Test

Reply #22 –
Quote
96kbps could be also interesting ...
[a href="index.php?act=findpost&pid=373482"][{POST_SNAPBACK}][/a]


Without wanting to start a revolt here, I agree :-) But so is 48 kbps, I guess.
davidnaylor.org

New Listening Test

Reply #23 –
I'd also vote for ~90 kbps (leaving headspace for 112kbps ISDN and DSL, while being a good enough bitrate, I think.)

for testing?  The Classics : nero LC-AAC HE-AACv1, vorbis (prebeta5), LAME (latest alpha? Recommended version?) Windows media from wmp10 (std) and itunes mp3, itunes aac, wmp10 mp3.