IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
A new recommendation for listening tests?
Gabriel
post Nov 21 2004, 14:05
Post #1


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



I would like to suggest a little change to the recommended practices in listening tests.

Most of modern codecs are working based on the recent context. They usually have a way to adapt the bitrate to the content that take into consideration the past recent bitrate (a window). Many encoders also have a psychoacoustic model that take into consideration the previous psychoacoustic parameters/results.

Right now, when listening to samples, we usually encode a short sample with the encoder and listen to the result.
But the encoder needs some time to adapt its models (bitrate and psychoacoustic), and of course will not be able to properly adapt at the very beginning of the sample. If the sample hasn't been extracted from the full track, the encoder would have some time to adapt its models. It means that encoding a short sample is not totally representative of how this portion would be encoded in a "real" encode.

That is why I am proposing the following:
When encoding a short sample, allow a 1 second margin at the beginning and at the end of the sample so the encoder can adapt its models. This should not be 1s of silence, but a real 1s of content.
For ease of use, this could even be taken into consideration by the testing tools.


For video, the vqeg already has a similar recommendation: 1s at the beginning and 1s at the end should not be considered for tests, in order to let encoders stabilize themselves.
Go to the top of the page
+Quote Post
rjamorim
post Nov 21 2004, 14:14
Post #2


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



http://www.hydrogenaudio.org/forums/index....ndpost&p=127328 (second paragraph)

Not really the same issue, but it's somewhat similar.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
Gabriel
post Nov 21 2004, 14:53
Post #3


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



What you are pointing to would be ideal, especially for 2 pass codecs targetting bitrate.

The problem is that it is sometimes impractical for various reasons (download time, impossibility to easily cut some formats,....)
Go to the top of the page
+Quote Post
rjamorim
post Nov 21 2004, 15:04
Post #4


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (Gabriel @ Nov 21 2004, 10:53 AM)
The problem is that it is sometimes impractical for various reasons (download time, impossibility to easily cut some formats,....)
*


Yes, what draw me to give up that approach was precisely the difficulty of cutting some formats. MPC is impossible to cut, WMA too as far as I know. Atrac3 and Real Audio too, of course, since they are completely undocumented.

Also, the only available AAC cutter (BeSplit) can't cope with HE AAC and PS AAC, as far as I know.

That leaves me with MP3, Vorbis and LC AAC as cuttable formats...


And, of course, you can imagine what would happen to a tester that would cut only "cuttable" formats and just encode small samples in other formats. It's either all or nothing for fairness purposes.

This post has been edited by rjamorim: Nov 21 2004, 15:05


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
Gabriel
post Nov 21 2004, 15:16
Post #5


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



Well, for wma, atrac3 and real cutting should not usually be an issue, as in group listening tests you were distributing thos samples as decoded files.
We could imagine encoding the full track, decoding, cutting, encode the cut part as flac.
Go to the top of the page
+Quote Post
2Bdecided
post Nov 22 2004, 11:37
Post #6


ReplayGain developer


Group: Developer
Posts: 5060
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



I'm going on an unreliable memory, but isn't there a chance that even 1s extra isn't "fair" to WMA? Maybe 5 or 10 seconds is more appropriate? But them with the 30 second copyright limit, you're rather limiting the length of "useable" sample! Maybe 2 or 5 seconds would be OK.

If you want to do this, you'll either have to modify the test tools (like ABC/HR), or else include a simple .wav cutter, called from the .bat file which creates+decodes the test samples.

I think cutting encoded bitstreams is a non starter anyway - even if you could, wouldn't the frame boundaries be all over the place? Meaning the test tool has to hide the small extra/missing parts of each sample?

Cheers,
David.
Go to the top of the page
+Quote Post
Gabriel
post Nov 22 2004, 15:33
Post #7


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



Well, I suggested 1s because we have to find a reasonable value.
I do not know about wma encoders, but even 1s is not optimal to Lame, as the ATH adjustement might need more than 1s to stabilise.
But 1s is still way better than nothing and does not reduces the sample that much.

Regarding the testings themselves, I think that it would be very nice to have the tools automatically restrict the default time by 1s at both ends.
Go to the top of the page
+Quote Post
ChangFest
post Nov 22 2004, 15:48
Post #8





Group: Members
Posts: 423
Joined: 3-February 04
Member No.: 11743



QUOTE
If you want to do this, you'll either have to modify the test tools (like ABC/HR), or else include a simple .wav cutter, called from the .bat file which creates+decodes the test samples.


Why not encode as whole files and when the test is distributed have an internal .wav cutter randomly select 30 seconds from the whole song file. That way from a statistical standpoint, sample would be more random and add a validity to the test. It would also solve Gabriel's suggestion as well.
Go to the top of the page
+Quote Post
freakngoat
post Nov 23 2004, 21:13
Post #9





Group: Members
Posts: 170
Joined: 10-January 02
From: Manhattan Beach
Member No.: 979



QUOTE (ChangFest @ Nov 22 2004, 07:48 AM)
QUOTE
If you want to do this, you'll either have to modify the test tools (like ABC/HR), or else include a simple .wav cutter, called from the .bat file which creates+decodes the test samples.


Why not encode as whole files and when the test is distributed have an internal .wav cutter randomly select 30 seconds from the whole song file. That way from a statistical standpoint, sample would be more random and add a validity to the test. It would also solve Gabriel's suggestion as well.
*


I suppose this would work in certain cases, but in general, I don't think that's a good idea, because usually there is a particular artifact at a particular time in the song that people would like to point out to others, and usually there are certain parts of the song that have problems while others don't. When doing listening tests, its common to discuss things like, "I hear a weird sound at 4.3 seconds" for example. Plus 30 seconds is a very long duration for a listening test.
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 29th July 2014 - 10:09