IPB

Welcome Guest ( Log In | Register )

9 Pages V  « < 4 5 6 7 8 > »   
Closed TopicStart new topic
New Listening Test, Pre-Discussion Thread
Dzamburu
post Apr 12 2006, 21:44
Post #126





Group: Banned
Posts: 56
Joined: 9-April 06
Member No.: 29317



QUOTE (Sebastian Mares)
And Dzamburu, there is no such thing as Vorbis 2.
I mean on this Aotuv vorbis, is amazing @ 48, did you try WMA 10, becouse i can force MP10 to 48kbs i have lastest vista 5342 or something like that.
Go to the top of the page
+Quote Post
IgorC
post Apr 12 2006, 21:45
Post #127





Group: Members
Posts: 1532
Joined: 3-January 05
From: ARG/RUS
Member No.: 18803



If there will be 2 samples. Very bad quality 1 point and other even worse 0.5 point but min. mark is 1.

So the mark scale 0-100 will be more appropriate.
Go to the top of the page
+Quote Post
Sebastian Mares
post Apr 13 2006, 09:43
Post #128





Group: Members
Posts: 3629
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



QUOTE (IgorC @ Apr 12 2006, 10:45 PM) *
If there will be 2 samples. Very bad quality 1 point and other even worse 0.5 point but min. mark is 1.

So the mark scale 0-100 will be more appropriate.


Well, isn't this the reason why we have a low anchor? The low anchor gets the worst score and the other samples are ranked according to the low anchor. So, if low anchor gets 1, the others get 1.3 for example. If one sample sounds even worse than the low anchor, the low anchor can get 1.5 and the sample 1.


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
stephanV
post Apr 13 2006, 11:00
Post #129





Group: Members
Posts: 394
Joined: 6-May 04
Member No.: 13932



I agree with Sebastian that a scale of 0-100 is not really justifiable and seems too granular. When a person ranks a sample as 73, he should be able to explain why he didn't rank it 72 or 74 instead. I can only speak for myself of course, but I certainly wouldn't be able to do this.


--------------------
"We cannot win against obsession. They care, we don't. They win."
Go to the top of the page
+Quote Post
Gabriel
post Apr 13 2006, 11:21
Post #130


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



I think that a 0-100 range would be appropriate if we expect big impairements.
In this context you are likely to only use the lower part of the scale for most contenders.
We will perhaps encounter 1 good contender, that would be ranked at 70% of the scale, and several "less good" ones that would be ranked between 25 and 50% of the scale, the low anchor beein ranked at about 10%.
In such case, you only have about 25% of dynamic range to rank most contenders. To me, a 0-100 scale makes sense.

0-100 scale and tools:
Right now we have a 40 points scale (1-5 with a .1 resolution). The 0-100 scale is more a matter of visual presentation to the user than a real granularity matter.
I think that a modified testing tool could display a 0-100 scale to the user, but output results in the 1-5 range in results file, allowing to keep tools modifications to a minimum.
Go to the top of the page
+Quote Post
IgorC
post Apr 13 2006, 14:45
Post #131





Group: Members
Posts: 1532
Joined: 3-January 05
From: ARG/RUS
Member No.: 18803



QUOTE (stephanV @ Apr 13 2006, 02:00 AM) *
When a person ranks a sample as 73, he should be able to explain why he didn't rank it 72 or 74 instead.


The same for mark scale 1-5 . Why to choose 3.7 and not 3.6 or 3.8? Sometimes I wanted to put somethig like 3.75 .

The idea of previous post is 100% scale has higher "definition" (range) . Personaly I will feel myself more comfortable/free to mark samples this way.

Sebastian Mares hm. I didn't think about low anchor at that moment. But 100% scale has error 0.01 , and 1-5 40 steps - error 0.025.
From previous 48 he-aac test CT and Nero were on par. With error 0.025 CT aac+ would have 3.18*102.5% = 3.25 , with error 0.01 - 3.18*101% = 3.21

This post has been edited by IgorC: Apr 13 2006, 14:54
Go to the top of the page
+Quote Post
schnofler
post Apr 14 2006, 02:02
Post #132


Java ABC/HR developer


Group: Developer
Posts: 175
Joined: 17-September 03
Member No.: 8879



OK, just so the discussion is not needlessly simplified by lack of suitable software, I just put support for custom rating scales into ABC/HR for Java: Binaries, Sources
Go to the top of the page
+Quote Post
Shade[ST]
post Apr 14 2006, 02:43
Post #133





Group: Members
Posts: 1189
Joined: 19-May 05
From: Montreal, Canada
Member No.: 22144



You do quite the kick-azz job, schnofler. Way to go!
Go to the top of the page
+Quote Post
Gabriel
post Apr 14 2006, 06:46
Post #134


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



QUOTE (schnofler @ Apr 14 2006, 03:02 AM) *
OK, just so the discussion is not needlessly simplified by lack of suitable software, I just put support for custom rating scales into ABC/HR for Java: Binaries, Sources

Thank you very much.
Go to the top of the page
+Quote Post
Sebastian Mares
post Apr 14 2006, 08:42
Post #135





Group: Members
Posts: 3629
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



Any chance you could add an option for incrementing 2000 to the calculated offsets? Currently, you have to calculate the offsets and then go through each sample and manually type in the offset.


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
schnofler
post Apr 14 2006, 14:19
Post #136


Java ABC/HR developer


Group: Developer
Posts: 175
Joined: 17-September 03
Member No.: 8879



QUOTE (Sebastian Mares @ Apr 13 2006, 11:42 PM) *
Any chance you could add an option for incrementing 2000 to the calculated offsets? Currently, you have to calculate the offsets and then go through each sample and manually type in the offset.

Here you go: Binaries, Sources
Go to the top of the page
+Quote Post
Sebastian Mares
post Apr 14 2006, 14:58
Post #137





Group: Members
Posts: 3629
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



Thanks. Do I have to enter the additional offset before calculating the offsets, after, or doesn't it even matter?

This post has been edited by Sebastian Mares: Apr 14 2006, 14:59


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
schnofler
post Apr 14 2006, 15:07
Post #138


Java ABC/HR developer


Group: Developer
Posts: 175
Joined: 17-September 03
Member No.: 8879



QUOTE (Sebastian Mares @ Apr 14 2006, 05:58 AM) *
Thanks. Do I have to enter the additional offset before calculating the offsets, after, or doesn't it even matter?

Doesn't matter. The additional offset is simply added to the individual sample offsets.
Go to the top of the page
+Quote Post
Sebastian Mares
post Apr 15 2006, 17:12
Post #139





Group: Members
Posts: 3629
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



OK, thanks schnofler!

As for the new WMA codec, after several e-mails, and newgroups / forums posts I was told that I have to check with the licensing department of Microsoft. However, I doubt that they are going to give me a permission in time (or at all). There are two things that are possible now:
  1. Wait until the public beta test of Windows Vista begins with the hope that Microsoft will also change the EULA allowing results to be posted
  2. Test the old WMA codec

Personally, I would go with 2. However, I am not sure if the test is going to be so interesting.

The public beta test is scheduled to start in May.


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
Dzamburu
post Apr 15 2006, 18:15
Post #140





Group: Banned
Posts: 56
Joined: 9-April 06
Member No.: 29317



Damn, I don't know becouse wma9 don't belong in group of low bitrate encoders and have terrible quality at 48kbs. If you make second decision then let's test begin, personally, i would to see that new Microsoft beast
Go to the top of the page
+Quote Post
SwiftBiscuit
post Apr 16 2006, 08:08
Post #141





Group: Members (Donating)
Posts: 26
Joined: 31-July 02
From: Miami, Fl
Member No.: 2877



If you intend for this to be the last 48kbps test for a while then I'd say wait for the public beta.
Go to the top of the page
+Quote Post
Sebastian Mares
post Apr 16 2006, 08:20
Post #142





Group: Members
Posts: 3629
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



Well, I don't know what to intend. Unless the big players HE-AAC and Vorbis don't face major improvements, it's going to be the last at 48 kbps for at least a year.


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
Sebastian Mares
post Apr 21 2006, 08:19
Post #143





Group: Members
Posts: 3629
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



Small update: Microsoft is discussing my request internally and someone will be back in contact with me soon.


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
IgorC
post Apr 22 2006, 13:58
Post #144





Group: Members
Posts: 1532
Joined: 3-January 05
From: ARG/RUS
Member No.: 18803



QUOTE (Sebastian Mares @ Apr 20 2006, 11:19 PM) *
Small update: Microsoft is discussing my request internally and someone will be back in contact with me soon.


was MS going to present open beta test for WMA 10 already in may?
Go to the top of the page
+Quote Post
Sebastian Mares
post Apr 22 2006, 14:09
Post #145





Group: Members
Posts: 3629
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



Nobody ever mentioned an open beta test of WMA 10. The only open beta test that is planned for May is for Windows Vista, but I am not sure if they are also going to change the EULA to allow publishing of benchmark results and that sort of things.


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
SirGrey
post Apr 22 2006, 15:16
Post #146





Group: Members
Posts: 241
Joined: 8-February 04
Member No.: 11863



Threre is a small chance that Microsoft will provide you with a stand-alone app, then Vista's EULA does not matter.
Let's wait until MS decision, then discuss tongue.gif
Go to the top of the page
+Quote Post
Sebastian Mares
post Apr 22 2006, 16:05
Post #147





Group: Members
Posts: 3629
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



Yeah, I hope so too.


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
Supacon
post Apr 25 2006, 18:24
Post #148





Group: Members (Donating)
Posts: 543
Joined: 19-March 04
From: Alberta, Canada
Member No.: 12841



Today's new major update to Nero's AAC codec should be relevant to this listening test.

Also, Sebastian, any love from Microsoft?
Go to the top of the page
+Quote Post
Sebastian Mares
post Apr 25 2006, 18:32
Post #149





Group: Members
Posts: 3629
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



Nope.

I also have problems getting the new April build to run under VMware - it keeps crashing with a BSOD. I have to squeeze my Ubuntu partition a bit.


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
digitalradiotech
post May 2 2006, 14:14
Post #150





Group: Members
Posts: 50
Joined: 17-April 04
Member No.: 13565



QUOTE (Ivan Dimkovic @ Apr 12 2006, 07:55 AM) *
QUOTE

I agree with you, range 0-100 is silly.


It is the range standardized in ITU for the ITU-R BS.1534 recommendation - testing of audio signals with large impairment, such as codecs at 48 kbps.



I don't see the point in the MUSHRA test, to be honest. It seems to be more of an excuse to allow mediocre-performing codec/bit rate combinations to get high scores. If a codec/bit rate combination does cause a large impairment, then I think it should be scored as a large impairment on the original BS.1116 test. At the end of the day, reduced bit rate audio coding should be about trying to get as close to the original as possible, IMO, not letting codecs off just because they're using a low bit rate.

And if you change the scale from 0-5 to 0-100 you run the risk of people confusing results from a BS.1116 test with a MUSHRA test, which has a totally different scale:

MUSHRA (EBU Tech Review article about MUSHRA):

80 - 100 Excellent
60 - 80 Good
40 - 60 Fair
20 - 40 Poor
0 - 20 Bad

which is not the same as the BS.1116 scale. And according to the EBU article it says:

"MUSHRA uses an unprocessed original programme material of full bandwidth as the reference signal. In addition, at least one additional signal (anchor) being a low-pass filtered version of the unprocessed signal should be used. The bandwidth of this additional signal should be 3.5 kHz."

So it's not really surprising that you get very high scores for mediocre performance when you've got such an incredibly low quality anchor.

For example, CT 48kbps HE AAC scored 88 in a MUSHRA test (if I remember correctly), whereas in the recent test on HA it only got somewhere in the region of 3.5 out of 5, and such a big difference has got to be down to the difference in the testing process.

I think you need to stick with the 0-5 scale using the BS.1116 test, as you've always used, or if you're going to change the scale to 0-100 you should fully implement the MUSHRA testing methodology, but not mix the two tests together.

This post has been edited by digitalradiotech: May 2 2006, 15:03
Go to the top of the page
+Quote Post

9 Pages V  « < 4 5 6 7 8 > » 
Closed TopicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 23rd July 2014 - 16:31