IPB

Welcome Guest ( Log In | Register )

Listening test using 2013-03-09 build
RobertM
post Mar 9 2013, 10:49
Post #1





Group: Members
Posts: 12
Joined: 17-February 13
Member No.: 106691



I completed a listening test against Opus files encoded with the latest build (as of 2013-09-03). This time I've actually been more thorough - ABX test results from foobar2000 are attached along with the Opus-encoded files. I also took azaqiel's advice and updated the version reported by the encoder, to prevent any confusion.

"Sample 01" from the page below was used for the test. May repeat the test later with other difficult samples.
http://people.xiph.org/~greg/opus/ha2011/


Summary:

Results were very much as expected. Opus quality has definitely improved over time and gets closer to transparency with higher bitrate.

1. 64kb/s from the above page (old opus version) and 64kb/s from the newest Opus version

There was a noticeable improvement in quality with the new Opus version

2. 64kb/s vs original

It was fairly easy to tell the difference, but still quite good quality

3. 96 kb/s vs original

Could still tell the difference but artifacts were noticably improved from the 64kb/s file

4. 128 kb/s vs original

Still can hear a very subtle artifact introduced by the codec (which appears on the note between 2.155 seconds and 2.423 seconds) but had to strain to hear it.

5. 256 kb/s vs original

Very close to transparent. I managed to tell the difference sometimes by listening very hard for the artifact. However, my ability to tell the two apart was far from perfect.

6. 500 kb/s vs original

This was transparent to me.
Attached File(s)
Attached File  sample01_RM.txt ( 3.73K ) Number of downloads: 169
Attached File  Test1.zip ( 1.17MB ) Number of downloads: 179
 
Go to the top of the page
+Quote Post
 
Start new topic
Replies
kabal4e
post Mar 12 2013, 02:54
Post #2





Group: Members
Posts: 8
Joined: 10-March 13
From: Waikato, NZ
Member No.: 107144



Thanks to RobertM I have an opus-tools build from 2013.03.09.
I mistakenly believed it had variable framesize as in opus_exp branch built in. Unfrtunately, it didn't, but after some ABX-ing I realised I couldn't distinguish the difference between the latest general and experimental builds anyway.
However, a while ago, maybe not in Opus branch of HA, a sweep sample was tested. And Opus performed very bad. I was hoping to see some improvement, but there wasn't any. Please, listen to samples attached and judge yourself.

This post has been edited by kabal4e: Mar 12 2013, 03:35
Attached File(s)
Attached File  sweep_16bit.flac ( 365.71K ) Number of downloads: 115
Attached File  sweep_16bit.opus ( 99.05K ) Number of downloads: 109
 
Go to the top of the page
+Quote Post
jmvalin
post Mar 12 2013, 03:17
Post #3


Xiph.org Speex developer


Group: Developer
Posts: 487
Joined: 21-August 02
Member No.: 3134



QUOTE (kabal4e @ Mar 11 2013, 20:54) *
Thanks to RobertM I have an opus-tools build from 2013.03.09.
I mistakenly believed it had variable framesize as in opus_exp branch built in. Unfrtunately, it didn't, but after some ABX-ing I realised I couldn't distinguish the difference between the latest general and experimental builds anyway.
However, a while ago, maybe not in Opus branch of HA, a sweep sample was tested. And Opus performed very bad. I as hoping to see some improvement, but there wasn't any. Please, listen to samples attached and judge yourself.


Wow! As much as I think sine sweep tests are stupid for codecs, there's no excuse for the behaviour you're seeing on this file with 1.1-alpha and later. That sine sweep is actually hitting a corner case in the bandwidth detection code of the encoder (see commit 7509fdb8). Thankfully, it shouldn't be too hard to fix. It's quite spectacular, but not that big a deal overall because fortunately it's highly unlikely to occur on real music.
Go to the top of the page
+Quote Post

Posts in this topic
- RobertM   Listening test using 2013-03-09 build   Mar 9 2013, 10:49
- - zerowalker   Isnīt that pretty bad, to not be able to reach tra...   Mar 10 2013, 22:44
|- - saratoga   QUOTE (zerowalker @ Mar 10 2013, 16:44) I...   Mar 10 2013, 22:56
|- - jmvalin   QUOTE (saratoga @ Mar 10 2013, 16:56) Its...   Mar 11 2013, 00:05
- - db1989   Yes, it was a sample that is known to be difficult...   Mar 10 2013, 22:55
|- - zerowalker   QUOTE (db1989 @ Mar 10 2013, 22:55) Yes, ...   Mar 10 2013, 23:00
- - IgorC   RobertM, Let me comment two things. First, one s...   Mar 10 2013, 23:53
|- - RobertM   QUOTE (IgorC @ Mar 11 2013, 09:53) Robert...   Mar 11 2013, 08:30
- - eahm   Is there a Windows compiled 2013-09-03?   Mar 11 2013, 00:26
- - wswartzendruber   Is there a place that houses updated builds of the...   Mar 11 2013, 02:51
- - RobertM   In an effort to be "fair" to the Opus en...   Mar 11 2013, 10:05
- - kabal4e   Thanks to RobertM I have an opus-tools build from ...   Mar 12 2013, 02:54
|- - jmvalin   QUOTE (kabal4e @ Mar 11 2013, 20:54) Than...   Mar 12 2013, 03:17
|- - kabal4e   QUOTE (jmvalin @ Mar 12 2013, 15:17) As m...   Mar 12 2013, 03:34
||- - jmvalin   QUOTE (kabal4e @ Mar 11 2013, 21:34) Howe...   Mar 12 2013, 03:45
|- - jmvalin   QUOTE (jmvalin @ Mar 11 2013, 21:17) Wow...   Mar 12 2013, 18:45
|- - RobertM   QUOTE (jmvalin @ Mar 13 2013, 03:45) QUOT...   Mar 12 2013, 20:14
|- - jmvalin   QUOTE (RobertM @ Mar 12 2013, 15:14) I...   Mar 12 2013, 20:44
|- - kabal4e   QUOTE (jmvalin @ Mar 13 2013, 08:44) I hi...   Mar 12 2013, 23:39
|- - db1989   QUOTE (kabal4e @ Mar 12 2013, 22:39) Foob...   Mar 13 2013, 00:13
|- - jmvalin   QUOTE (db1989 @ Mar 12 2013, 19:13) QUOTE...   Mar 13 2013, 03:04
|- - db1989   QUOTE (jmvalin @ Mar 13 2013, 02:04) If t...   Mar 13 2013, 09:35
|- - bawjaws   QUOTE (db1989 @ Mar 13 2013, 00:35) QUOTE...   Mar 14 2013, 17:54
- - db1989   Please explain how a bit-comparison provides any i...   Mar 14 2013, 18:49
- - jmvalin   QUOTE (db1989 @ Mar 14 2013, 13:49) Can a...   Mar 14 2013, 21:38
- - db1989   I definitely don’t disagree, and I can appre...   Mar 14 2013, 22:00
- - jmvalin   QUOTE (db1989 @ Mar 14 2013, 17:00) I def...   Mar 14 2013, 23:09
- - kabal4e   QUOTE (jmvalin @ Mar 15 2013, 11:09) What...   Mar 15 2013, 00:15
- - db1989   I do apologise if I misread anything or underestim...   Mar 15 2013, 01:33


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 25th December 2014 - 16:02