IPB

Welcome Guest ( Log In | Register )

4 Pages V  < 1 2 3 4 >  
Reply to this topicStart new topic
AAC@128kbps listening test - CLOSED, Everyone is invited
rjamorim
post Jun 6 2003, 04:14
Post #26


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



Indeed, VBR is best for AAC.

But using VBR profiles leads to issue of fairness. The most obvious is that CBR can't theoretically compete with VBR.

Besides, sometimes the same VBR profile can end outputting 100kbps on a sample, 150 on other. Now, comparing 150 to 128 to 100 is kinda useless. "Yeah, Psytel at 150 is better than QuickTime at 128. So what? Can it compete toe-to-toe?"

It's like when people criticized you because you were comparing Wavpack at 280 versus MPC at 190 smile.gif (or something like that)

The proverbial apples vs. oranges.


now - I plan to extend this test later, comparing the best AAC encoder vs. MPC, Vorbis, MP3 and WMA. If the winner is a CBR-only codec, like SOrenson or QuickTime, we have a problem - it's unfair to compare it to MPC, that is VBR-only. So, I would end up taking this CBR winner and conduct a quick test against the highest-ranked VBR-capable encoder. And hope the VBR encodes win. >_<

But those are still just plans, and I'm working with JohnV to figure all that out.

Regards;

Roberto.

This post has been edited by rjamorim: Jun 6 2003, 04:16


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
den
post Jun 6 2003, 06:31
Post #27





Group: Members
Posts: 315
Joined: 18-February 03
From: Perth, Western Australia
Member No.: 5050



Thanks Roberto. Makes sense, level playing field and all that.

Congrats on organising the test. Good stuff!

Apples and oranges, apples and oranges... It's all coming back to me now.

Den. >_<


--------------------
Den
My blog - http://www.iinet.net.au/~den
Go to the top of the page
+Quote Post
rjamorim
post Jun 6 2003, 06:55
Post #28


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



I'm a moron

Damn. ff123 just brought to my attention that the offsets haven't been corrected, therefore some people might have taken offset-induced artifacts as encoder artifacts (what's a flaw in the test, of course)

I take full responsability in the error, and I'm very sorry.

Some people might have to take the test again, else their results can't be usable (I'll get in touch with each one of you through e-mail. Again, very sorry). Some others seem to have ignored the offset issues and concentrated on encoder artifacs, fortunately.

New packages with corrected config files are being uploaded to the server.

For those that already downloaded the packages, you don't need to download the corrected ones - just download this file and decompress it on ABC/HR's directory, overwriting the faulty configs:
http://pessoal.onda.com.br/rjamorim/newconfigs.zip

heh, maybe I was hoping too much to have the first listening test I conduced error free. Living and learning... smile.gif

Thanks a lot for your support and, once again (for the last time), my apologies.

Regards;

Roberto.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
Gecko
post Jun 6 2003, 16:07
Post #29





Group: Members
Posts: 943
Joined: 15-December 01
From: Germany
Member No.: 662



All done, results sent. I only tested one sample set without offset correction (but I didn't notice).

I would be very interested in the follow-up high quality test.
Go to the top of the page
+Quote Post
Anacondo
post Jun 6 2003, 23:35
Post #30





Group: Members
Posts: 123
Joined: 16-July 02
From: Spain
Member No.: 2610



QUOTE (rjamorim @ Jun 4 2003 - 10:53 PM)
--I'm on Linux/MacOS/something other than Windows. Can I take the test?  You can. But you won't be able to use the ABC/HR utility, that is Win32 only. Since the reference file must be hidden (That's the HR in the name), tests performed without it wouldn't be valid. So, I can't plot your scores against others. But your comments on sample quality and artifacts would still be welcome.


I've just come across with this: http://www.beryllium.net/~remco/linabx/

Then I remembered this test and the Linux issue, so maybe someone might find it useful. I haven't tried it myself, but looks good. It seems to be based on PCABX.

Cheers.
Go to the top of the page
+Quote Post
ff123
post Jun 7 2003, 11:57
Post #31


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



My results are trickling in (7 so far). How are you doing for listeners? You'll probably want at least a dozen results per sample. It would probably be much nicer if the number of listeners was double that or more.

ff123
Go to the top of the page
+Quote Post
Garf
post Jun 7 2003, 12:16
Post #32


Server Admin


Group: Admin
Posts: 4885
Joined: 24-September 01
Member No.: 13



QUOTE (Anacondo @ Jun 7 2003 - 12:35 AM)
I've just come across with this: http://www.beryllium.net/~remco/linabx/

Then I remembered this test and the Linux issue, so maybe someone might find it useful. I haven't tried it myself, but looks good. It seems to be based on PCABX.

There's a very good ABX program for Linux included with LAME.
Go to the top of the page
+Quote Post
rjamorim
post Jun 7 2003, 14:23
Post #33


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (ff123 @ Jun 7 2003 - 07:57 AM)
My results are trickling in (7 so far).  How are you doing for listeners?  You'll probably want at least a dozen results per sample.  It would probably be much nicer if the number of listeners was double that or more.

So far, 5 people answered with full result sets, and 3 with partial results.

Also, some people said they would take their time and answer next week.

Hopefully, people will have some free time this weekend to participate in the test, at least with some of the samples. smile.gif

Thanks a lot to everyone that participated so far.

Regards;

Roberto.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
ErikS
post Jun 7 2003, 16:04
Post #34





Group: Members
Posts: 757
Joined: 8-October 01
Member No.: 247



Yes, we do have more time for fun like this now... Just started today with two of them.

But what's wrong with the software? In the ABX window when having played one part many times I get a click at the end of the selection. It goes away if I change the slider slightly but comes back after a couple of times more. I would rate this bug as "Very annoying" on that scale.
Go to the top of the page
+Quote Post
ff123
post Jun 7 2003, 19:17
Post #35


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



QUOTE (ErikS @ Jun 7 2003 - 07:04 AM)
Yes, we do have more time for fun like this now... Just started today with two of them.

But what's wrong with the software? In the ABX window when having played one part many times I get a click at the end of the selection. It goes away if I change the slider slightly but comes back after a couple of times more. I would rate this bug as "Very annoying" on that scale.

EricS,

Can you PM me with the name of the song, and which codec number it is? I'll try to see if I can duplicate this on my machine. Also the particular starting and ending times you were using. In the ABX window, which side was the sample and which side was the reference?

If this clicking happens with every song, and with every codec, and no matter what portion you've selected, etc., just reply to this thread.

How many times do you have to play the part before it starts clicking?

ff123
Go to the top of the page
+Quote Post
ErikS
post Jun 8 2003, 02:54
Post #36





Group: Members
Posts: 757
Joined: 8-October 01
Member No.: 247



It happened with both samples I have tried so far. And no matter which selection I was playing. It came after around say 30-50 switches in the abx window. And once it was there and after I changed the selection slightly to make it go away temporarily, it came back sooner - maybe 10-15 switches. I had B set to the reference. Have a TB Santa Cruz if that helps any...

I'll send you a PM if I can find out something more about it.
Go to the top of the page
+Quote Post
ff123
post Jun 8 2003, 03:13
Post #37


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



QUOTE (ErikS @ Jun 7 2003 - 05:54 PM)
It happened with both samples I have tried so far. And no matter which selection I was playing. It came after around say 30-50 switches in the abx window. And once it was there and after I changed the selection slightly to make it go away temporarily, it came back sooner - maybe 10-15 switches. I had B set to the reference. Have a TB Santa Cruz if that helps any...

I'll send you a PM if I can find out something more about  it.

So you're switching between A, B, and X, and after 30 to 50 times of pressing any one of these buttons, you start to hear clicking, is that right?

Have you tried something similar using PCABX? If it happens with that program, it might not be related to ABC/HR.

I will try it out now to see what happens on my machine.

ff123
Go to the top of the page
+Quote Post
ff123
post Jun 8 2003, 03:56
Post #38


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



I can't verify the problem. I made a small selection of about 1 second and played it all the way through without pressing the stop button. Then I just hit A, B, or X, 50 or more times. Does this approximate what you're doing?

ff123
Go to the top of the page
+Quote Post
ff123
post Jun 8 2003, 06:32
Post #39


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



EricS,

One thing you might try for troubleshooting: change all the offsets to 0 (there should be two in each of the new config files which are not 0). Perhaps the clicking is related to or caused by the offset processing ABC/HR does.

Another thing to try: does the clicking happen when you switch many times in the main (abc/hr) window, or just in the abx window?

ff123
Go to the top of the page
+Quote Post
ErikS
post Jun 8 2003, 11:04
Post #40





Group: Members
Posts: 757
Joined: 8-October 01
Member No.: 247



Ok. It turned out to be an error by me. So these last few messages can safely be ignored/removed...

This post has been edited by ErikS: Jun 8 2003, 11:05
Go to the top of the page
+Quote Post
AndyMutz
post Jun 8 2003, 12:17
Post #41





Group: Members
Posts: 106
Joined: 19-May 03
From: Earth
Member No.: 6705



i'm downloading the sample files right now and i will conduct the test past the next few days smile.gif

btw, i'm not new to this forum, i'm just not posting anything wink.gif

-andy-
Go to the top of the page
+Quote Post
guruboolez
post Jun 9 2003, 01:00
Post #42





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



QUOTE (rjamorim @ Jun 7 2003 - 02:23 PM)
Hopefully, people will have some free time this weekend to participate in the test, at least with some of the samples. smile.gif

Found time today. I sent you result smile.gif , and some comments sad.gif . I hope you can take a look to them. Anyway, thank you for this test. We need to have clear ideas on the quality reached by the publicly available aac/mp4 codec.
Go to the top of the page
+Quote Post
Garf
post Jun 9 2003, 13:40
Post #43


Server Admin


Group: Admin
Posts: 4885
Joined: 24-September 01
Member No.: 13



Bump
Go to the top of the page
+Quote Post
askoff
post Jun 9 2003, 16:15
Post #44





Group: Members
Posts: 445
Joined: 23-December 02
Member No.: 4214



So when can we expect to see some results?
Go to the top of the page
+Quote Post
rjamorim
post Jun 9 2003, 16:40
Post #45


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (askoff @ Jun 9 2003 - 12:15 PM)
So when can we expect to see some results?

According to the first post, next Sunday. smile.gif


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
rjamorim
post Jun 10 2003, 02:20
Post #46


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



I'd like to thank whoever posted about the test on slashdot.

http://developers.slashdot.org/article.pl?...tid=141&tid=188


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
mpcfiend
post Jun 10 2003, 03:09
Post #47





Group: Members
Posts: 125
Joined: 13-January 02
From: Cold Lake, Alberta
Member No.: 997



QUOTE
I'd like to thank whoever posted about the test on slashdot.


On behalf of Dibrom, or the community?

Slashdottings are not fun. wink.gif


--------------------
The sky is blue.
Go to the top of the page
+Quote Post
rjamorim
post Jun 10 2003, 03:11
Post #48


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (mpcfiend @ Jun 9 2003 - 11:09 PM)
On behalf of Dibrom, or the community?

Slashdottings are not fun.  wink.gif

In behalf of the test, of course. Test quality will increase if more people participate in it.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
Dibrom
post Jun 10 2003, 03:41
Post #49


Founder


Group: Admin
Posts: 2958
Joined: 26-August 02
From: Nottingham, UK
Member No.: 1



Just a few of the gems so far in the /. comments rolleyes.gif

http://developers.slashdot.org/comments.pl...031&cid=6157272
http://developers.slashdot.org/comments.pl...031&cid=6157415
http://developers.slashdot.org/comments.pl...031&cid=6157454
http://developers.slashdot.org/comments.pl...031&cid=6157462
Go to the top of the page
+Quote Post
rjamorim
post Jun 10 2003, 03:53
Post #50


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



Great ones. But I think the winner is link #2


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post

4 Pages V  < 1 2 3 4 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 18th September 2014 - 01:12