IPB

Welcome Guest ( Log In | Register )

4 Pages V   1 2 3 > »   
Reply to this topicStart new topic
Pre-Test discussion, Looking for opinions.
rjamorim
post Jul 16 2003, 04:07
Post #1


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



Hello.

As most of you already know, I'm planning an extension test for the AAC@128kbps test. It'll be the AAC winner (QuickTime) versus Musepack, Vorbis, Lame ABR, WMAv9 and maybe Atrac3 (more on that later). The test should start (if everything goes right) in a week (the 23rd)

There are some subjects where I'm still not sure how to proceed, so I would like to rely on the forum users' knowledge:

1 - Should I include Atrac3 (used on Sony MiniDiscs and Real Audio at mid-high bitrates) in this test? Is there anyone out there interested in this format? (besides Den ;) )

2 - I plan to use anchors in this test. For those that don't know, anchors are samples that suffer no processing from the original apart from lowpass. They are used to put the results into perspective and avoid participants rating the encoded samples too high and/or too low. The questions that arise are:
a) 2 anchors (high and low) are needed, or only one anchor is enough? (IMO, two anchors are more useful in low bitrate tests, and don't make much sense in high bitrate ones)
b) JohnV votes for one lowpass for each sample, else some samples might sound better lowpassed than their original versions. ff123 votes for one fixed lowpass for all samples, to put things into perspective. What's your personal opinion?
c) Anyone has any idea of good lowpass(es) for the test? It's a difficult choice since low lowpasses might sound too unpleasant to the fellas with good hearing and pro gear, but high lowpasses might sound too transparent to the fellas with not-so-good hearing or simple gear.

Thanks for your help.

Regards;

Roberto Amorim.

This post has been edited by rjamorim: Jul 16 2003, 04:09


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
rpop
post Jul 16 2003, 04:22
Post #2





Group: Super Moderator
Posts: 332
Joined: 20-May 03
From: Pittsburgh, USA
Member No.: 6718



I'd go along with JohnV on this one, since I've had problems with samples sounding better lowpassed. As for the frequency, I used the PCABX Training Room to test my limits, and I'd say either the 15 or the 18 kHz, as those are the ones where I actually had to focus.

Edit: I'm a newbie, so don't take my opinion as being worth very much. Just an extra push one way or the other, if the opinions are split even. I'm sure some of the regulars could make much better arguments than I did (actually, I don't think I made any arguments).

I look forward to the test! biggrin.gif

This post has been edited by rpop: Jul 16 2003, 05:45


--------------------
[url=http://noveo.net/ph34r.htm]Happiness[/url] - The agreeable sensation of contemplating the misery of others.
Go to the top of the page
+Quote Post
ff123
post Jul 16 2003, 04:30
Post #3


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



I was thinking of a 10 kHz lowpass, which should sound pretty bad to most people on most samples. The idea is to prevent a codec, which may sound pretty good if evaluated by itself, from being given unfairly poor ratings when compared with the other codecs. For example, lame might be predicted to be the loser of this test. But surely it doesn't deserve to be rated near 1.0, which might happen sometimes if there's nothing to keep things in perspective (what then about truly crappy codecs like blade?)

I understand JohnV's concern, though. Some types of music (for example solo piano) might still sound good with a 10 kHz lowpass.

Actually, another possible idea is to use blade as the anchor. I'm pretty sure this would always be rated the worst, and is likely to occupy the position near a rating of 1.0

I think only one anchor is needed (the low anchor). The hidden reference, which must be chosen from in each comparison, should be functioning as the high anchor.

ff123
Go to the top of the page
+Quote Post
ff123
post Jul 16 2003, 04:33
Post #4


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



QUOTE (rpop @ Jul 15 2003, 07:22 PM)
I'd go along with JohnV on this one, since I've had problems with samples sounding better lowpassed. As for the frequency, I used the PCABX Training Room to test my limits, and I'd say either the 15 or the 18 kHz, as those are the ones where I actually had to focus.

Either 15 kHz or 18 kHz would be useless for me as a participant (and I presume for some others as well). Those lowpasses are transparent to me in music.

ff123
Go to the top of the page
+Quote Post
boojum
post Jul 16 2003, 05:41
Post #5





Group: Members (Donating)
Posts: 819
Joined: 8-November 02
From: Astoria, OR
Member No.: 3727



I would be interested in seeing how ATRAC stacks up (alliterative pun not intended) only because I have an MD player and find the sound pretty good. I would like to see how it rates against the other compression systems. Not a big deal; only if it is not a pain in the butt. wink.gif


--------------------
Nov schmoz kapop.
Go to the top of the page
+Quote Post
Joseph
post Jul 16 2003, 05:59
Post #6





Group: Members (Donating)
Posts: 108
Joined: 7-April 03
From: Newark, CA
Member No.: 5871



Someone should really create a Listening Tests Forum. All these tests are hard to find when they get old.

The following topics could be moved to the "Listening Tests" forum:

Link 1
Link 2




<edit: added links>

This post has been edited by Joseph: Jul 16 2003, 06:12
Go to the top of the page
+Quote Post
den
post Jul 16 2003, 06:31
Post #7





Group: Members
Posts: 315
Joined: 18-February 03
From: Perth, Western Australia
Member No.: 5050



QUOTE
Should I include Atrac3 (used on Sony MiniDiscs and Real Audio at mid-high bitrates) in this test? Is there anyone out there interested in this format? (besides Den wink.gif )


and boojum! Wow, I'm not alone! wink.gif

In all seriousness. My first thought is to give ATRAC3 a miss because of the limited interest and usefulness. On the other hand, it would be interesting to see just how it does compare with others at this bitrate, not that your average Minidisc user can do anything about it. Mind you, there are the new Sony mp3/ATRAC3 CD Discmans (or is that Discmen laugh.gif ) now, so maybe it would be useful for somebody out there.

Roberto, how are you planning to make the ATRAC3 files and then present them for testing decoded into wavs? If you use the Sony software, there is no export as wav option. If you use RealProducer, my understanding is that this version of ATRAC3 is not as polished as the latest versions used in the Sony software... unsure.gif

Den.


--------------------
Den
My blog - http://www.iinet.net.au/~den
Go to the top of the page
+Quote Post
ff123
post Jul 16 2003, 06:53
Post #8


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



QUOTE (den @ Jul 15 2003, 09:31 PM)
If you use the Sony software, there is no export as wav option.

Sounds like a job for TotalRecorder. It's a pain in the ass, but it can be done.

ff123
Go to the top of the page
+Quote Post
den
post Jul 16 2003, 07:26
Post #9





Group: Members
Posts: 315
Joined: 18-February 03
From: Perth, Western Australia
Member No.: 5050



QUOTE
Sounds like a job for TotalRecorder. It's a pain in the ass, but it can be done.


Does this approach leave any opportunities for noise being added to the file due to the recording step, or is it noiseless/lossless?

I don't want to seem picky, I'm just curious. smile.gif

Den.


--------------------
Den
My blog - http://www.iinet.net.au/~den
Go to the top of the page
+Quote Post
ErikS
post Jul 16 2003, 07:36
Post #10





Group: Members
Posts: 757
Joined: 8-October 01
Member No.: 247



1. I'm also interested in how ATRAC performs compared to other alternatives for portable use like mp3 and aac.

2. Sounds like a good idea to use an anchor. And for an anchor to be useful it should be fixed, right? Anything below 12 kHz would be fine for me. Not too low though - if less than say 6 kHz I think the gap to the tested codecs will be too big to actually be able to relate their quality...
Go to the top of the page
+Quote Post
Gabriel
post Jul 16 2003, 08:14
Post #11


LAME developer


Group: Developer
Posts: 2950
Joined: 1-October 01
From: Nanterre, France
Member No.: 138



Even if atrac3 does not seems really interesting for us, I think that it should be included because Sony is trying to push it on portable devices instead of mp3.
Go to the top of the page
+Quote Post
den
post Jul 16 2003, 10:01
Post #12





Group: Members
Posts: 315
Joined: 18-February 03
From: Perth, Western Australia
Member No.: 5050



QUOTE
Sounds like a job for TotalRecorder. It's a pain in the ass, but it can be done.


It's amazing what you can find if you really, really put your mind to it. I've tried to find ways before of playing with ATRAC3 outside of the Sony software for my own educational purposes, but I always lucked out. huh.gif

I just tried again, and stumbled across this... wink.gif

ATRAC3.zip

It is an acm style ATRAC3 codec, and once installed you can encode and decode ATRAC3 in the three common bitrates using the windows audio tools. I quickly tried it with Audiograbber (it was handy), and hey presto it spewed out an ATRAC3 132 kbits file inside a wav wrapper, which I could then playback/replaygain/write to wav etc with trusty foobar. B)

I then encoded the same file with OpenMG, and tried to do a file comparison to look for differences, but the Sony version is an .omg file and there is a slight different in size ~4k in a 8.5 Mb track (@132 kbits). The Open MG generated file has tag data on the end when you check it in a hex editor. Someone else who know's what they are doing might want to look at this more closely and see if this is a reasonable way to generate some files for testing that are representative of what the Sony software churns out.

I also had a quick look at the decoded file in EAC. Has a radical haircut above 17500 Hz... :x

This should make things easier for you Roberto if you do decide to include ATRAC3 in the test. I'm also going to have a play with this further just for my own interest...It makes ABX tests with ATRAC a shed load easier.

Den.


--------------------
Den
My blog - http://www.iinet.net.au/~den
Go to the top of the page
+Quote Post
guruboolez
post Jul 16 2003, 11:01
Post #13





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



The ACM ATRAC3 encoder is totally different from SonicStage's one.
Speed, first (on my Duron 800) :

ACM = x4
SonicStage = x16

Then, quality :

LP4 with atrac3 ACM is a joke. I rarely heard something worst. SonicStage encodings are totally different.
LP2 is more problematic. SonicStage encoding are not very good : lack of details, easy to ABX on classical instrument for exemple. It sounds a bit synthetic to me.
AMC LP2 encodings are crisp, more detailed and more natural : ABXing is therefore more difficult on non-critical parts. Nevertheless, I noticed that cymbals are suffering with this codec (ACM). SonicStage, in comparison, has less problems.


In my opinion, ATRAC3 is interesting to include in this test for one reason only : Minidiscs performance. If the codec doesn't match with MD device, it's useless. In other words : sonicstage (+ Total Recorder) or nothing.
Go to the top of the page
+Quote Post
bond
post Jul 16 2003, 11:10
Post #14





Group: Members
Posts: 881
Joined: 11-October 02
Member No.: 3523



i vote for atrac3 too (there is an interest from the ripping community side too)


--------------------
I know, that I know nothing (Socrates)
Go to the top of the page
+Quote Post
den
post Jul 16 2003, 12:33
Post #15





Group: Members
Posts: 315
Joined: 18-February 03
From: Perth, Western Australia
Member No.: 5050



QUOTE
In my opinion, ATRAC3 is interesting to include in this test for one reason only : Minidiscs performance. If the codec doesn't match with MD device, it's useless. In other words : sonicstage (+ Total Recorder) or nothing.


Dammit... You really know how to let a guy down gently guru... wink.gif

Shame about the ACM codec being not representative. I really had my hopes up there for a while. Oh well, back to TotalRecorder. I thought it was a bit suss when the codec file dates were 2001. rolleyes.gif

Thanks for the clarification guru. smile.gif

Den.

PS: It would be nice to know what the differences are though...


--------------------
Den
My blog - http://www.iinet.net.au/~den
Go to the top of the page
+Quote Post
S_O
post Jul 16 2003, 12:39
Post #16





Group: Members
Posts: 296
Joined: 27-July 02
From: Germany
Member No.: 2821



I would like to see atrac3 in the test, too.
Why donīt you use helix-producer as encoder? You can download it for free.
Also it is easy to decode in GraphEdit when RealOne and Gabest RealMedia Splitter is installed:

atracfile.rm -> Real Media Splitter -> Real Audio Decoder -> WAV Dest -> out.wav

Real supports Atrac3 at these bitrates, all only at 44100Hz, Stereo:

66 kbps Lowpass: 12,4kHz (flavor 0)
94 kbps Lowpass: 15,1kHz (flavor 1)
105 kbps Lowpass: 13,7kHz (flavor 2)
132 kbps Lowpass: 16,5kHz (flavor 3)
132 kbps Lowpass: 16,5kHz (surround mode) (flavor 8)
146 kbps Lowpass: 16,5kHz (flavor 4)
146 kbps Lowpass: 16,5kHz (surround mode) (flavor 9)
176 kbps Lowpass: 19,2kHz (flavor 5)
176 kbps Lowpass: 19,2kHz (surround mode) (flavor 10)
264 kbps Lowpass: 22kHz (flavor 6)
264 kbps Lowpass: 22kHz (surround mode) (flavor 11)
352 kbps Lowpass: 22kHz (flavor 7)
352 kbps Lowpass: 22kHz (surround mode) (flavor 12)

This information is from the Codec Information Table by Real. I donīt know what the surround mode means, maybe a different (none?) channel coupling. Also I donīt know if itīs a mistake that the lowpass of 105kbps is lower than the of 94 kbps. "flavor" is the number you tell the encoder (write in the audience-file) on what mode/bitrate to encode.
Go to the top of the page
+Quote Post
Tripwire
post Jul 16 2003, 12:41
Post #17





Group: Members
Posts: 156
Joined: 28-December 02
Member No.: 4272



When you'll be trying WMA9, be sure to test both Std and Pro. The latter one is very well able to compress 44khz stereo at 128kbit, and it seems to sound better. It's also a PC playable format, neglecting it would be disappointing.
Go to the top of the page
+Quote Post
JohnV
post Jul 16 2003, 13:25
Post #18





Group: Developer
Posts: 2797
Joined: 22-September 01
Member No.: 6



QUOTE (Tripwire @ Jul 16 2003, 02:41 PM)
When you'll be trying WMA9, be sure to test both Std and Pro. The latter one is very well able to compress 44khz stereo at 128kbit, and it seems to sound better. It's also a PC playable format, neglecting it would be disappointing.

I think if both WMA versions will be tested, Atrac must be dropped. Otherwise there are way too many codecs to rate. Imo 5codecs+anchor is already pretty much maximum.

Personally I'd be ready to drop MusePack from this low bitrate test. Imo MPC lowbitrate has no practical meaning to most people or even to MusePack users outside "nice to know"-factor. I don't believe many people will be using MusePack at this low bitrate even if it does well in the test.


--------------------
Juha Laaksonheimo
Go to the top of the page
+Quote Post
kl33per
post Jul 16 2003, 13:58
Post #19


A/V Moderator


Group: Members
Posts: 841
Joined: 9-June 03
From: Brisbane, AUS
Member No.: 7078



QUOTE
Personally I'd be ready to drop MusePack from this low bitrate test. Imo MPC lowbitrate has no practical meaning to most people or even to MusePack users outside "nice to know"-factor. I don't believe many people will be using MusePack at this low bitrate even if it does well in the test.

My thoughts exactly.

I would also like to see ATRAC3 tested.

Most people around here are quality freaks so I don't see the point in testing the standard WMA9. Anybody who is after the best quality with WMA9 will use the Pro version, therefore only WMA9 Pro should be included in the test.

I would also like to see QT AAC at 128kbps up against Nero 6 -streaming.

This post has been edited by kl33per: Jul 16 2003, 14:03


--------------------
www.sessions.com.au - Sessions Entertainment
Go to the top of the page
+Quote Post
dev0
post Jul 16 2003, 14:01
Post #20





Group: Developer
Posts: 1679
Joined: 23-December 01
From: Germany
Member No.: 731



Personally I'd rather include Musepack than ATRAC3, but I can understand that people, who already own a minidisc player, might be interested in the results.

Including both versions of WMA is pointless IMO. Most users won't be using Pro and it does not yet have the hardware- and softwaresupport of the earlier StreamVersion (AFAIK the streamversion is officially called WMA2), so its inclusion would be just as pointless as Musepacks.

dev0
Go to the top of the page
+Quote Post
rjamorim
post Jul 16 2003, 16:10
Post #21


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



Hello, people. Thanks a lot for your help.

About atrac3: Well, so I guess it's decided, atrac3 will be tested.

@Den: I already expected the ACM to be worse, I had it for years, so it can only be deprecated. smile.gif

About SonicStage? Is it freely available or only comes with Sony gear? In this case, I might have to ask someone to encode-decode the samples for me.


About Musepack: I know the results might not interest much the HA community, but keep in mind they will be published on several boards I participate, at the front page of CD-rw.org and maybe even Slashdot.

Besides, it's known Musepack does well at this bitrate, but I still believe a comparision to other formats will be welcome. Specially since portable support isn't a dream anymore, it seems.


About WMA: Should I test WMA PRO (that has better quality but worse support) or WMA Standard (That has worse quality but much better support)? I know the ideal would be testing both, but IMO testing more than 6 formats, plus anchor, will be unbearable. (And these results will _really_ not interest much people, due to the bias against MS and whatnot)


About the anchor: Do people agree with ff123, about using Blade to create the anchor? (lol)


@kl33per: Nero will be tested against other formats later this year/early next year (maybe). Again, I would like to test it now, but few people would be inclined to test a multitude of samples.


About the samples: Guruboolez raised a question that there is no real classical sample (both classical samples in the 64kbps test were replaced with problem cases). So, do people agree with exchanging, say, LifeShatters (that is a bit redundant, stylewise, to BlackWater and Thear1) with Bachpsichord? Or people have another suggestion about a sample to be replaced?


Well, I guess these are enough issues raised. smile.gif

Again, thank-you very much.

Regards;

Roberto.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
djemberay
post Jul 16 2003, 16:18
Post #22





Group: Members
Posts: 5
Joined: 30-June 03
Member No.: 7488



I'd be very interested in seeing how the encoder from Nero 6 stacks up as well. It sounds as though this may not be in the spirit of this particular test, but I thought I'd throw it out there...

-raymond
Go to the top of the page
+Quote Post
guruboolez
post Jul 16 2003, 17:10
Post #23





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



QUOTE (rjamorim @ Jul 16 2003, 04:10 PM)
About the samples: Guruboolez raised a question that there is no real classical sample (both classical samples in the 64kbps test were replaced with problem cases). So, do people agree with exchanging, say, LifeShatters (that is a bit redundant, stylewise, to BlackWater and Thear1) with Bachpsichord? Or people have another suggestion about a sample to be replaced?

I suggest to replace BeautySlept - harpsichord sound, but maybe synthetic - by a real instrument recording (as Bachpsichord, or many others).

Nevertheless, I think that this test should include a most common classical sound : orchestra, choral, piano... something familiar. Two at least are on ff123 website : Macabre.wav, and fossile.wav - this last one was used on ff123's 128 kbps test. Fossile is ABXable. So why not this one, as common classical sound, and a harpsichord, as a more problematic classical sample?
Go to the top of the page
+Quote Post
ErikS
post Jul 16 2003, 17:16
Post #24





Group: Members
Posts: 757
Joined: 8-October 01
Member No.: 247



QUOTE (rjamorim @ Jul 16 2003, 04:10 PM)
About the samples: Guruboolez raised a question that there is no real classical sample (both classical samples in the 64kbps test were replaced with problem cases). So, do people agree with exchanging, say, LifeShatters (that is a bit redundant, stylewise, to BlackWater and Thear1) with Bachpsichord? Or people have another suggestion about a sample to be replaced?

You mean we have to listen to the same samples once again? I got tired of them already in the first test smile.gif Is it because it's a good test suite or because you want to be able to compare results between the tests? Anyway, I probably can't take this test, so I wonder why I even care... smile.gif
Go to the top of the page
+Quote Post
guruboolez
post Jul 16 2003, 17:24
Post #25





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



I forgot this one :
SonicStage can be downloaded ; the software is big (70 MB), and I don't know any link for the moment. Recently, many were dead.

SonicStage produce .omg file : atrac3 + encryption. I can't sent to somebody an .omg file. DRM system of Sony forbid to anyone to read an .omg file that wasn't created on his PC.
Therefore, if someone want to upload you the files, they must be in PCM format (decoded + encoded losslessly). It's difficult for me to do it : I'm limited this summer to a 56K connection.

Keep in mind that we had to cheat with SonicStage in order to play the ATRAC3 file. If the original (.wav) file is still present on the system (same path, same name), this one will be played (and captured by TotalRecorder). It's easy to do a mistake, and to conclude to the superiority of Sony's format (I saw an occurence on a MD dedicated board, where the LP2 encoded was identical to the original).
Go to the top of the page
+Quote Post

4 Pages V   1 2 3 > » 
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 31st August 2014 - 06:13