IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
"near transparent" listening test, based on the 64 kpbs test
elmar3rd
post Aug 24 2003, 10:36
Post #1





Group: Members
Posts: 144
Joined: 5-May 02
Member No.: 1974



Are the samples and the ini/batch-files for ABC/HR tool in the 64 kbps-test a good starting point for a quick listening test in the "near transparent" range, too?

This test schould not work out the best "near transparent" codec/setting, but how the common and easy available codecs perform in comparison.

IMHO it's useful and interesting to examine thesis like "Regarding hardware compatibility, LAME APS ist still a good choice".

Codecs/Settings for example:
- OGG Vorbis q6 and/or Ogg Vorbis GT3 q5
- LAME APS
- MMJ fastenc 192
- Musepack quality 5
- Nero ACC and/or QT AAC
- WMA9

This post has been edited by elmar3rd: Aug 24 2003, 10:41
Go to the top of the page
+Quote Post
tigre
post Aug 24 2003, 10:51
Post #2


Moderator


Group: Members
Posts: 1434
Joined: 26-November 02
Member No.: 3890



This is a nice idea but it won't work IMO for many reasons:

- At "near transparent" settings you need a lot of training, patience, time etc. to perform a test like this. I doubt that there'd be enough ppl participating and providing useful results

- All rating would be somthing close to 5, so after statistical evaluation there's probably nothing useful left as codecs are simply too close.

- With most samples of the 64kbps test most codecs won't fail, so probably > 90% can't be ABXed and get a 5 rating.

- Choosing known problem samples for this kind of test isn't an option either IMO as most problem samples are specific for one codec while others perform much better or have no problems at all.

This post has been edited by tigre: Aug 24 2003, 10:52


--------------------
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello
Go to the top of the page
+Quote Post
music_man_mpc
post Aug 24 2003, 17:22
Post #3





Group: Members (Donating)
Posts: 707
Joined: 20-July 03
From: Canada
Member No.: 7895



QUOTE (elmar3rd @ Aug 24 2003, 01:36 AM)
- MMJ fastenc 192

What the hell is that?? and how is it going to compete if it is 192 ABR or CBR, which is how it seems to be presented.


--------------------
gentoo ~amd64 + layman | ncmpcpp/mpd | wavpack + vorbis + lame
Go to the top of the page
+Quote Post
bubka
post Aug 24 2003, 17:40
Post #4





Group: Members
Posts: 239
Joined: 21-July 02
Member No.: 2692



this is the test i would be really interested in, something we all use


--------------------
Chaintech AV-710
Go to the top of the page
+Quote Post
DeeZi
post Aug 24 2003, 17:50
Post #5





Group: Members
Posts: 35
Joined: 17-February 03
Member No.: 5044



I think this test would be very interresting, especially the aac- and wma 9-part.
Go to the top of the page
+Quote Post
phong
post Aug 24 2003, 18:47
Post #6





Group: Members
Posts: 346
Joined: 7-July 03
From: 15 & Ryan
Member No.: 7619



The test would have to be performed with problem samples. Normal samples are transparent even to golden ears at those settings and most problem samples tend to be difficult at best. Further, while some things have a standard setting for "transparent" (mpc --standard, lame --alt-preset standard), other codecs do not. Should we consider oggenc -q5? I've found that oggenc -q6 is more often than not, bigger than lame -aps, even though vorbis is supposed to be better than mp3.

Also, if known problem samples are used, the test will be certainly skewed against codecs that are used around here most often. In addition, the results won't be that meaningful because each codec is at a different bitrate. Lets say, hypothetically (I doubt this would occur), that oggenc -q6 beats mppenc --standard. That's meaningless because the vorbis files are much bigger. There's no way to know, from these results, which would perform better at the same bitrate.

I think a test at 160kbps vbr (or something similar) would be better. You'd still have a problem of finding suitable samples and suitable listeners, and the skew that results from having only problem samples.


--------------------
I am *expanding!* It is so much *squishy* to *smell* you! *Campers* are the best! I have *anticipation* and then what? Better parties in *the middle* for sure.
http://www.phong.org/
Go to the top of the page
+Quote Post
music_man_mpc
post Aug 24 2003, 19:55
Post #7





Group: Members (Donating)
Posts: 707
Joined: 20-July 03
From: Canada
Member No.: 7895



QUOTE (phong @ Aug 24 2003, 09:47 AM)
The test would have to be performed with problem samples. Normal samples are transparent even to golden ears at those settings . . .

Thats not what this thread: http://www.hydrogenaudio.org/forums/index....=0&#entry122726 would seem to suggest.

edit: not that I'm saying this test is workable. guruboolez, geko and trnsz would likely be the only ones with useful input . . . . but you never know I'm thinking about trying to train my ears, just to see if I have what it takes.

This post has been edited by music_man_mpc: Aug 24 2003, 20:15


--------------------
gentoo ~amd64 + layman | ncmpcpp/mpd | wavpack + vorbis + lame
Go to the top of the page
+Quote Post
tigre
post Aug 24 2003, 20:52
Post #8


Moderator


Group: Members
Posts: 1434
Joined: 26-November 02
Member No.: 3890



QUOTE (music_man_mpc @ Aug 24 2003, 10:55 AM)
I'm thinking about trying to train my ears, just to see if I have what it takes.

Think twice before you start! wink.gif I have done too much testing/ABXing during the last 2 weeks (trying to find good samples for 64kbps test and http://www.soundexpert.info/). Now some of my favourite downloaded mp3s (some of them 224kbps) sound annoying to me and I had to buy the CDs (hard to find). ph34r.gif

This post has been edited by tigre: Aug 24 2003, 20:53


--------------------
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello
Go to the top of the page
+Quote Post
mdmuir
post Aug 24 2003, 21:26
Post #9





Group: Members (Donating)
Posts: 195
Joined: 14-May 03
From: Omaha, Nebraska USA
Member No.: 6617



As a side note I think a lot of how one reads the results of all these abx tests is still irrecoverably linked to what one perceives the usefulness of the results.

To clarify-when Roberto was asking for samples for his 64 kbps test, I decided to do some abx tests of four of the codecs involved (vorbis, wma9pro, HE AAC , mp3pro) with just music I like, not problem samples. My results showed me that wma9pro was the least objectionable to MY ears, but not by much. And I would never choose 64 kbps encoding for my personal needs.

So, when the results of Roberto's 64 kbps tests are in-my reading of it will be not
"which codec is the best for 64kbps encoding" but rather "which codec is the least crappy at 64 kbps". This is my admitted built in bias against using anything with 64kbps encoding. And such bias would immediately preclude me from being a tester in the first place.

This is why I am so glad I chose to store all my music on computer losslessly-even though 99% of the time I could never abx it against 'transparent" lossy encoder files, just knowing they are just a compressed version of the wav means I don't have to think or worry about it-just re-encode for whatever situation demands something different-ie aps for an mp3 portable cd player from the lossless files.

This post has been edited by mdmuir: Aug 24 2003, 21:28


--------------------
you will make mp3's for compatibility reasons.
Go to the top of the page
+Quote Post
ScorLibran
post Aug 24 2003, 22:05
Post #10





Group: Banned
Posts: 769
Joined: 1-July 03
Member No.: 7495



QUOTE (tigre @ Aug 24 2003, 03:52 PM)
QUOTE (music_man_mpc @ Aug 24 2003, 10:55 AM)
I'm thinking about trying to train my ears, just to see if I have what it takes.

Think twice before you start! wink.gif I have done too much testing/ABXing during the last 2 weeks (trying to find good samples for 64kbps test and http://www.soundexpert.info/). Now some of my favourite downloaded mp3s (some of them 224kbps) sound annoying to me and I had to buy the CDs (hard to find). ph34r.gif

This is exactly the way I feel, too! I just (a few days ago) started encoding my collection into Ogg Vorbis Post 1.0 CVS -q 4.25 because I did some ABX tests on some of my music (about 20 samples), and couldn't differentiate it even once from source PCM WAV files. As a bonus, I can now fit my entire 357 album (402 CD) collection onto a 20GB HDD. Before, I was using FLAC, and could only fit about 60 CDs in the same space. I'm not sure whether to feel proud or embarrassed to have found a 136kbps nominal bitrate as my preferred format. And actually, though 136kbps is the "stated" nominal bitrate, I'm getting more like a 126-127kbps average overall (about 0.93 MB/min measured after having encoded about 100 CDs).

I'm astounded that I could be truly happy with this -q level, and I'm afraid if I start "training" my ears, then I'll start hearing artifacts in my music that I didn't hear before. The primary reason I didn't participate in Roberto's 128 kbps listening tests is that I had only marginal-quality equipment...Yamaha AC-XG sound adapter, Sony MDR-ED228 earbuds. Now that I have better hardware (Echo Indigo, Xin headphone amp, Grado SR-80s), I'm still reluctant to participate in any arranged listening test. To feel of any value in doing so, I'd first want to spend time training my ears to hear different artifact types. After I was done, I could probably participate in a valuable way in a formal listening test, but after it was all done, I'd have to encode all my music again into FLAC! Why couldn't I just step up the -q level a notch or two? Because one of my primary listening platforms (car) can't currently decode anything higher than -q 4.25 Vorbis (1.0), so to go any higher, it's LAME MP3 --aps...gappy, so that's out. Next step is FLAC. ohmy.gif (Doesn't speak MPC.)

Anyway, it's great in a way that a <136kbps (nominal) encoding level is transparent to my ears for the bulk of my music. It's an odd, different kind of Warm Fuzzy Feeling, plus not needing to upgrade my 20 GB discs quite so soon is nice too. At this point, I don't want to start messing with my level of audio perception...it's right where I want it! smile.gif
Go to the top of the page
+Quote Post
phong
post Aug 25 2003, 05:40
Post #11





Group: Members
Posts: 346
Joined: 7-July 03
From: 15 & Ryan
Member No.: 7619



QUOTE
Thats not what this thread: http://www.hydrogenaudio.org/forums/index....=0&#entry122726 would seem to suggest.

Well, that thread was really about one problem sample that Guruboolez dug up, and about a general class of problem samples for MPC. I don't think it reflected on general transparency of MPC at --standard. As I mentioned in that thread, I was able to find a problem sample (much easier to ABX than Guruboolez's sample) for lame --alt-preset insane and oggenc -q6 on my first try (sample here.) I was not able to do the same (so far) for MPC.

Also, there's the problem mentioned in that thread and again here - training yourself to hear artifacts is a curse in a way. Participating in the 128kbps test was my training, and I almost wish I could go back. Ignorance really is bliss. There's a track on an album I own (Fibber Island off of They Might Be Giants - No!) that was previously distributed as a 128kbps mp3 on eMusic. When I was listening to my CDs looking for samples to volunteer for the 64kbps test, I heard what sounded like significant pre-echo in some guitar at the beginning. This is a real pressed CD I bought from the store. I'm fairly convinced now that they pressed the mp3ed version onto the CD. Here's a sample. I hope somebody can tell me that I'm crazy. Edit: it turns out (to my relief) that I'm crazy.

It used to be that 128kbps mp3s bothered me, but now they're really offensive to my ears. I can't even listen to them and wonder how I ever could. I heard a friend's XM radio the other day and gagged. It sounded worse than a FM station does (at least, when you've got good reception). It was muffled (sounded like an 11khz lowpass or lower) and there were significant artifacts. Turns out, all the stations are 64kbps or sometimes LOWER! I can't believe how people rave about the quality (stuff like "It's way better than 160k mp3s"). Sure, there weren't commercials, and the reception was always good, but I can't believe people pay like 200 bucks for the reciever and then 10 bucks a month.

This post has been edited by phong: Aug 25 2003, 15:18


--------------------
I am *expanding!* It is so much *squishy* to *smell* you! *Campers* are the best! I have *anticipation* and then what? Better parties in *the middle* for sure.
http://www.phong.org/
Go to the top of the page
+Quote Post
rjamorim
post Aug 25 2003, 05:53
Post #12


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (phong @ Aug 25 2003, 01:40 AM)
Well, that thread was really about one problem sample that Guruboolez dug up, and about a general class of problem samples for MPC.

Besides, you must take in consideration that guruboolez's hearing is supernatural. :B

I'm actually afraid he's one of those Volcano guys, with weird pointed ears and all. That's why he can pick up every damn artifact.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
den
post Aug 25 2003, 05:57
Post #13





Group: Members
Posts: 315
Joined: 18-February 03
From: Perth, Western Australia
Member No.: 5050



QUOTE
Also, there's the problem mentioned in that thread and again here - training yourself to hear artifacts is a curse in a way.


Whenever these types of discussions come up, I am reminded of the following -

QUOTE
Neo: What truth?
Morpheus: That you are a slave, Neo. Like everyone else you were born into bondage, born into a prison that you cannot smell or taste or touch. A prison for your mind.... Unfortunately, no one can be told what the Matrix is. You have to see it for yourself. This is your last chance. After this there is no turning back. You take the blue pill, the story ends, you wake up in your bed and believe whatever you want to believe. You take the red pill, you stay in Wonderland, and I show you how deep the rabbit hole goes.... Remember, all I'm offering is the truth, nothing more.... Follow me....


and then

QUOTE
Cypher: You know, I know what you're thinking, because right now I'm thinking the same thing. Actually, I've been thinking it ever since I got here. Why, oh why didn't I take the blue pill?...


Why, oh why did I start reading HA, and started noticing all sorts of annoying artifacts in my tunes... laugh.gif


--------------------
Den
My blog - http://www.iinet.net.au/~den
Go to the top of the page
+Quote Post
ScorLibran
post Aug 25 2003, 07:50
Post #14





Group: Banned
Posts: 769
Joined: 1-July 03
Member No.: 7495



QUOTE (den @ Aug 25 2003, 12:57 AM)
QUOTE
Cypher: You know, I know what you're thinking, because right now I'm thinking the same thing. Actually, I've been thinking it ever since I got here. Why, oh why didn't I take the blue pill?...


Why, oh why did I start reading HA, and started noticing all sorts of annoying artifacts in my tunes... laugh.gif

la - la - la - la - la - :fingers in my ears: - la - la - la - la - la - la - la - la - la - la - :not listening: - la - la - la - la - la - :happy in my cell: - la - la - la - la - la - la - la - la - la - :the world's too big out there: - la - la - la - la - la - la - la - la - la - la - :but I don't wanna hear any artifacts: - la - la - la - la - :I'm hearing it the way the music's supposed to sound: - la - la - la - la - la - la - la - :I'm happy with it just the way it is: - la - la - la - la - :5000 songs on 20 GB: - la - la - la - la - la - la - :5000 songs on 20 GB: - la - la - la - la ...

....... must .......... not ................. hear ........................................ artifacts .....

..... too busy looking at the woman in the red dress ....
Go to the top of the page
+Quote Post
Gecko
post Aug 25 2003, 18:59
Post #15





Group: Members
Posts: 943
Joined: 15-December 01
From: Germany
Member No.: 662



QUOTE (ScorLibran @ Aug 25 2003, 08:50 AM)
la - la - la - la - la - :fingers in my ears: - la - la - la - la - la - la - la - la - la - la - :not listening: - la - la - la - la - la - ...

Maybe you should change your sig then. To "Viva status-quo!" maybe. smile.gif

Anyway, back on topic. The 64k test samples weren't chosen to be easy to encode or anything. They are just interesting pieces of music from various genres which people here enjoy. Any real world music is suitable for a listening test. If they sound transparent then the codec has succeeded in its task and you can be happy to have such a wonderful encoder.
Go to the top of the page
+Quote Post
music_man_mpc
post Aug 25 2003, 19:35
Post #16





Group: Members (Donating)
Posts: 707
Joined: 20-July 03
From: Canada
Member No.: 7895



QUOTE (rjamorim @ Aug 24 2003, 08:53 PM)
I'm actually afraid he's one of those Volcano guys, with weird pointed ears and all.

Do you mean a 'Vulcan' like Spock, from Star Trek??

edit: sorry about the off-topicness

edit: OK, lets make this a less off-topic post. I already think that 128 mp3 sounds like shit, I can even ABX --alt-preset cbr 192 without too much trouble, but I can't COME CLOSE to being able to ABX MPC. I can't even ABX waiting at --quality 4.

This post has been edited by music_man_mpc: Aug 25 2003, 19:44


--------------------
gentoo ~amd64 + layman | ncmpcpp/mpd | wavpack + vorbis + lame
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 17th September 2014 - 22:56