IPB

Welcome Guest ( Log In | Register )

2 Pages V  < 1 2  
Reply to this topicStart new topic
High bitrate MP3 vs. Lossless ABX Tests?, Are there any studies...?
Lyx
post Jun 5 2005, 02:40
Post #26





Group: Members
Posts: 3353
Joined: 6-July 03
From: Sachsen (DE)
Member No.: 7609



QUOTE (Woodinville @ Jun 5 2005, 03:26 AM)
None the less, I don't think there is a 128kb/s MP3 encoder that meets the "ABX transparent" standard, or at least I haven't meant one, even in tests where all of the degradations were reported as being over 4.0 on the scale, the ID for the 'reference' signal in an ABC/hr test wasn't even close to any reasonable random hypothesis.  The MPEG test results, at least, as well as the CRC results that are on much, much newer encoders, have been published. I don't know the citations offhand, but the results were not ambiguous in any real fashion.
*

Well, that 128kbit MP3 (especially CBR) is not transparent to everyone is nothing new. I doubt anyone here would argue on that(although some people would probably point out, that LAME VBR around 110-140kbit is very difficult to most people, but certainly not to almost everyone).
Newer doesn't automatically mean better. If i remember correctly, then Fhg is mostly tuned for low-mid bitrates, and its VBR-implementation is not on par with current LAME). The situation at low-bitrates is the opposite way around: There, fhg is more optimized and has intensity-stereo (which lame lacks). So, simplified one could say that below 112kbit fhg tends to be better, while above LAME usually is better.

And, i highly doubt that they did test something which "ain't an mp3 encoder" ;-) So, i suppose they did test fhg. So, the problem is that we have lots of listening tests on this board with encoder A, and listening tests from the MPEG-folks on encoder B. But both can only be compared against each other, not merged.

Thanks for the infos,
- Lyx

This post has been edited by Lyx: Jun 5 2005, 02:43


--------------------
I am arrogant and I can afford it because I deliver.
Go to the top of the page
+Quote Post
2Bdecided
post Jun 6 2005, 11:54
Post #27


ReplayGain developer


Group: Developer
Posts: 5283
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



Woodinville,

I'm not quite sure what you're trying to say about the official tests.

It's true that mp2 and mp3 "failed" at the target bitrates.

However, AAC Main Profile at 128kbps did rather well (if not a 100% perfect pass) here:

http://www.mp3-tech.org/programmer/docs/w2006.zip

I haven't seen any such test for "high bitrate" mp3, though we know some specific samples where Lame mp3 has difficulty, while commonly high bitrate Lame passes ABX tests for most samples - though some listeners find problems on certain types of sound.

Cheers,
David.
Go to the top of the page
+Quote Post
Woodinville
post Jun 6 2005, 18:46
Post #28





Group: Members
Posts: 1403
Joined: 9-January 05
From: JJ's office.
Member No.: 18957



QUOTE (2Bdecided @ Jun 6 2005, 02:54 AM)
However, AAC Main Profile at 128kbps did rather well (if not a 100% perfect pass) here


The AAC Main Profile, with what the developers of the codec believed was the best available encoder, still provided a non-random (to many orders of magnitude, I've forgotten how many) ID rate in reference identification in the final AAC verification test, which I suspect is the best test run with AAC.

The impairments that were listed were quite small, but many (not all) listeners were easily able to ID the reference signal, which shows that there is "indistinguishable" according to MPEG (all items over 4.0), and "transparent" according to ABX id rate, and that while MPEG's definition is very good, it's not perfect.

In that test, some listeners were excluded for having negative impairment scores, this was used as a screening method. None the less, there were far, far too many people who ID'ed beyond the significance required by the number of trials and subjects, especially on the most difficult tracks.

I think this says it all, quoting the report:

QUOTE
The AAC Main 128 codec is indistinguishable from the original for 8 of 10 items, the AAC Main 96 for 3 items, the AAC LC 128 for 8 of the 10 items,


Note that this is a weaker test than asking what the ID rate for the reference is, as well.

This post has been edited by Woodinville: Jun 6 2005, 18:47


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
ff123
post Jun 6 2005, 20:24
Post #29


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



A few things to note about Roberto's tests:

1. People who pull the hidden reference slider down on any codec are typically excluded from the analysis (although I think if there are only a few listeners, he may decide to rely on ABX results, if there are any, to make a final decision about whether or not to exclude a listener's results). This is a much tougher standard than just excluding somebody who has a negative impairment score. However, it does mean that the results will always be less than 5.0 for any codec. Remember that Roberto isn't trying to find out whether or not a codec is statistically distinguishable from the reference, only what the relative ratings are of the codecs under test.

2. Although there may be a few "problem" samples mixed in with Roberto's tests, the prevailing idea has been to be representative of "real" music. I don't know if the MPEG tests are tilted towards problem samples or not.

3. The group of people who participate in Internet tests are probably skewed towards more discriminating listeners.

All this is saying, which should be rather obvious, is that Roberto's tests don't necessarily compare with the MPEG tests, although I think they certainly serve their purpose. But that purpose does not include determining if a codec is statistically the same as the reference.

ff123
Go to the top of the page
+Quote Post
Pensive
post Jan 29 2006, 00:39
Post #30





Group: Banned
Posts: 10
Joined: 2-June 05
Member No.: 22482



QUOTE (Jebus @ Jun 3 2005, 12:15 AM)
QUOTE (Pensive @ Jun 2 2005, 03:11 PM)
I'm new, hi everyone, i want to make a suggestion on this subject.

This is my slant:

In a few years ill be buying either an sacd or dvd-a player depending on the climate, which of course will be a wireless device and send the digital data straight to the amplifier, which will also be wifi connected to my media storage device, containing my music collection, very likely with storage in the realm of terabytes.

When i play my SACDs, and then flip to my mp3 encoded tracks, they are going to sound terrible. Even cds will begin to sound a little lifeless. I'm putting all my CDs on my HD with Flac, and thats that - it makes sense for the future, sooner or later we'll all have media devices instead of dvd players.

Don't spoil your expensive systems you'll buy in the future with a poor substitute for lossless when hard drives are getting so cheap and so big.

Soonish, I'm going to mirror raid myself up a terabyte archive beast (current costs = 8*250 gig HDs - Im waiting till i can run 4*500 gig hds without breaking the bank) and once ive got that there is no excuse for worrying about the space my music collection takes up. Some of you may have enormous collections, which causes a problem, but i still like the idea of wifi music throughout my house at CDDA quality with no cds in sight!!

Just my perspective, in respect of yours
*


*sigh*. No one has yet been able to prove they can hear a difference between a PROPERLY MASTERED CD (that's a hard thing to come by, unfortunately) and a DVD-A or SACD disc. If you can, please do so. If not, don't spout such misinformation. TOS #8 violation and whatnot.

I'm really getting sick of saying this. I think this is like the 5th time in 2 days.
*



For goodness sake read my post first. I believe you just violated one of your own TOS.

I was saying that SACDs will make an MP3 collection sound bad, as opposed to say - an uncompressed CDDA library. If I was saying what you suggested then it wouldnt belong in this thread now would it? Also I'm sure SACDs will make reissues from the master vinyls of old records sound a lot better than a cd equivalent reissue, since it will support the full frequency ranges of audibility, and also has the dynamic range to cope without any processing whatsoever. Im sure you're all aware that they have to compress the audio samples to fit the dynamic range of AudioCDs, since you're all so very knowledgable.

So far youve totally misunderstood a perfectly clear post, been completely arsey and called me a troll, and accused me of spreading misinformation. Its a great welcome to the forum. You know what you can do with your TOS. Cya

This post has been edited by Pensive: Jan 29 2006, 00:44
Go to the top of the page
+Quote Post
Zoom
post Jan 29 2006, 02:29
Post #31





Group: Members (Donating)
Posts: 263
Joined: 23-February 04
From: United States
Member No.: 12219



QUOTE (Pensive @ Jan 28 2006, 06:39 PM)
For goodness sake read my post first. I believe you just violated one of your own TOS.

I was saying that SACDs will make an MP3 collection sound bad, as opposed to say - an uncompressed CDDA library. If I was saying what you suggested then it wouldnt belong in this thread now would it? Also I'm sure SACDs will make reissues from the master vinyls of old records sound a lot better than a cd equivalent reissue, since it will support the full frequency ranges of audibility, and also has the dynamic range to cope without any processing whatsoever. Im sure you're all aware that they have to compress the audio samples to fit the dynamic range of AudioCDs, since you're all so very knowledgable.

So far youve totally misunderstood a perfectly clear post, been completely arsey and called me a troll, and accused me of spreading misinformation. Its a great welcome to the forum. You know what you can do with your TOS. Cya
*


My emphasis on "will" and "sure".

ToS#8
"Hydrogenaudio is supposed to be an objectively minded community that relies on double-blind testing and relevant methods of comparison in discussion about sound quality..."

That is the point I believe he was trying to get across. You cannot say without proof that SACD will make an mp3 file sound bad, unless you've done testing or cite testing that proves this. This has been discussed over and over on HA. If you aren't going to go about comparisons based on scientific merit, and instead focus on purported capabilities then you are in violation of ToS#8.
Go to the top of the page
+Quote Post
William
post Jan 29 2006, 02:39
Post #32





Group: Members
Posts: 288
Joined: 8-January 02
Member No.: 959



QUOTE (Pensive @ Jan 28 2006, 11:39 PM)
For goodness sake read my post first. I believe you just violated one of your own TOS.

Maybe I misunderstood too, but I found your post to be BS.

QUOTE (Pensive @ Jan 28 2006, 11:39 PM)
I was saying that SACDs will make an MP3 collection sound bad, as opposed to say - an uncompressed CDDA library. If I was saying what you suggested then it wouldnt belong in this thread now would it?

It doesn't matter. You post confuses a lot of us anyways.

QUOTE (Pensive @ Jan 28 2006, 11:39 PM)
Also I'm sure SACDs will make reissues from the master vinyls of old records sound a lot better than a cd equivalent reissue, ...blah...blah

How "sure" are you?

QUOTE (Pensive @ Jan 28 2006, 11:39 PM)
So far youve totally misunderstood a perfectly clear post, been completely arsey and called me a troll, and accused me of spreading misinformation. Its a great welcome to the forum. You know what you can do with your TOS. Cya

I don't think your post is "perfectly clear", as this:

QUOTE (Pensive @ Jan 28 2006, 11:39 PM)
When i play my SACDs, and then flip to my mp3 encoded tracks, they are going to sound terrible. Even cds will begin to sound a little lifeless.

Prove it with ABX results.

This post has been edited by William: Jan 29 2006, 02:40
Go to the top of the page
+Quote Post
Pensive
post Jan 29 2006, 12:21
Post #33





Group: Banned
Posts: 10
Joined: 2-June 05
Member No.: 22482



QUOTE
Prove it with ABX results.


Okay, my post was clear, it was just that you got confused because you didnt read much of it at all. Then your buddy just started dickriding you. That was the only confusion.

One small problem with ABXing that stuff:
1) I have no SACDs
2) I cant be sure any SACD sample I get hold of will be well produced and mastered, taking full advantage of the frequency response, from a master press, which of course would have to be analogue recorded, mixed and mastered all the way. Supply me with this and I'll abx it.


But again youve missed my point.

I was comparing the change between a well mastered SACD, and a 192kb MP3. I dont need to ABX it to know that - in my car I can tell EVERY cd that has been burnt from MP3s by ear, although I admit it gets harder with 320kbps ones. If I put an original CD in there the difference is phenomenal between it and 192kb mp3s. Are you really suggesting that I should waste hours of my time abxing it when I - and everyone else with half a brain on this site knows what the result is going to be?

I think you're just being awkward, perhaps my post was a little out of place but it certainly wasnt BS, you misread it, and what you misread it to be could have been interpreted as BS.

The bottom line is: my statement was based on the fact that 128-192kb mp3s, dont sound as good as CDDA, or SACD. IF you think that is BS without ABX testing then I'm glad you have that much spare time to waste.

I dont. Good day.

Edit:
One addition I should mention is that the main issue i have with mp3s, is listener fatigue, especially at high volums.

ABX testing cannot possibly give any decent results which indicate anything about fatiguing. ABX is not the one ring to rule them all, and it does not sort everything from everything else. These are textured sound waves, you cant use a binary "is it better" system like ABX to say if it sounds GOOD. I think you are all in a world of your own. Try abxing some heavy sections of NIN- The Fragile @ 320kb and @ CDDA. MP3 encoding falls apart with kind of guitar texture.

This post has been edited by Pensive: Jan 29 2006, 12:56
Go to the top of the page
+Quote Post
William
post Jan 29 2006, 16:34
Post #34





Group: Members
Posts: 288
Joined: 8-January 02
Member No.: 959



QUOTE (Pensive @ Jan 29 2006, 11:21 AM)
Supply me with this and I'll abx it.

An artifact is an artifact. If it exists, and your ear can hear it, then you can hear it. It doesn't matter about mastering.

QUOTE (Pensive @ Jan 29 2006, 11:21 AM)
I was comparing the change between a well mastered SACD, and a 192kb MP3. I dont need to ABX it to know that - in my car I can tell EVERY cd that has been burnt from MP3s by ear, although I admit it gets harder with 320kbps ones.

You have already violated TOS #8.

QUOTE (Pensive @ Jan 29 2006, 11:21 AM)
If I put an original CD in there the difference is phenomenal between it and 192kb mp3s. Are you really suggesting that I should waste hours of my time abxing it when I - and everyone else with half a brain on this site knows what the result is going to be?

Yes, because it has been proven before that can simply be placebo.

QUOTE (Pensive @ Jan 29 2006, 11:21 AM)
I think you're just being awkward, perhaps my post was a little out of place but it certainly wasnt BS, you misread it, and what you misread it to be could have been interpreted as BS.

I really hope so, but it seems it is not.

QUOTE (Pensive @ Jan 29 2006, 11:21 AM)
The bottom line is: my statement was based on the fact that 128-192kb mp3s, dont sound as good as CDDA, or SACD. IF you think that is BS without ABX testing then I'm glad you have that much spare time to waste.

I dont. Good day.

Again, prove it. Don't you know that you just showed me you dare not do so?

QUOTE (Pensive @ Jan 29 2006, 11:21 AM)
Edit:
One addition I should mention is that the main issue i have with mp3s, is listener fatigue, especially at high volums.
(again, BS)

I believe I need to call moderator now. You are BS and troll.
Go to the top of the page
+Quote Post
Pio2001
post Jan 30 2006, 02:19
Post #35


Moderator


Group: Super Moderator
Posts: 3936
Joined: 29-September 01
Member No.: 73



In case you don't browse the forum using the sections, you can find some ABX tests in the "listening test" forum : http://www.hydrogenaudio.org/forums/index.php?showforum=40

Here are some old ABX tests at high bitrate :

http://www.hydrogenaudio.org/forums/index....wtopic=7783&hl=
http://www.hydrogenaudio.org/forums/index....howtopic=12031&
http://www.hydrogenaudio.org/forums/index....topic=12534&hl=

I've been here since september 2001, and never saw a study about lossy compression at high bitrate, exept Guruboolez' one, below in this forum section.

128 kbps tests run by Roberto Amorim and Sebastan Mares are the hardest that a large number of people can undergo. For higher bitrates, only some poeple can perform some ABX tests with some samples.

I've heard many times about a professionnal german study that tested MP3 at 256 kbps, before 2001, and that concluded that it was transparent to the listeners, but it seems that the paper is not on the Internet anymore. I've read that they listened with Senheiser Orpheus headphones.
However, many problem samples are known, see the above links.

The alt-presets for Lame were developed around 2001. As far as I remember, no ABX tests were published during the development. I think that the dm-presets have undergo some ABX tests, but it was in a forum that has now been taken offline (r3mix.net)
Go to the top of the page
+Quote Post
ff123
post Jan 30 2006, 04:23
Post #36


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



QUOTE (Pio2001 @ Jan 29 2006, 05:19 PM)
I've heard many times about a professionnal german study that tested MP3 at 256 kbps, before 2001, and that concluded that it was transparent to the listeners, but it seems that the paper is not on the Internet anymore. I've read that they listened with Senheiser Orpheus headphones.
However, many problem samples are known, see the above links.
*


The article you reference is here:

http://www.heise.de/ct/00/06/092/default.shtml

ff123

This post has been edited by ff123: Jan 30 2006, 04:24
Go to the top of the page
+Quote Post

2 Pages V  < 1 2
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 28th November 2014 - 13:55