IPB

Welcome Guest ( Log In | Register )

3 Pages V  < 1 2 3 >  
Closed TopicStart new topic
graph shows 320 kbps LAME 'never achieves transparency'?
pdq
post Aug 29 2011, 20:59
Post #26





Group: Members
Posts: 3407
Joined: 1-September 05
From: SE Pennsylvania
Member No.: 24233



Clearly this is someone who believes in the validity of whatever supports his own personal beliefs. If this graph had been drawn a little differently he would have argued vigorously how worthless it was, probably using many of the same points that have been made here.
Go to the top of the page
+Quote Post
Northpack
post Aug 29 2011, 22:05
Post #27





Group: Members
Posts: 455
Joined: 16-December 01
Member No.: 664



QUOTE (LithosZA @ Aug 29 2011, 16:16) *
I think by full 'Quality' on the graph they mean that it will never be bit exact quality from the source file. You never will be guaranteed that it will be lossless even if you pump up the bitrate to 320Kbit/s.
Transparent might be the wrong word. Transparent means that is 'sounds' transparent to the user.
Even though 192Kbit/s or even 320Kbit/s Vorbis sounds 'transparent' to me doesn't mean that is is full 'Quality'.

The way you put it, it's wrong. Quality is always a subjective term that can or can not be determined by objective parameters. Only if such a determination has been established, one can talk of "quality" as an objective term. Now if you want to compare a signals perceived quality to that a reference signal, you can safely assume that, if the signal is identical to the reference signal, the quality is also potentially identical (given that you listen to it exactly the same way). Thus lossless encoding can be said to have a "quality" of 100%, or in different words, to be transparent. Now if we leave away parts of the original signal, its quality potentially degrades, but we can't say exactly in which way, because of the unknown (infinitely complex) relation between our perception and the data-reducing (psycho-acoustical) algorithmns involved.

So we have to test empirically. That's how those algorithms are established and that's how we compare their outcome. We can do an ABX test if we want to check for transparency, or simply an AB test if we want to compare for quality. If such a test has been properly conducted and the number of trials and individual participants is large enough, we may draw relations like those (improperly) shown on this graph. Such a graph could very well show a quality of 100% (or 10 in this case) for a lossy encode. This relation however, is as solid as is the emprical data behind it. Strictly speaking, it's never possible to empirically prove that it would be transparent for every individual under any circumstance, but if our data is good enough we can assume it with pretty much the same confidence we have in the sun to rise again tomorrow.

This post has been edited by Northpack: Aug 29 2011, 22:10
Go to the top of the page
+Quote Post
Zarggg
post Aug 30 2011, 00:36
Post #28





Group: Members
Posts: 556
Joined: 18-January 04
From: bethlehem.pa.us
Member No.: 11318



QUOTE (Notat @ Aug 29 2011, 00:17) *
What is the definition of "transparent"?

Assuming you do not intend this question to be rhetorical, I would suggest a definition of "demonstrable over the course of multiple double-blind listening tests to be indistinguishable from the uncompressed (or losslessly-compressed) source with a statistical signifcance of p<0.05".
Go to the top of the page
+Quote Post
greynol
post Aug 30 2011, 01:53
Post #29





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



Perhaps I'm completely ignorant about p values, but I thought they are used to demonstrate the probability that someone has had a successful outcome without guessing. It is also my understanding that when someone cannot distinguish a difference and is left to guessing, the p value increases approaching 1. Based on this, saying someone is guessing with p<0.05 doesn't make any sense.

Can someone enlighten me?

This post has been edited by greynol: Aug 30 2011, 01:54


--------------------
I should publish a list of forum idiots.
Go to the top of the page
+Quote Post
db1989
post Aug 30 2011, 12:07
Post #30





Group: Super Moderator
Posts: 5275
Joined: 23-June 06
Member No.: 32180



Yep, the p-value is the probability that the observed result could otherwise have been obtained by chance. A result over the chosen significance level (usually 0.05) leads one to (not accept, but) fail to reject the null hypothesis, which usually proposes no effect of whichever treatment is being investigated.

I too am not sure how one can demonstrate that there is not an audible difference. One cannot confirm the null hypothesis (i.e. prove absence of an effect), only fail to reject it. Thus, transparency could be argued if multiple users failed to achieve statistically significant p-values, but not the converse.
Go to the top of the page
+Quote Post
Notat
post Aug 30 2011, 16:07
Post #31





Group: Members
Posts: 581
Joined: 17-August 09
Member No.: 72373



QUOTE (Zarggg @ Aug 29 2011, 17:36) *
QUOTE (Notat @ Aug 29 2011, 00:17) *
What is the definition of "transparent"?

Assuming you do not intend this question to be rhetorical, I would suggest a definition of "demonstrable over the course of multiple double-blind listening tests to be indistinguishable from the uncompressed (or losslessly-compressed) source with a statistical signifcance of p<0.05".

It was not rhetorical. I have read the HA wiki and Wikipedia. Neither your proposed definition nor either of these articles tells me precisely what listener we're talking about. The HA article suggests it is fair to use killer samples to demonstrate non-transparency. The graph we're discussing reportedly used conventional material. The lack of error bars are a problem but, the graph does appear to make a case for a lack of "transparency" according the definitions I've read.
Go to the top of the page
+Quote Post
db1989
post Aug 30 2011, 17:01
Post #32





Group: Super Moderator
Posts: 5275
Joined: 23-June 06
Member No.: 32180



QUOTE (Notat @ Aug 30 2011, 16:07) *
I have read the HA wiki and Wikipedia. Neither your [Zarggg] proposed definition nor either of these articles tells me precisely what listener we're talking about.

Those discussing transparency usually either refer to the majority (a problematic concept in itself as it usually refers to fairly tech-savvy listeners, but we'll have to make do until Big Brother mandates listening tests) or to the specific listener being addressed in a given discussion. Since you seem to have been referring specifically to this by krabapple . . .
QUOTE (krabapple @ Aug 29 2011, 00:47) *
[. . .] Apparently he thinks the graph shows mp3s can never be transparent. You see the artificial gulf there? [. . .] [This] hinges on the strictest definition of 'transparent', and puts a burden on that graph that it wasn't meant to bear. Misses the forest for the trees.
. . . I will agree that it seem to be implying some objective definition of transparency, which is obviously a non-starter, as is made plain on the Knowledgebase page:
QUOTE
Transparency, like sound quality, is subjective.

But I do not understand your question about the listener(s), and krabapple is correct to point out the nonsensicality of amirm saying that MP3 can never be transparent. It remains to be answered whether the latter is simply questioning the graph, which has already been discredited; or is lambasting the format overall as being insufficient for every possible use-case, in which event he is blatantly incorrect.
Go to the top of the page
+Quote Post
2Bdecided
post Aug 30 2011, 17:08
Post #33


ReplayGain developer


Group: Developer
Posts: 5139
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



I think the wiki article on "transparent" is pretty good. With all those caveats, something is transparent for you if you can't hear the difference (in a double blind test - but that goes without saying wink.gif ).

But more generally, the things we accept as transparent are things where reports of hearing a difference in a double-blind test are rare and uncorroborated.


I think far longer has now been spent discussing that graph than went into making it. It would probably have been better if the quality scale had no units. I'm not aware of any reliable units of subjective audio quality which have a globally understood and repeatable quantity in the same way as the units on a ruler.

Cheers,
David.
Go to the top of the page
+Quote Post
Notat
post Aug 30 2011, 19:05
Post #34





Group: Members
Posts: 581
Joined: 17-August 09
Member No.: 72373



QUOTE (2Bdecided @ Aug 30 2011, 10:08) *
I think far longer has now been spent discussing that graph than went into making it. It would probably have been better if the quality scale had no units. I'm not aware of any reliable units of subjective audio quality which have a globally understood and repeatable quantity in the same way as the units on a ruler.

That strikes me as throwing the baby out with the bathwater. The scale on a MUSHRA test is calibrated by its anchors. We're always going to have uncertainty in these results so the ruler is not a good model.

The original question here even before the graph entered the conversation was, "Is 320 kb LAME transparent?" So long as we can agree on a technical definition of "transparent" it seems like a simple experiment to conduct. The graph indicates that it has been attempted and the unverified results indicate that it is not transparent.

But, I'm getting the sense that there is no technical definition for transparent; That it's a "subjective" thing with context-dependent meaning. The HA article, for instance, describes how to demonstrate non-transparency then lists some transparent audio formats that I'm pretty sure have been shown to be non-transparent by these guidelines at least with specific program material and ears.
Go to the top of the page
+Quote Post
Nick.C
post Aug 30 2011, 19:37
Post #35


lossyWAV Developer


Group: Developer
Posts: 1791
Joined: 11-April 07
From: Wherever here is
Member No.: 42400



QUOTE (Notat @ Aug 30 2011, 19:05) *
The HA article, for instance, describes how to demonstrate non-transparency then lists some transparent audio formats that I'm pretty sure have been shown to be non-transparent by these guidelines at least with specific program material and ears.
It lists formats and bitrates at which they may be artifact free.... It does not state that they are transparent at those bitrates for all material.

A point about the article - the statement "use double-blind tests where the high anchor is the original uncompressed audio" infers that compressed audio should not be used. Why would the original audio compressed in FLAC be unsuitable? Surely either "uncompressed" should be removed or changed to "lossless" or "unprocessed" or something like that.


--------------------
lossyWAV -q X -a 4 --feedback 4| FLAC -8 ~= 320kbps
Go to the top of the page
+Quote Post
[JAZ]
post Aug 30 2011, 20:16
Post #36





Group: Members
Posts: 1783
Joined: 24-June 02
From: Catalunya(Spain)
Member No.: 2383



QUOTE (Notat @ Aug 30 2011, 20:05) *
"Is 320 kb LAME transparent?" So long as we can agree on a technical definition of "transparent" it seems like a simple experiment to conduct.


There are (just need to search hydrogenaudio) killer samples that can be quite easily ABXed between MP3 at 320kbps and original.

One could say that this is a proof that MP3 is not transparent at 320kbps.

Yet, that would be a pretty bad definition of transparent, especially since we are talking about lossy codecs, and I explain myself:


If I get a single file which is encoded with a lossless codec, and the audio portion is not bit-for-bit identical to the original, I can say that the encoder is buggy/is not working as expected/its quality is not 100%.

If I get a single file which is encoded with a lossy codec at its highest setting, and I am able to ABX it against the original, I can only say that the file is a killer sample for this encoder and version, which might or might not be a killer sample for the format itself, and I will have to test if other lossy codecs do a better job or also fail on it.

I cannot imply that the format or encoder is not transparent. I can only say that there is a sample which is not transparent, but transparency with a lossy codec implies what happens in the general case.

Said it in other words: there are many files where one would fail to ABX an MP3 at 320kbps versus its original. Since this number is empirically bigger than the cases where it can be ABXed, this setting is considered transparent for the general case.


At last, there is no way to demonstrate transparency, other than demonstrating that the file is bit-for-bit identical (In which case, the transparency is a logical consequence of the files being the same, not the opposite).
As said above, one can only demonstrate a difference using an ABX test. A non-difference cannot be demonstrated with this method.

Go to the top of the page
+Quote Post
greynol
post Aug 30 2011, 20:18
Post #37





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



<sigh>

A link to the discussion was given regarding the origin of that graph. Who bothered to read it?

Please allow me to repost (bold type added by me):
QUOTE (Gabriel @ Sep 27 2005, 06:10) *
What about a perceived quality vs size graph?

Arbitrarily, I'd give the following quality levels:

--abr 56: 3
--abr 90: 5
-V5: 7
-V4: 8
-V3:8.5
-V2: 8.7
-V0: 9.1
-b 320: 9.2

This is purely informal, but if you trace a graph of perceived quality vs average size, you will probably obtain a nice curve with valuable indication regarding efficiency of the settings.

As for the -V1 datapoint:
QUOTE (Synthetic Soul @ Sep 27 2005, 08:36) *
I added -V1 in there, to get the unsightly kink out of the first graph (qualily value is simply the average between -V2 and -V0).

IOW, these numbers are not based on scientific testing. That we need to somehow go down the rabbit hole justifying the unjustifiable by attempting to (re?)define the word transparent is deserving of ridicule.

Throwing out the baby with the bath water? There is no baby to throw.

This post has been edited by greynol: Aug 30 2011, 22:01


--------------------
I should publish a list of forum idiots.
Go to the top of the page
+Quote Post
Destroid
post Aug 30 2011, 20:22
Post #38





Group: Members
Posts: 550
Joined: 4-June 02
Member No.: 2220



In regards to the OP, what is the motive for slamming LAME anyway? I didn't read this "expert's" thread because hearing FUD about a codec gets... lame, pun intended I guess. Is he/she pimping another MP3 encoder, or another lossy format? I don't get the motivation for using the graph as evidence when it is clearly a loose (a very, very, loose) representation of bitrate-to-quality tradeoff.

I really don't see a vast number of HA users- the ones that encode to MP3- using LAME because of its known inferior design of no-transparency, duh wink.gif

As other users already suggested, he/she should do some ABX, or better yet participate in one of the public listening tests and just see how much (or little) a contribution they can give.

This post has been edited by Destroid: Aug 30 2011, 20:25


--------------------
"Something bothering you, Mister Spock?"
Go to the top of the page
+Quote Post
Notat
post Aug 30 2011, 21:50
Post #39





Group: Members
Posts: 581
Joined: 17-August 09
Member No.: 72373



Sorry about reader fatigue and sloppy research.

I find this topic interesting because the religion here at HA is that modern first-generation lossy encoding sounds the same as the original. "Transparent" is commonly used to describe this feature. And yet we have many audio lovers (including HA members) who insist on using lossless formats. And why is that? Because lossless gives them peace of mind? Because they think lossless sounds better? Or is it because perceptual coding, no matter the sophistication or bitrate, potentially introduces significant and audible degradation?
Go to the top of the page
+Quote Post
pdq
post Aug 30 2011, 22:34
Post #40





Group: Members
Posts: 3407
Joined: 1-September 05
From: SE Pennsylvania
Member No.: 24233



I think the main reason that many of us here at HA use lossless encoding is to avoid generational degradation. If I thought that I would never need to reencode to a different format EVER, then I might be perfectly happy with a high quality lossy encode for my collection.
Go to the top of the page
+Quote Post
db1989
post Aug 30 2011, 22:40
Post #41





Group: Super Moderator
Posts: 5275
Joined: 23-June 06
Member No.: 32180



Ignore killer samples and golden ears, and instead think of transparency of referring to general usage and general users. Does the problem now magically evaporate?

Transparency is subjective in the sense that it is limited to certain user(s) by definition, but this is a much better kind of subjective than the things ToS8 exists to avoid. Is there any point in quibbling over semantics and lamenting the fact that one can never objectively describe the quality of representation provided by a lossy codec?
Go to the top of the page
+Quote Post
Destroid
post Aug 30 2011, 22:44
Post #42





Group: Members
Posts: 550
Joined: 4-June 02
Member No.: 2220



QUOTE (Notat @ Aug 30 2011, 21:50) *
And yet we have many audio lovers (including HA members) who insist on using lossless formats. And why is that? Because lossless gives them peace of mind? Because they think lossless sounds better? Or is it because perceptual coding, no matter the sophistication or bitrate, potentially introduces significant and audible degradation?
QUOTE (pdq @ Aug 30 2011, 22:34) *
I think the main reason that many of us here at HA use lossless encoding is to avoid generational degradation.

When dealing with master tracks lossy is a non-option. In that last regard MP3 could be considered ephemeral and for casual listening but never for archival.

Not to throw the thread out of orbit, I think there might a benefit to defining terms for "transparency" in regards to the degree of "lossy quality." Of course, users' ears will differ and just as much as users' music genres/preferences will differ: i.e. classical vs. pop rock will have different average bitrates at -V2. Artifacts are likely to surface in lossy but how annoying they are and what the listener considers annoying (pre-echo vs. warbling) is also another area of opinion.

This post has been edited by db1989: Aug 30 2011, 23:01
Reason for edit: adding quote attributions


--------------------
"Something bothering you, Mister Spock?"
Go to the top of the page
+Quote Post
mjb2006
post Aug 30 2011, 22:51
Post #43





Group: Members
Posts: 811
Joined: 12-May 06
From: Colorado, USA
Member No.: 30694



I have gone ahead and updated the caption for the graph in the wiki, and linked to the relevant discussions.

Old caption: Here a trial to see how the perceived listening quality improves with settings/averaged filesize

New caption: This informal graph shows how LAME's highest quality settings result in progressively larger file sizes, but yield relatively smaller gains in perceived listening quality.

This post has been edited by mjb2006: Aug 30 2011, 22:51
Go to the top of the page
+Quote Post
greynol
post Aug 30 2011, 23:23
Post #44





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



QUOTE (Destroid @ Aug 30 2011, 14:44) *
I think there might a benefit to defining terms for "transparency" in regards to the degree of "lossy quality."

For any particular individual at any given time, something is either transparent or it isn't. Within these constraints there are no shades of gray.


--------------------
I should publish a list of forum idiots.
Go to the top of the page
+Quote Post
greynol
post Aug 30 2011, 23:25
Post #45





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



QUOTE (mjb2006 @ Aug 30 2011, 14:51) *
I have gone ahead and updated the caption for the graph in the wiki, and linked to the relevant discussions.

Do away with the scale and data points and I might buy into the idea of there being a graph, otherwise dump it altogether.

This post has been edited by greynol: Aug 30 2011, 23:25


--------------------
I should publish a list of forum idiots.
Go to the top of the page
+Quote Post
MichaelW
post Aug 31 2011, 02:21
Post #46





Group: Members
Posts: 631
Joined: 15-March 07
Member No.: 41501



"Transparent" is a word, and words have different kinds of usage.

I take it that, in a technically defined sense, "transparent" means that in a particular (series of) test(s), particular listener(s) were unable to tell the difference between specific lossily-coded sample(s) and their original(s). Even in this defined sense, transparency can't be proved, since individual listeners can have good days and bad days.

As a word in general conversation, it seems to often be used in a way that relates to, but is not the same as, the technical sense. So in answer to a question like, "At what setting does LAME get to be transparent?" (which in some contexts would be a sensible enough practical question) you might say "V 3," meaning "V 3 for most people with most material most of the time." On HA people are careful to include the qualifiers, being conscious of the technical sense; in a non-technical setting, in which there's a danger of good advice being disregarded because of the MEGO factor.

IME, we are all likely to slip into the trap of treating ordinary-language words as though they were explicitly defined technical terms.
Go to the top of the page
+Quote Post
greynol
post Aug 31 2011, 17:18
Post #47





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



I don't ever recall there being a problem with the meaning of the word as it relates to how it is used on this forum.

transparent = perceptually indistinguishable


--------------------
I should publish a list of forum idiots.
Go to the top of the page
+Quote Post
Notat
post Aug 31 2011, 18:16
Post #48





Group: Members
Posts: 581
Joined: 17-August 09
Member No.: 72373



QUOTE (MichaelW @ Aug 30 2011, 19:21) *
I take it that, in a technically defined sense, "transparent" means that in a particular (series of) test(s), particular listener(s) were unable to tell the difference between specific lossily-coded sample(s) and their original(s). Even in this defined sense, transparency can't be proved, since individual listeners can have good days and bad days.

I would be tempted to read a corollary from this: You will always be able to find someone, somewhere, somehow who will experience perceptible degradation with any lossy coding system.
Go to the top of the page
+Quote Post
halb27
post Aug 31 2011, 20:18
Post #49





Group: Members
Posts: 2435
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



So transparency depends on the listener's abilities including listening conditions and the sample(s) he listens to. It's personal.

A practical definition of a codec's transparency could mean that there was no listener so far who was able to successfully ABX a sample. Sure the quality of this kind of transparency depends heavily on the number of listeners (and their ABXing abilities) trying hard to find non-transparent (in the personal sense above) samples.

As for the graph simple words about the diminishing returns fact would be more honest, but people like graphics. It's a good exercise never to simply beleive in graphics. People can be easily fooled by graphics (for instance by using a suggestive scale) even when the graphics is formally correct.
In our case it's clear that the quality scale can't have a real meaning and is chosen arbitrarily at the author's will to demonstrate the diminishing returns phenomenon.

This post has been edited by halb27: Aug 31 2011, 20:19


--------------------
lame3100m -V1 --insane-factor 0.75
Go to the top of the page
+Quote Post
MichaelW
post Aug 31 2011, 22:56
Post #50





Group: Members
Posts: 631
Joined: 15-March 07
Member No.: 41501



QUOTE (greynol @ Sep 1 2011, 04:18) *
I don't ever recall there being a problem with the meaning of the word as it relates to how it is used on this forum.

transparent = perceptually indistinguishable

I didn't say there was a problem here; in fact, if you were actually to read things, you would see I explicitly made an exception for here. There is a problem elsewhere.

BTW, perceptually indistinguishable to whom, under what circumstances, always, sometimes?
Go to the top of the page
+Quote Post

3 Pages V  < 1 2 3 >
Closed TopicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 20th September 2014 - 15:55