IPB

Welcome Guest ( Log In | Register )

3 Pages V  < 1 2 3 >  
Reply to this topicStart new topic
What does ABX really prove?
shadowking
post Nov 11 2007, 05:07
Post #26





Group: Members
Posts: 1527
Joined: 31-January 04
Member No.: 11664



I wouldn't even want to see any lossless abx logs - assuming all samples were bit identical.


--------------------
Wavpack -b450s0.7
Go to the top of the page
+Quote Post
William
post Nov 11 2007, 06:14
Post #27





Group: Members
Posts: 288
Joined: 8-January 02
Member No.: 959



audioslut512:

What is your definition of ABX? What is your testing method?
Go to the top of the page
+Quote Post
audioslut512
post Nov 11 2007, 07:54
Post #28





Group: Members
Posts: 19
Joined: 8-November 07
Member No.: 48571



QUOTE (William @ Nov 11 2007, 00:14) *
audioslut512:

What is your definition of ABX? What is your testing method?


I just used the ABX in foobar2000.
Go to the top of the page
+Quote Post
Bad Monkey
post Nov 11 2007, 10:11
Post #29





Group: Members
Posts: 90
Joined: 22-August 07
Member No.: 46407



Just run the bit-comparator to check you are in fact comparing like for like... it will tell you if there is any difference already...

On 2nd thoughts, why is this nonsense being allowed on HA? I thought I was on Head-Fi for a moment...
Go to the top of the page
+Quote Post
tgoose
post Nov 11 2007, 10:18
Post #30





Group: Members
Posts: 407
Joined: 12-April 05
Member No.: 21399



QUOTE (greynol @ Nov 10 2007, 18:40) *
An ABX test cannot prove that there isn't a difference. It only has the ability to demonstrate that the specific person taking the test was able to determine that there was a difference. These are not the same thing.

Well in actuality, it can never even prove that the person was able to determine a difference, it can only show how likely it is that the listener can tell a difference. That may seem academic but it's important for the next point:



QUOTE (audioslut512 @ Nov 11 2007, 03:56) *
All I did was ABX Radiohead's Kid A, encoded in FLAC level 4 and level 8, both directly from the CD. I matched the correct tracks to one another 9 of the 10 times I ran it. That's all. Maybe I screwed something up while encoding them that caused one to sound different from the other. Ultimately I had no idea which one was which, I just managed to notice the differences and match the tracks accordingly. Hence my post...

I can't remember exactly how the output of fb2k looks (or if it's even the same as last time I used it) but it should have had a percentage at the end which says how likely it is you were just guessing. 9/10 is quite high but if you conduct enough ABX tests clicking randomly you will get every result from 0/10 to 10/10. If you can get the same result again even with ten tests then it becomes much more likely that you have a problem somewhere in your encoding. Otherwise it's probably just chance.

This post has been edited by tgoose: Nov 11 2007, 10:21
Go to the top of the page
+Quote Post
Alexxander
post Nov 11 2007, 11:06
Post #31





Group: Members
Posts: 463
Joined: 15-November 04
Member No.: 18143



QUOTE (audioslut512 @ Nov 11 2007, 05:56) *
I would if I had them. All I did was ABX Radiohead's Kid A, encoded in FLAC level 4 and level 8, both directly from the CD. I matched the correct tracks to one another 9 of the 10 times I ran it. That's all. Maybe I screwed something up while encoding them that caused one to sound different from the other. Ultimately I had no idea which one was which, I just managed to notice the differences and match the tracks accordingly. Hence my post...

You sure must have done something wrong somewhere between putting the CD in drive and clicking the ABX utility in foobar2000. As said before in an other post, bitcompare the decoded FLAC files and you'll see the resulting wav files are different.
Go to the top of the page
+Quote Post
kjoonlee
post Nov 11 2007, 12:34
Post #32





Group: Members
Posts: 2526
Joined: 25-July 02
From: South Korea
Member No.: 2782



"More often than not" does not equal "statistically significant".

Let's say you got 6 out of 10 correct: this is usually written 6/10 at HA. 6/10 has a 37.6% chance of being sheer luck. This is no good.

2/3, 3/4, 3/5, 4/6, 4/7, 5/8, 5/9, 6/10, 6/11... these are all no good.

If you're saying something like 60/100 then that might mean something, but unless you give us those numbers, you're just being vague and might well be fooling yourself.


--------------------
http://blacksun.ivyro.net/vorbis/vorbisfaq.htm
Go to the top of the page
+Quote Post
robert
post Nov 11 2007, 13:03
Post #33


LAME developer


Group: Developer
Posts: 788
Joined: 22-September 01
Member No.: 5



QUOTE (audioslut512 @ Nov 11 2007, 04:56) *
All I did was ABX Radiohead's Kid A, encoded in FLAC level 4 and level 8, both directly from the CD.
Do I get you right, you ripped it twice from the CD? And what does the difference you hear sound like? Some popping noise, some click?
Go to the top of the page
+Quote Post
audioslut512
post Nov 11 2007, 23:38
Post #34





Group: Members
Posts: 19
Joined: 8-November 07
Member No.: 48571



QUOTE (robert @ Nov 11 2007, 07:03) *
QUOTE (audioslut512 @ Nov 11 2007, 04:56) *
All I did was ABX Radiohead's Kid A, encoded in FLAC level 4 and level 8, both directly from the CD.
Do I get you right, you ripped it twice from the CD? And what does the difference you hear sound like? Some popping noise, some click?



It was actually quite interesting...

When I listened to substantial chunks of each track I couldn't tell the difference at all.

I then came up with a little method, of switching between A,X,Y,B, in rapid succession, with about 1-2 seconds in between each switch.

I then noticed that if when I did this, cycling through the four buttons clockwise for example, I would hear shifts in the overall sound (im not sure how to desribe those shifts, I just referred to them in my mind as up and down).

So in doing so, if I heard upshift, downshift, upshift, downshift, I would then answer accordingly, and got it right 9 out of 10 times.

Like I said before though, I was never challenging ABX, and admitted that I might have screwed up the encoding processes of the tracks, and maybe that was what caused me to be able to discern the difference.

The tracks were both ripped from the same CD however, and both were ripped under the same conditions (i.e, no other programs running at the time).

I suppose I could and do it again to see if its repeatable, and to maybe stop so many people from bashing me, but my post was never about the ABX test I did on those tracks to begin with, it was about a question that those tests led me to...
Go to the top of the page
+Quote Post
greynol
post Nov 11 2007, 23:43
Post #35





Group: Super Moderator
Posts: 10092
Joined: 1-April 04
From: San Francisco
Member No.: 13167



The point is that you cannot make a proper conclusion from an ABX test if you don't know how conduct one correctly.

eevan really hammered the point home if you ask me:
QUOTE
Why should this be of any interest if you say that it's irrelevant to pinpoint the cause of your test result.


This post has been edited by greynol: Nov 12 2007, 00:06


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
audioslut512
post Nov 11 2007, 23:44
Post #36





Group: Members
Posts: 19
Joined: 8-November 07
Member No.: 48571



QUOTE (greynol @ Nov 11 2007, 17:43) *
The point is that you cannot make a proper conclusion from an ABX test if you don't know how correctly conduct one.


ok well what did I do wrong then?
Go to the top of the page
+Quote Post
greynol
post Nov 12 2007, 00:00
Post #37





Group: Super Moderator
Posts: 10092
Joined: 1-April 04
From: San Francisco
Member No.: 13167



You chose not to accept the fact that you couldn't ABX substantial chunks and you did not take the proper steps to ensure that there wasn't an error in the encodings or that they used the exact same source.

Some good may come out of this discussion after all since you may have identified a problem with the way foobar2000's ABX test. Too bad it took all the way to the 34th post for some useful albeit still qualitative information to come out.

You are aware that foobar2000 decodes the files you wish to compare to wave and places them in a temporary directory? Being that these were all lossless encodes, you could have done a binary comparison as a simple sanity check.

It's quite possible that subsequent rips of the same track can produce different results, even under seemingly similar circumstances; and yes, these would have been revealed with a binary comparison. A ripping error is the only logical conclusion if a binary comparison revealed differences. Lossless is lossless after all!

This post has been edited by greynol: Nov 12 2007, 00:06


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
Heresiarch
post Nov 12 2007, 00:09
Post #38





Group: Members
Posts: 2
Joined: 22-October 07
Member No.: 48083



There is just no point in ABXing lossless files. If you suspect an encoding error you should do a bit comparison. Either there is no difference or there is. No point in listening for a difference in either case.
Go to the top of the page
+Quote Post
greynol
post Nov 12 2007, 01:13
Post #39





Group: Super Moderator
Posts: 10092
Joined: 1-April 04
From: San Francisco
Member No.: 13167



QUOTE (Heresiarch @ Nov 11 2007, 15:09) *
There is just no point in ABXing lossless files.
It all depends on what you're trying to compare. What if a track gives me ripping errors and I want to see if I can hear them?


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
Bad Monkey
post Nov 12 2007, 02:08
Post #40





Group: Members
Posts: 90
Joined: 22-August 07
Member No.: 46407



He doesn't even know if there are errors. It's the first thing to do. Run the bit comparator already!
Go to the top of the page
+Quote Post
sld
post Nov 12 2007, 05:34
Post #41





Group: Members
Posts: 1017
Joined: 4-March 03
From: Singapore
Member No.: 5312



QUOTE (audioslut512 @ Nov 11 2007, 09:59) *
just forget it kids.

You're tacitly admitting you're trolling this forum?
Go to the top of the page
+Quote Post
audioslut512
post Nov 12 2007, 06:57
Post #42





Group: Members
Posts: 19
Joined: 8-November 07
Member No.: 48571



QUOTE (greynol @ Nov 11 2007, 18:00) *
You chose not to accept the fact that you couldn't ABX substantial chunks and you did not take the proper steps to ensure that there wasn't an error in the encodings or that they used the exact same source.

Some good may come out of this discussion after all since you may have identified a problem with the way foobar2000's ABX test. Too bad it took all the way to the 34th post for some useful albeit still qualitative information to come out.

You are aware that foobar2000 decodes the files you wish to compare to wave and places them in a temporary directory? Being that these were all lossless encodes, you could have done a binary comparison as a simple sanity check.

It's quite possible that subsequent rips of the same track can produce different results, even under seemingly similar circumstances; and yes, these would have been revealed with a binary comparison. A ripping error is the only logical conclusion if a binary comparison revealed differences. Lossless is lossless after all!



It's not that I chose not to accept that I couldn't ABX substantial chunks, it's just that I didn't know any better, and for that matter still don't.

Now that you mention binary comparison, I will look into it. That is yet another thing I am trying to understand...
Go to the top of the page
+Quote Post
audioslut512
post Nov 12 2007, 07:09
Post #43





Group: Members
Posts: 19
Joined: 8-November 07
Member No.: 48571



QUOTE (sld @ Nov 11 2007, 23:34) *
QUOTE (audioslut512 @ Nov 11 2007, 09:59) *

just forget it kids.

You're tacitly admitting you're trolling this forum?



Trolling for whom?

I am just trying to understand how this stuff works and get crucified for it.

Thanks again.
Go to the top of the page
+Quote Post
Light-Fire
post Nov 12 2007, 08:04
Post #44





Group: Members
Posts: 420
Joined: 5-August 06
From: Canada
Member No.: 33645



QUOTE (audioslut512 @ Nov 10 2007, 22:56) *
QUOTE (Mike Giacomelli @ Nov 10 2007, 21:16) *

I still don't really understand what you did. Could you just post the logs?



I would if I had them. All I did was ABX Radiohead's Kid A, encoded in FLAC level 4 and level 8, both directly from the CD. I matched the correct tracks to one another 9 of the 10 times I ran it. That's all. Maybe I screwed something up while encoding them that caused one to sound different from the other. Ultimately I had no idea which one was which, I just managed to notice the differences and match the tracks accordingly. Hence my post...


You more likely screwed up when testing and not when encoding. You are saying that you perceived a difference between two things that are EXACTLY the same.
Go to the top of the page
+Quote Post
SiriusB
post Nov 12 2007, 08:25
Post #45





Group: Members
Posts: 26
Joined: 30-August 07
Member No.: 46625



QUOTE (audioslut512 @ Nov 10 2007, 16:47) *
Statistically significant is that which is unlikely to occur by chance.

A statistically significant difference in this case, merely implies that statistical evidence indicates that a discernible difference has a greater probability of existing than it does a probability of not existing.

Whether that difference is due to errors in encoding, playback, or whatever, is entirely irrelevant.



If you repeatedly, 'successfully' (>16 trials each, p<0.05) ABXd flac from flac, or flac from .wav, all using the same track, then your flac encoding/decoding is broken, or your ABX test was crap. Seriously. If you understood what you are claiming, you'd know this must be true.

This post has been edited by SiriusB: Nov 12 2007, 08:26
Go to the top of the page
+Quote Post
Vitecs
post Nov 12 2007, 09:14
Post #46





Group: Members
Posts: 40
Joined: 6-November 07
Member No.: 48526



Not to mention what author is doing with his ABXing but to note people who says that ABX-ing lossless is pointless in any cases - I have one example (theoretical though): for testing decoder part. Can it happen in portable player that lack of processing power will lead to produce decoding errors in higer compression levels? Or software bugs will allow one lossless format sound better than another? You can't easily bit-compare DAP output...
Go to the top of the page
+Quote Post
cliveb
post Nov 12 2007, 09:54
Post #47


WaveRepair developer


Group: Developer
Posts: 847
Joined: 28-July 04
Member No.: 15845



QUOTE (Vitecs @ Nov 12 2007, 08:14) *
Can it happen in portable player that lack of processing power will lead to produce decoding errors in higer compression levels?

There's a heated argument that occasionally flares up over on the Slim Devices forum that's related to this speculation:

Some people claim that if they decode their FLACs on the server and stream uncompressed WAV to a Squeezebox, it sounds different than if they stream FLAC and have it decoded in the Squeezebox.

The hypothesis is that the additional CPU activity required to decode the FLAC in the Squeezebox might possibly generate some RFI that affects the analogue circuitry. (Note: there is absolutely no suggestion that the two processes result in different digital data). If you have an open mind, you have to concede that this is at least possible, however unlikely. (On the other hand, those who have taken the trouble to do a blind comparison failed to distinguish streaming FLAC and WAV).

I'm sure the same people who believe FLAC and WAV sound different in the above scenario would have no trouble speculating that the different CPU activity required to decode different levels of FLAC compression might result in different RFI which would in turn affect surrounding analogue circuitry in different ways.

My view? It seems unlikely in the extreme that any such effect (if it exists) will be audible. Life is certainly too short to bother testing it rigorously.
Go to the top of the page
+Quote Post
sld
post Nov 12 2007, 10:06
Post #48





Group: Members
Posts: 1017
Joined: 4-March 03
From: Singapore
Member No.: 5312



QUOTE (audioslut512 @ Nov 12 2007, 14:09) *
Trolling for whom?

I am just trying to understand how this stuff works and get crucified for it.

Thanks again.

I'm sorry, then.what.was.the.quote.for?

QUOTE (audioslut512 @ Nov 11 2007, 09:59) *
just forget it kids.


Wasn't it on the part of your closed mind to refuse to accept the logical explanations provided by the more knowledgeable individuals in this forum? And you blame others for rightly 'crucifying' your obstinacy?

The entire forum is still waiting for your bit-comparisons and ABX logs with p values < 0.1%.

This post has been edited by sld: Nov 12 2007, 10:10
Go to the top of the page
+Quote Post
MedO
post Nov 12 2007, 10:33
Post #49





Group: Members
Posts: 341
Joined: 24-August 05
Member No.: 24095



Well, I just noticed that even I *might* be able to ABX FLAC -4 from -8, given the CPU usage pattern explanation... you see, my CPU emits a (very low-volume) high-pitched whine when it's idle, that changes when the CPU is used. The problem is well known for the Core2-based MacBook IIRC, but also occurs on other systems including my laptop, apparently.

Well, if decoding the two files causes different CPU usage, it might very well be possible to hear a small difference because of this phenomenon. Because of its high pitch, I do hear the noise over music I play back when running Linux (it's worse there, don't ask me why), but I usually have something running in the background (Seti@Home, Prime95) to keep the processor occupied, which stops the whine.

If both files are decoded completely before playback however, I don't think it should be possible.
Go to the top of the page
+Quote Post
SamHain86
post Nov 12 2007, 11:43
Post #50





Group: Members
Posts: 432
Joined: 1-January 07
From: Luebeck, DE
Member No.: 39196



@MedO-
-When you run anythingABX test, I always found the testing program to create two WAVs of the inputted files, eliminating any decoding lags or "hiccups." I have seen this with a stand alone ABX utility, and I believe FB2K does this. There is no way, to my understanding, that this ABX ws performed properly. EDIT: It is unlikely that the processor lag or hardware strain noise could give one an advantage when performing an ABX test on two lossless sources.

This post has been edited by SamHain86: Nov 12 2007, 15:34


--------------------
OP can't edit initial post when a solution is determined :'-(
Go to the top of the page
+Quote Post

3 Pages V  < 1 2 3 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 1st November 2014 - 13:45