IPB

Welcome Guest ( Log In | Register )

3 Pages V   1 2 3 >  
Reply to this topicStart new topic
Blind test challenge, Find the SB64 recording !
Pio2001
post Feb 18 2003, 22:57
Post #1


Moderator


Group: Super Moderator
Posts: 3936
Joined: 29-September 01
Member No.: 73



Can anyone ABX any of these samples vs the original (that is resampled and leveled, but in the digital domain, all others are analog copies) ?

Please, tell me I'm either tired or deaf.

KikeG, there are soundcards analog recordings, you might be interested.

Right click to download original.flac
Right click to download 2.flac
Right click to download 3.flac
Right click to download 4.flac
Right click to download 5.flac

Less than 900 kB each
Go to the top of the page
+Quote Post
KikeG
post Feb 19 2003, 09:12
Post #2


WinABX developer


Group: Developer
Posts: 1578
Joined: 1-October 01
Member No.: 137



Nice, I'll give a try when I have some time.
Go to the top of the page
+Quote Post
Pio2001
post Feb 19 2003, 12:56
Post #3


Moderator


Group: Super Moderator
Posts: 3936
Joined: 29-September 01
Member No.: 73



I was unable to ABX any of these samples with the original, though I can hear the SB64 one. Isn't it strange ? I can hear the coloration added by the SB64, and a spectral analysis showed exactly the defects I hear in the frequency, but I got only 4/8 on ABX.

I think that ABXing the sound of such samples, that show continuous difference in the sound itself, rather than localized artifacts audible at a given place, is more difficult. The difference I hear seems to vanish after listening 4 or 5 times to the samples.

I'll try slower, maybe one session per day, to see if there is an improvement in my scores. It is possible that some differences in the sound are very difficult to get right in an ABX test with short samples played over and over.

I still can use another sample for the test (audiophiles would maybe prefer a Rebecca Pidgeon recording), and / or resample and level all analog recording to 44.1 kHz so as to have a bit perfect original to compare with various distorded files.
The later would be interesting if tested on audiophile CD players, and if the ABX is impossible (would prove that some device don't add distortion to the sound), while the current setup is better at comparing the tested devices between them (analog samples are closer to the analog source).
Go to the top of the page
+Quote Post
Patsoe
post Feb 19 2003, 13:55
Post #4





Group: Members (Donating)
Posts: 591
Joined: 11-February 03
From: UK
Member No.: 4952



I'm not really understanding what this is about. But since you sent me to this thread, I guess its about using the analog out of your soundcard with different cables?
Should we listen to the samples before knowing what they are made of? I mean, if I put them in abx-software it will still be blind even if you tell us what you did to the samples, right?

Anyway, I'm not very familiar with ABX yet, so I'll first practice with some example files before trying these, probably hardly distinguishable recordings. I'll report back in a few days, then.

QUOTE
...while the current setup is better at comparing the tested devices between them (analog samples are closer to the analog source).


I really don't get this. Could you explain some more?
Go to the top of the page
+Quote Post
KikeG
post Feb 19 2003, 17:04
Post #5


WinABX developer


Group: Developer
Posts: 1578
Joined: 1-October 01
Member No.: 137



QUOTE (Pio2001 @ Feb 19 2003 - 12:56 PM)
I was unable to ABX any of these samples with the original, though I can hear the SB64 one. Isn't it strange ? I can hear the coloration added by the SB64, and a spectral analysis showed exactly the defects I hear in the frequency, but I got only 4/8 on ABX.

I think that ABXing the sound of such samples, that show continuous difference in the sound itself, rather than localized artifacts audible at a given place, is more difficult. The difference I hear seems to vanish after listening 4 or 5 times to the samples.

If this is what is happening to you (you get good scores at first, and bad at last, differences vanish as the test goes on) you need to take the test in a more relaxed manner, without stressing too much, taking your time after every trial. Sometimes listening tireness causes this effect. On the other side, concentrating a lot can help overcoming tireness... I guess the best procedure dependes on everyone.

QUOTE
I'll try slower, maybe one session per day, to see if there is an improvement in my scores.

That could be a posible way of avoiding tireness.

QUOTE
It is possible that some differences in the sound are very difficult to get right in an ABX test with short samples played over and over.

Mmm... the important thing is if you can tell them appart reliabily in a blind manner. "Usual" ABX is only a standard procedure to achieve this, but the precise method can be "flexibilized" to suit your preferences, as long as blindness and statistical validity is assured.

QUOTE
The later would be interesting if tested on audiophile CD players, and if the ABX is impossible (would prove that some device don't add distortion to the sound)...


All analog input/output devices add distortion, the thing is how big is this distortion and if it's audible or not. This depends also on the type of music used, I'd say noisy and already distorted music is worse to detect further distortion.

This post has been edited by KikeG: Feb 19 2003, 17:15
Go to the top of the page
+Quote Post
Pio2001
post Feb 20 2003, 00:22
Post #6


Moderator


Group: Super Moderator
Posts: 3936
Joined: 29-September 01
Member No.: 73



QUOTE (Patsoe @ Feb 19 2003 - 03:55 PM)
I'm not really understanding what this is about. But since you sent me to this thread, I guess its about using the analog out of your soundcard with different cables?


I sent you there because among these samples, there is a recording of the analog output of the Marian Marc 2 soundcard, about which you asked. I can tell you in private message which one it is. This way, listening to it, you may see if there is an obvious problem with the sound for you.
The other samples are, and the subject shows, a recording of the SoundBlaster 64V PCI output, and two other ones, that are not recorded from soundcards.

QUOTE (Patsoe @ Feb 19 2003 - 03:55 PM)
Should we listen to the samples before knowing what they are made of? I mean, if I put them in abx-software it will still be blind even if you tell us what you did to the samples, right?


Yes, it would still be blind. But knowing what they are, you may try to find some expected sound characteristics in the samples. It's better to let people give a unbiased opinion rather than to ask them which sample sounds rather like this or like that, especially if we want to test, blindly, if a device really has the sound we expect it to have.
Example : let's say I hid a vinyl recording among them. If you know it, you are going to search for clicks and noise, maybe mistake a CD click or MP3 noise for them, and tell that a CD or MP3 file "clicks like a vinyl", while otherwise, you would maybe not have thought about it this way.

QUOTE (Patsoe @ Feb 19 2003 - 03:55 PM)
I'll report back in a few days, then.


Thanks in advance for anyone willing to participate, but no one is forced to do so, if you have not got the time, but are interested, you can ask me in a private message which sample is what (but I'll ask you not to reveal them until I do it myself).

QUOTE (Patsoe @ Feb 19 2003 - 03:55 PM)
QUOTE
...while the current setup is better at comparing the tested devices between them (analog samples are closer to the analog source).


I really don't get this. Could you explain some more?


I've got a problem : the external ADC I use for recording analog sources is a Sony DTC 55ES, and it doesn't support 44100 Hz sample rate. Only 32 and 48 kHz, as most early consumer DAT decks. However, the original is 44.1 kHz, since it is a CD.
So to compare original vs copies, we end up comparing 44.1 kHz and 48 kHz. This is not good, since soundcards can behave differently at different sample rates (think about resampling).

There are two solutions :
1) Resample the 48 kHz recordings to 44.1 kHz
2) Resample the original to 48 kHz.

I did the second choice, so as to provide test samples as close as possible to the analog source. This way, comparing samples between them, the differences in quality are not attenuated by a common resampling process.
On the other hand, since some samples can be difficult to ABX, the first option would have allowed to compare a soundcard recording, for example, to the unmodified original. In the case a difference is audible, it will be difficult to know if it is the soundcard's fault or the resampling to 44.1 kHz fault, but imagine that someone uses a listening system superior to the devices tested, eg a high end audiophile CD Player, reading a CDR with the samples burned on it. If he doesn't hear the difference between the bit exact original and the analog resampled copy, it would mean the the soundcard tested (for example) is really very good.

I can upload 44.1 kHz versions if someone wants to burn the files on CDR and perform the tests on a high end hifi system.
Go to the top of the page
+Quote Post
Patsoe
post Feb 20 2003, 10:40
Post #7





Group: Members (Donating)
Posts: 591
Joined: 11-February 03
From: UK
Member No.: 4952



Thanks for the thorough explanation. I'm very interested in doing this as an abx comparison, so I won't ask for the answers just now.

Just a last question though: do you think I should be able to detect a difference, since I have no equipment like a Marian Marc (not yet, that is tongue.gif)? I'm using a SB128 with Academy monitoring headphones. I could borrow a SB Live from my room mate and install it with kX drivers, but I guess that wouldn't help too much...?
Go to the top of the page
+Quote Post
Pio2001
post Feb 20 2003, 13:02
Post #8


Moderator


Group: Super Moderator
Posts: 3936
Joined: 29-September 01
Member No.: 73



You may detect the SB64 sample. The other ones will be very difficult. It would be better to perform the test on a CD player, if you've got a good one, but in this case, I should upload a 44.1 kHz version.

EDIT : I wonder if an MP3 compatible standalone DVD Player can play wav files as well (48 kHz).
Go to the top of the page
+Quote Post
Patsoe
post Feb 20 2003, 13:14
Post #9





Group: Members (Donating)
Posts: 591
Joined: 11-February 03
From: UK
Member No.: 4952



QUOTE (Pio2001 @ Feb 20 2003 - 01:02 PM)
You may detect the SB64 sample. The other ones will be very difficult. It would be better to perform the test on a CD player, if you've got a good one...

I don't have a very high-end unit. It's a Denon, type DCD-6..something (screaming front saying 20bit labda dac or so). Sorry, I would have enjoyed participating. Perhaps I'll just go out and get the Marian card tongue.gif
Go to the top of the page
+Quote Post
KikeG
post Feb 20 2003, 15:32
Post #10


WinABX developer


Group: Developer
Posts: 1578
Joined: 1-October 01
Member No.: 137



I've been able to ABX one of the samples conclusively.

I have to admit I did a little trick to save time and effort. I did a quick and not accurate comparative spectral analysis in order to know which is the file that seemed to have worse frequency response, and found one with quite many attenuation of frequencies above 20 KHz, so I assumed this was the worst one, and the one I tried to ABX.

The computer I used for the test has a not very good SoundMax built-in soundcard, and I did tests in a quite noisy environment, with headphones connected to the soundcard.

First I tried with my somehow boomy Sony MDR-7506, and seemed to hear some diferences, but I was unable to get any good ABX scores. Then I tried plugging my old Seenheiser HD560, that gave a quite more detailed picture, and started hearing some subtle but clear differences.

On two different parts of the song, I got 7/7 (p<0.8) and 14/17 (p<0.6%). I didn't try the other samples.

Pio has more details, that I will post when he thinks it's ok to do so.
Go to the top of the page
+Quote Post
KikeG
post Feb 20 2003, 16:23
Post #11


WinABX developer


Group: Developer
Posts: 1578
Joined: 1-October 01
Member No.: 137



More:

I guess I couldn't abx the file with the Sony headphones also because I was looking for a treble reduction, or something similar. Now that I know where the difference lies (I've also performed a detailed frequency response analysis), I've been able to ABX without much difficulty the file too with the Sony headphones (10/11 p<0.6%, 12/14 p<0.6%).
Go to the top of the page
+Quote Post
Continuum
post Feb 21 2003, 20:19
Post #12





Group: Members
Posts: 473
Joined: 7-June 02
Member No.: 2244



I have only tried original><file 5 yet. I was quiet confident that I heard a difference in the first 4 trials (4/4) but then the impression vanished and the results dropped.
Go to the top of the page
+Quote Post
Bedeox
post Feb 21 2003, 21:22
Post #13





Group: Members
Posts: 246
Joined: 20-December 02
From: Quite quiet place in Poland
Member No.: 4181



I got 14/15 result after half hour meditation. biggrin.gif
Without, I had only 7/15... weird, isn't it?

<edit>
I'm using headphones too...

Of course it might have been a lucky shot... I'll try to do it again.

Heh. Forgot to mention I was testing against file 5.
</edit>

This post has been edited by Bedeox: Feb 21 2003, 22:07


--------------------
I've changed only because of myself.
Remember, when you quote me, you're quoting AstralStorm.
(read: this account is dead)
Go to the top of the page
+Quote Post
voltron
post Feb 22 2003, 02:35
Post #14





Group: Members
Posts: 293
Joined: 2-August 02
Member No.: 2909



Here are my ABX results. I never want to hear those samples ever again! dry.gif






I am not sure if I even did this test correctly, but my best guess is that the SB64 is sample 2. Can someone message me the correct answer? Thank you.

System specs: Turtlebeach SantaCruz with Sony MDR-v250 Headphones.

voltron

This post has been edited by voltron: Feb 22 2003, 02:39
Go to the top of the page
+Quote Post
Pio2001
post Feb 22 2003, 03:40
Post #15


Moderator


Group: Super Moderator
Posts: 3936
Joined: 29-September 01
Member No.: 73



Thank you people !

Well, the results are not going in the direction I expected ! You have better ears than me. This test would have been conclusive if some of the samples were not ABXable at all, but maybe I'll have to ask for two more ABX (between two samples each, that I yet have to record) to draw a conclusion.

If you've got extra time, you can already try to ABX 3 vs 5. According to what is usually said by the hardcore ABXers in HA, this should be impossible.
Go to the top of the page
+Quote Post
Continuum
post Feb 22 2003, 08:44
Post #16





Group: Members
Posts: 473
Joined: 7-June 02
Member No.: 2244



QUOTE (voltron @ Feb 22 2003 - 02:35 AM)
Here are my ABX results. I never want to hear those samples ever again!  dry.gif

Uhm.. wouldn't it be easier to use the text written to the log-file?
wink.gif
Go to the top of the page
+Quote Post
KikeG
post Feb 23 2003, 23:02
Post #17


WinABX developer


Group: Developer
Posts: 1578
Joined: 1-October 01
Member No.: 137



Voltron, in order to ABX succesfully a sample you must get a minimum probability of guessing of < 5%, and <1% is better to make totally sure it's not by chance. So, the only sample you could say to have abx'ed is 4.wav.

This post has been edited by KikeG: Feb 23 2003, 23:03
Go to the top of the page
+Quote Post
KikeG
post Feb 23 2003, 23:28
Post #18


WinABX developer


Group: Developer
Posts: 1578
Joined: 1-October 01
Member No.: 137



By the way, the sample I abxed was 4.wav.

Bedeox, 14/15 is below 0.1% of guessing, I doubt you did it by chance.
Go to the top of the page
+Quote Post
Bedeox
post Feb 24 2003, 11:17
Post #19





Group: Members
Posts: 246
Joined: 20-December 02
From: Quite quiet place in Poland
Member No.: 4181



It might be chance as well (this means, 0.1% hit, very rare, but possible)
I did the test again, and it IS different. (6/7) (5.wav)

<edit>
Forgot to say that I can't ABX 3.wav vs 5.wav.
</edit>

This post has been edited by Bedeox: Feb 24 2003, 11:20


--------------------
I've changed only because of myself.
Remember, when you quote me, you're quoting AstralStorm.
(read: this account is dead)
Go to the top of the page
+Quote Post
KikeG
post Feb 25 2003, 09:48
Post #20


WinABX developer


Group: Developer
Posts: 1578
Joined: 1-October 01
Member No.: 137



I ABXed yesterday 5.wav (17/22 p= 0.8%). At first I couldn't really notice any differences, just my imagination, so I didn't get any good scores. Then, I compared spectrums of the files, and saw some differences. After that, I repeated the test focusing on the theorical differences, and started hearing them.

This post has been edited by KikeG: Feb 25 2003, 09:50
Go to the top of the page
+Quote Post
Pio2001
post Feb 27 2003, 22:40
Post #21


Moderator


Group: Super Moderator
Posts: 3936
Joined: 29-September 01
Member No.: 73



It's going to be time to tell the results, and propose further samples (the current results are little informative, due to the limited choice I made).
Patsoe, must we wait for you, or do you give up ?
Go to the top of the page
+Quote Post
Patsoe
post Feb 28 2003, 08:54
Post #22





Group: Members (Donating)
Posts: 591
Joined: 11-February 03
From: UK
Member No.: 4952



QUOTE (Pio2001 @ Feb 27 2003 - 10:40 PM)
It's going to be time to tell the results, and propose further samples (the current results are little informative, due to the limited choice I made).
Patsoe, must we wait for you, or do you give up ?

I give up I guess. I couldn't get hold of a decent soundcard this week. Thanks for waiting smile.gif
Go to the top of the page
+Quote Post
KikeG
post Feb 28 2003, 09:54
Post #23


WinABX developer


Group: Developer
Posts: 1578
Joined: 1-October 01
Member No.: 137



QUOTE (Patsoe @ Feb 28 2003 - 08:54 AM)
I couldn't get hold of a decent soundcard this week. Thanks for waiting  smile.gif

I don't think you need a very good card to detect some of the differences found here. I did it with a mediocre one, I'd say it could be more important what speakers or headphones you use.
Go to the top of the page
+Quote Post
Garf
post Feb 28 2003, 10:53
Post #24


Server Admin


Group: Admin
Posts: 4885
Joined: 24-September 01
Member No.: 13



QUOTE (KikeG @ Feb 24 2003 - 12:28 AM)
Bedeox, 14/15 is below 0.1% of guessing, I doubt you did it by chance.

It's 7/15 + 14/15 what makes 21/30, and I don't know how singificant that is.

If you do multiple tries, add them up, or the probabilities become meaningless.
Go to the top of the page
+Quote Post
Moneo
post Feb 28 2003, 11:23
Post #25





Group: Developer
Posts: 501
Joined: 22-January 03
From: Netherlands
Member No.: 4684



QUOTE (Garf @ Feb 28 2003 - 10:53 AM)
It's 7/15 + 14/15 what makes 21/30, and I don't know how singificant that is.

About 1,2%.
Go to the top of the page
+Quote Post

3 Pages V   1 2 3 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 21st September 2014 - 23:29