IPB

Welcome Guest ( Log In | Register )

10 Pages V  « < 2 3 4 5 6 > »   
Reply to this topicStart new topic
16 bit vs 24 bit, any samples that work?
Martin Kantola
post Jan 21 2009, 22:04
Post #76





Group: Members
Posts: 38
Joined: 21-January 09
Member No.: 65841



Greetings, found your forum recently, so here's my first post! Wanted to share another set of files that might be useful for listening tests. The source material is a 'raw' unprocessed master recording downsampled from 96kHz. Uploaded the files here:

Digital audio resolution test files

Below you can see a quick ABX result I got using my laptop and a pair of Beyerdynamic DT250 headphones.



The second set of files are from an old master tape, comparing 24 bit versus 12 bit versus 320kbps mp3.

Martin
Go to the top of the page
+Quote Post
Kees de Visser
post Jan 22 2009, 07:37
Post #77





Group: Members
Posts: 648
Joined: 22-May 05
From: France
Member No.: 22220



QUOTE (Martin Kantola @ Jan 21 2009, 22:04) *
Below you can see a quick ABX result I got using my laptop and a pair of Beyerdynamic DT250 headphones.
That's an interesting result. Thanks for sharing and welcome to the forum!
I've quickly checked the audio files:
The music peaks at -2.6dBFS which means it's not a low-level signal that makes it easier to hear noise floor.
AFAICT the difference between the 24 and 16 bit version is just (dither) noise.

It's a bit surprising that this difference can be ABX'ed, but your test results look valid.
How would you describe the difference that you've heard ? Is it just the noise level ?

Can anyone confirm that Foobar treats 24 and 16 bit files the same ? It might have been a good idea to convert the 16 bit file to 24 by padding the last 8 bits with zeros to exclude any additional variables.
Go to the top of the page
+Quote Post
2Bdecided
post Jan 22 2009, 13:45
Post #78


ReplayGain developer


Group: Developer
Posts: 5059
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



I can't ABX it. I agree it's a perfectly good 24>16 bit conversion.

The noise floor of the original recording comes quite close to the noise floor of the 16-bit conversion during the quiet points - watched on a spectral display.


I use this method to compare noise floors in Cool Edit Pro...
Open 24.wav
Open 16-bpt.wav
Channel Mixer - both channels = right (run on both files)
In 16-bpt.wav, select the left channel only, hit copy
In 24.wav, select the left channel only, click mix paste, select invert, 100.
Analyise: frequency analysis. 1024 fft. Linear view.

Now click play, and watch the dancing display. You can try any FFT size up to 4096 to get a dancing time/frequency display with varying time/frequency resolution. The lower curve is the 16-bit noise floor, the upper curve is the signal.

Note that, when playing, there's some averaging going on. Click stop, and put the cursor on a quiet bit of the music to see the exact FFT for that moment.


I tried running this through the last MATLAB version of lossyFLAC / lossyWAV. It doesn't have the refinements added by Nick, so it's not as trustworthy as his latest version, but it's an interesting indicator. I've plotted the number of bits it wanted to keep, and attached the graph.

As you can see, it often wanted to keep 17 or 18 bits during the quiet passages, if adding triangular dither without noise shaping. I.e. lossyWAV thinks it needs more than 16-bits to guarantee transparency.

If adding no dither (and remember lossyWAV aims to work below the noise floor, so no dither should be necessary - it's never added harmonic distortion IME), you can get away with 1 fewer bit than shown on that graph.

With Nick's refinements, and a shaped dither spectrum, it's quite possible that lossyWAV would decide 16-bits is sufficient.


Verifying the ABX

It would be helpful if Martin Kantola could describe the differences he heard, where in the file he heard them, and most importantly can verify that his laptop plays back 16-bit and 24-bit digital audio correctly.

The most important test is to check that 24-bit files are replayed correctly - if everything is truncated to 16-bits, then you are not comparing what you think! Replaying a -96dB 1kHz tone in a 24-bit file, examining the output using a high quality sound card or audio analyser, and looking for harmonic distortion, will verify true 24-bit capability, or a lack of it.

The next most important test is to verify that 16-bits padded to 24-bits with 8 zeros is still ABXable.

Cheers,
David.

This post has been edited by 2Bdecided: Jan 22 2009, 13:48
Attached thumbnail(s)
Attached Image
 
Go to the top of the page
+Quote Post
AndyH-ha
post Jan 22 2009, 22:12
Post #79





Group: Members
Posts: 2204
Joined: 31-August 05
Member No.: 24222



I ‘m going to see if I can borrow a fast connection to download, those files are just too big for this dial up.

As I specified before, only the 24 bit source is of interest. I know I can convert properly; possible audible differences with other conversions to 16 bit are of no value. Tests based on them on very suspect. I understand most anyone can do a proper conversion, given the necessary software, but a fuller specification of the particular process employed is desirable.

Blue printing on a black background is extremely difficult to read. Under most circumstances that kind of presentation is enough to make me very quickly decide the site is not worth the bother. I still haven’t figured out all the headings on this one.
Go to the top of the page
+Quote Post
Martin Kantola
post Jan 23 2009, 05:01
Post #80





Group: Members
Posts: 38
Joined: 21-January 09
Member No.: 65841



QUOTE (AndyH-ha @ Jan 22 2009, 16:12) *
I know I can convert properly; possible audible differences with other conversions to 16 bit are of no value. Tests based on them on very suspect. I understand most anyone can do a proper conversion, given the necessary software, but a fuller specification of the particular process employed is desirable.


What exactly do you need to know apart from the dither used? Please let me know. While I understand that this is slightly controversial, we have to remember that most professional audio is done in 24 bit today, probably for good reason. So we could at least suspect an audible difference. But it's a good idea to download only the source file to save download time.

QUOTE (AndyH-ha @ Jan 22 2009, 16:12) *
Blue printing on a black background is extremely difficult to read.


Oh, I'm sorry, looked fine on my screen, but fixed it now! Please reload the page. Thanks for pointing that out.

Martin




QUOTE (2Bdecided @ Jan 22 2009, 07:45) *
It would be helpful if Martin Kantola could describe the differences he heard, where in the file he heard them, and most importantly can verify that his laptop plays back 16-bit and 24-bit digital audio correctly.


While the hardware of my laptop is 24bit/192kHz capable, the specs are not exactly impressive:



Will get back to the differences I heard, have to try a 16 bit padded to 24 bits to make sure it works as it should.

Martin
Go to the top of the page
+Quote Post
Ron Jones
post Jan 23 2009, 05:58
Post #81





Group: Members
Posts: 412
Joined: 9-August 07
From: Los Angeles
Member No.: 46048



QUOTE (Martin Kantola @ Jan 22 2009, 20:01) *
While I understand that this is slightly controversial, we have to remember that most professional audio is done in 24 bit today, probably for good reason. So we could at least suspect an audible difference.

The primary reason we use 24-bit in production is due to the fact that 24-bit words allow for greater resolution of low-amplitude audio. In production, it's not rare to track or otherwise record a source at very conservative levels where the majority of the audio may only be effectively represented with 12 to 16 bits per sample so as to prevent full-scale clipping of rogue transients. If we were to record at 16 bits per sample, such audio would be only effectively represented with 6 to 8 bits per sample. This is especially true with certain types of recording, such as recording Foley, where often times there may be only one or at least very few opportunities to record a given sound, thus recording at conservative levels is critically important.

When audio is compressed, limited or normalized, we must necessarily bring up the noise floor to obtain an increase in volume (a decrease in dynamic range), so longer word lengths are extremely advantageous in this regard. Since most types of audio, music especially, end up passing through several layers of compression and limiting, working at or above 24-bit is almost mandatory (in my opinion) to achieve the highest quality result.

As an engineer, I never expect any audible difference between 16-bit and 24-bit for a compressed master. For a final delivery, where no additional processing is expected to be applied, 16-bit should almost always suffice.

That being said, I'll have a listen to the samples you've provided. The odds I'll be able to detect a difference is probably fairly slim, but I'll at least give it a go.

EDIT: Ran an ABX test. I didn't have the slightest chance, even listening at roughly 10dB higher than I would be comfortable listening to music at for any extended period of time. I focused my attention on the quietest passage, which begins at about 9.5 seconds and dives down to an RMS of about -41dBFS for about a second and a half. Absolutely nothing seems to stand out.

This post has been edited by Ron Jones: Jan 23 2009, 06:31
Go to the top of the page
+Quote Post
Martin Kantola
post Jan 23 2009, 06:23
Post #82





Group: Members
Posts: 38
Joined: 21-January 09
Member No.: 65841



QUOTE (Ron Jones @ Jan 22 2009, 23:58) *
As an engineer, I never expect any audible difference between 16-bit and 24-bit for a compressed master.


Thanks for the post, good explanation, I understand and agree with you. Please note that this is definitely not about a compressed master, but about our ability to hear differences between 16 and 24 bits in the first place. Hopefully also about what these possibly detected differences are from a musical perspective, because noise is hardly the issue. I should have explained myself better. Thanks for checking out the files.

Martin
Go to the top of the page
+Quote Post
Axon
post Jan 23 2009, 07:56
Post #83





Group: Members (Donating)
Posts: 1984
Joined: 4-January 04
From: Austin, TX
Member No.: 10933



18/32 - there's a harshness in the left channel around the 8.5 mark, which I had originally mistaken for ground hum, which I felt I could possibly distinguish in character between 16hpt and 24. I started pretty good at 5/6 but I felt like I could not hold on to the difference.

There seems to be a tremendous amount of distortion in this recording when I listen to it critically smile.gif

This post has been edited by Axon: Jan 23 2009, 07:57
Go to the top of the page
+Quote Post
2Bdecided
post Jan 23 2009, 11:47
Post #84


ReplayGain developer


Group: Developer
Posts: 5059
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



QUOTE (Martin Kantola @ Jan 23 2009, 04:01) *
While the hardware of my laptop is 24bit/192kHz capable, the specs are not exactly impressive:
I think your assumption that there's a likelihood of hearing a difference is very wide of the mark. There's a possibility, yes - but there's a stronger possibility that the sound from your laptop, like most other laptops, isn't all it's cracked up to be.

I can't find full RMAA results for what you have, but that codec on another board, tested at 16-bits, wasn't exactly spectacular...

http://www.hardwareluxx.de/Images/Alderwoo...ar%20output.htm


The first thing to demonstrate is that you're getting 16-bits out without distortion, and 24-bits out without distortion (at least down to ~ 18-20 bits). Having a noise floor substantially below the 18th-20th bit would help too.

That test lists 70dB SNR which is nominally (though not accurately) equal to about 12 bits.

Cheers,
David.
Go to the top of the page
+Quote Post
Martin Kantola
post Jan 23 2009, 16:11
Post #85





Group: Members
Posts: 38
Joined: 21-January 09
Member No.: 65841



QUOTE (2Bdecided @ Jan 23 2009, 05:47) *
I think your assumption that there's a likelihood of hearing a difference is very wide of the mark. There's a possibility, yes - but there's a stronger possibility that the sound from your laptop, like most other laptops, isn't all it's cracked up to be.


Thank you for looking into this. What I'm doing is putting that 'assumption' of mine to the test with ABX. While I agree that my laptop sound is not anywhere near 24-bit performance, we can't ignore that there are several types of both distortion and noise. It's been pointed out that speaker distortion is very high in level, yet we can detect distortion of other kinds listening through one. FWIW, there are two fellow forumers at Gearslutz who scored a full 10/10 on this test, but they probably had much better equipment. You should know that I found testing with my laptop quite a challenge. It was anything but easy, and required a lot of focusing.

Martin
Go to the top of the page
+Quote Post
2Bdecided
post Jan 23 2009, 16:30
Post #86


ReplayGain developer


Group: Developer
Posts: 5059
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



Found it...

http://www.gearslutz.com/board/so-much-gea...ot-audible.html

...interesting thread. Some amazing miscomprehension in there, but still interesting.

I'm not saying you can't hear it. My own algorithm (lossyWAV) says you can. (But I can't!). It's just that 90%+ of people who come to HA claiming to hear a difference between 16-bits and 24-bits, or 44.1kHz and 96kHz, are really hearing problems with their equipment.

Potential equipment "problems" aside, would you be willing to try another ABX or two?

Cheers,
David.

This post has been edited by 2Bdecided: Jan 23 2009, 16:51
Go to the top of the page
+Quote Post
2Bdecided
post Jan 23 2009, 17:25
Post #87


ReplayGain developer


Group: Developer
Posts: 5059
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



Here you go...

http://rapidshare.de/files/43321357/16-hpt-24-n.wav.html

Your 16 bit file, converted back to 24-bits, with a little high-pass noise added to make sure the bottom 8 bits are moving.

Is it better or worse than your 16-bit version?
(where better = closer to the 24-bit original!)

Cheers,
David.
Go to the top of the page
+Quote Post
Martin Kantola
post Jan 23 2009, 18:41
Post #88





Group: Members
Posts: 38
Joined: 21-January 09
Member No.: 65841



QUOTE (2Bdecided @ Jan 23 2009, 10:30) *
My own algorithm (lossyWAV) says you can. (But I can't!).


Have always thought that if you can hear a difference (as long as there is one) depends on a number of factors:

1. the source material
2. the equipment and room
3. you critical listening skill or experience
4. your ability to find and focus on the artifacts (this can change from day to day)

In other words, by having an excellent playback system and good acoustics in a relaxed environment, and possibly by discussing the differences, we can most likely maximize the number of people that detect a difference.

For me, the point it not if most people hear it or not. This is all about how sensitive our hearing is or can be, and how we can keep improving the sonic experience. To some extent questioning the quality of both our hearing mechanism and digital audio from a musical perspective. I hear a lot of claims that digital is so linear and accurate while our ears are so non-linear and inaccurate, but isn't that a simplified view and an attempt to compare two things of completely different nature?

Just because mp3s are today good enough for most people and applications, we don't have to give up our quest for better sound. Look at this way, the day we can record any musical performance and play it back with such quality that the playback cannot be distinguished from the original performance by anyone, then we can rest... Not saying that's what music recording should be about though...

Martin
Go to the top of the page
+Quote Post
krabapple
post Jan 24 2009, 01:01
Post #89





Group: Members
Posts: 2181
Joined: 18-December 03
Member No.: 10538



QUOTE (2Bdecided @ Jan 23 2009, 10:30) *
Found it...

http://www.gearslutz.com/board/so-much-gea...ot-audible.html

...interesting thread. Some amazing miscomprehension in there, but still interesting.

I'm not saying you can't hear it. My own algorithm (lossyWAV) says you can. (But I can't!). It's just that 90%+ of people who come to HA claiming to hear a difference between 16-bits and 24-bits, or 44.1kHz and 96kHz, are really hearing problems with their equipment.

Potential equipment "problems" aside, would you be willing to try another ABX or two?

Cheers,
David.



Let me get this straight... comparisons were 24bit vs dithered-to-16, and another was 24-bit .wav , converted to 16 vs 320 mp3 made directly from the 24 bit .wav ...and at least TWO people in this tiny sample of subjects managed perfect or near-perfect ABX scores on both -- with the mp3 trial reported being HARDER to ABX than the wordlength comparison??? (Except for the THIRD guy who claims he aced the 320 lossy vs lossless ABX, while failing to score significantly on the wordlength ABX). And another guy got 10/10 on just the wordlength ABX?

But Martin K can't successfully ABX two files made using Chris Johnson's superspecial dither?


blink.gif


I'm not implying nefarious intent on Martin K.s part, but to me these data suggest an obvious control: someone here needs to replicate the sample pairs (take the original 24-bit .wav and convert to 16 bit with TPDF dither; take the 24-bit file and convert it to 320 mp3), and see how those guys 'over there' do at ABX'ing them.

This post has been edited by krabapple: Jan 24 2009, 01:41
Go to the top of the page
+Quote Post
Martin Kantola
post Jan 24 2009, 03:11
Post #90





Group: Members
Posts: 38
Joined: 21-January 09
Member No.: 65841



QUOTE (krabapple @ Jan 23 2009, 19:01) *
But Martin K can't successfully ABX two files made using Chris Johnson's superspecial dither?


My result for the "avd" dither was 8/10, as posted in the thread. And yes, I really want to see more people trying this too! Regardless of outcome. But so far many many posts, but very few ABX results.

You are welcome scrutinize my conversions, I'd be happy to correct any mistakes I might have made, but these are very basic operations on a DAW, honestly think I can handle them.

Martin
Go to the top of the page
+Quote Post
Martin Kantola
post Jan 24 2009, 07:13
Post #91





Group: Members
Posts: 38
Joined: 21-January 09
Member No.: 65841



QUOTE (2Bdecided @ Jan 23 2009, 11:25) *
Here you go...

http://rapidshare.de/files/43321357/16-hpt-24-n.wav.html

Your 16 bit file, converted back to 24-bits, with a little high-pass noise added to make sure the bottom 8 bits are moving.

Is it better or worse than your 16-bit version?
(where better = closer to the 24-bit original!)


Hi David,

took a different approach with these two, hopefully you will find it useful.

Not exactly sure what that 'little high-pass noise' you added would do, so I just started out listening and guessing. This was my one and only attempt at the files, and hopefully you can see how the guessing slowly gets better in the log file. Tried not to think too much, just go by feel and keep going (with some breaks in between). By the time I had reached 100 attempts, the probability of guessing was down to 1% according to the computer.

This time it was much more random as I didn't have the energy to focus properly. It's one thing to stay sharp for 10 attempts, a different story with 100. Even after I found something to pay attention to, my concentration failed and I made many mistakes. It is very hard to do this on the laptop.





Here's the complete log:

foo_abx 1.3.3 report
foobar2000 v0.9.6.1
2009/01/23 22:20:10

File A: C:\Documents and Settings\Valued Customer\Desktop\TEST\samples\16-hpt-24-n.wav
File B: C:\Documents and Settings\Valued Customer\Desktop\TEST\samples\24.wav

22:20:10 : Test started.
22:37:01 : 01/01 50.0%
22:37:45 : 01/02 75.0%
22:38:28 : 02/03 50.0%
22:39:19 : 03/04 31.3%
22:40:15 : 04/05 18.8%
22:42:27 : 05/06 10.9%
22:47:28 : 05/07 22.7%
22:48:01 : 06/08 14.5%
22:50:24 : 06/09 25.4%
22:50:51 : 07/10 17.2%
22:51:23 : 07/11 27.4%
22:58:36 : 07/12 38.7%
22:59:43 : 07/13 50.0%
23:03:40 : 08/14 39.5%
23:03:59 : 08/15 50.0%
23:04:17 : 08/16 59.8%
23:04:25 : 08/17 68.5%
23:04:39 : 09/18 59.3%
23:04:53 : 09/19 67.6%
23:05:04 : 10/20 58.8%
23:05:28 : 10/21 66.8%
23:05:46 : 10/22 73.8%
23:06:45 : 11/23 66.1%
23:07:01 : 12/24 58.1%
23:07:13 : 13/25 50.0%
23:07:46 : 14/26 42.3%
23:07:59 : 14/27 50.0%
23:08:19 : 14/28 57.5%
23:09:20 : 14/29 64.4%
23:09:33 : 15/30 57.2%
23:11:16 : 16/31 50.0%
23:11:30 : 16/32 57.0%
23:32:28 : 17/33 50.0%
23:32:51 : 18/34 43.2%
23:33:21 : 19/35 36.8%
23:33:45 : 20/36 30.9%
23:34:27 : 21/37 25.6%
23:35:06 : 21/38 31.4%
23:35:35 : 22/39 26.1%
23:35:46 : 22/40 31.8%
23:36:04 : 22/41 37.8%
23:36:16 : 23/42 32.2%
23:36:25 : 24/43 27.1%
23:36:41 : 25/44 22.6%
23:37:59 : 26/45 18.6%
23:38:33 : 27/46 15.1%
23:38:40 : 28/47 12.1%
23:38:50 : 29/48 9.7%
23:40:29 : 30/49 7.6%
23:41:12 : 31/50 5.9%
23:42:28 : 31/51 8.0%
23:42:51 : 32/52 6.3%
23:43:03 : 33/53 4.9%
23:43:56 : 34/54 3.8%
23:46:56 : 35/55 2.9%
23:48:04 : 36/56 2.2%
23:48:59 : 36/57 3.1%
23:49:54 : 37/58 2.4%
23:50:23 : 37/59 3.4%
23:50:53 : 38/60 2.6%
23:51:51 : 38/61 3.6%
23:52:01 : 39/62 2.8%
23:53:02 : 39/63 3.8%
23:54:50 : 40/64 3.0%
23:55:01 : 40/65 4.1%
23:55:31 : 41/66 3.2%
23:55:42 : 42/67 2.5%
23:58:10 : 43/68 1.9%
00:00:02 : 43/69 2.7%
00:00:53 : 43/70 3.6%
00:01:34 : 44/71 2.8%
00:02:14 : 44/72 3.8%
00:02:32 : 45/73 3.0%
00:02:41 : 45/74 4.0%
00:02:53 : 46/75 3.2%
00:03:16 : 47/76 2.5%
00:03:32 : 48/77 2.0%
00:03:54 : 49/78 1.5%
00:05:15 : 50/79 1.2%
00:06:28 : 50/80 1.6%
00:06:38 : 51/81 1.3%
00:07:45 : 51/82 1.8%
00:08:06 : 52/83 1.4%
00:08:36 : 52/84 1.9%
00:10:12 : 52/85 2.5%
00:10:23 : 52/86 3.3%
00:10:50 : 53/87 2.7%
00:11:06 : 54/88 2.1%
00:11:25 : 54/89 2.8%
00:11:57 : 55/90 2.2%
00:12:35 : 56/91 1.8%
00:13:14 : 56/92 2.4%
00:13:40 : 57/93 1.9%
00:14:03 : 58/94 1.5%
00:14:51 : 59/95 1.2%
00:15:14 : 59/96 1.6%
00:15:51 : 60/97 1.2%
00:16:29 : 60/98 1.7%
00:16:41 : 61/99 1.3%
00:17:31 : 62/100 1.0%
00:18:38 : Test finished.

----------
Total: 62/100 (1.0%)


Martin
Go to the top of the page
+Quote Post
2Bdecided
post Jan 24 2009, 21:09
Post #92


ReplayGain developer


Group: Developer
Posts: 5059
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



Thank you Martin.

Putting noise in the last 8 bits actually removes most of the potential "equipment" issues, even though the resulting file is mathematically even further from the 24-bit original than the pure 16-bit version.

So I now think it's very likely that you're hearing a genuine difference, rather than an equipment artefact (unless it's quite an exotic one).


QUOTE (krabapple @ Jan 24 2009, 00:01) *
I'm not implying nefarious intent on Martin K.s part, but to me these data suggest an obvious control: someone here needs to replicate the sample pairs (take the original 24-bit .wav and convert to 16 bit with TPDF dither; take the 24-bit file and convert it to 320 mp3), and see how those guys 'over there' do at ABX'ing them.
I haven't even looked at the 320kbps mp3, but the 16-bit from 24-bit version seems fine - it's exactly what it says it is, no more, no less.

I agree we need to do what you suggest: try another known 24>16 conversion.

However, like I said, the noise floor in that 24-bit file does dip very close to the theoretical limit of 16-bits - such that, in some blocks, lossyWAV wants to keep 18 bits of the 24-bit original (if converting with triangular dither).


Cheers,
David.

This post has been edited by 2Bdecided: Jan 24 2009, 21:12
Go to the top of the page
+Quote Post
krabapple
post Jan 25 2009, 00:08
Post #93





Group: Members
Posts: 2181
Joined: 18-December 03
Member No.: 10538



QUOTE (Martin Kantola @ Jan 23 2009, 21:11) *
QUOTE (krabapple @ Jan 23 2009, 19:01) *
But Martin K can't successfully ABX two files made using Chris Johnson's superspecial dither?



My result for the "avd" dither was 8/10, as posted in the thread. And yes, I really want to see more people trying this too! Regardless of outcome. But so far many many posts, but very few ABX results.



An 8/10 result does not support the hypothesis of 'audible difference' at a p<0.05. You guys should keep the 'bino_dist' chart on this page handy:

http://www.kikeg.arrakis.es/winabx/

This post has been edited by krabapple: Jan 25 2009, 00:09
Go to the top of the page
+Quote Post
Martin Kantola
post Jan 25 2009, 00:28
Post #94





Group: Members
Posts: 38
Joined: 21-January 09
Member No.: 65841



QUOTE (krabapple @ Jan 24 2009, 18:08) *
An 8/10 result does not support the hypothesis of 'audible difference' at a p<0.05.


No, it means a 5.5% probability I'm only guessing as the software already tells me. While I could try again to see if the number goes up or down, I'd prefer to see somebody else take the test. Not only because it's such hard work but to get a better idea of how audible it is or isn't.

Martin
Go to the top of the page
+Quote Post
2Bdecided
post Jan 25 2009, 17:38
Post #95


ReplayGain developer


Group: Developer
Posts: 5059
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



Remember those stats are only true for the situation where you decide on (and fix) the number of trials before hand.

Sitting there, testing, and stopping when you feel like it (e.g when you get a good result), breaks the statistics.

Taking Multiple ABX tests, and only reporting one, or making false starts and re-starting, both also break the statistics.

This isn't what you've done here, but these things have been hinted at by others. They're complete no-no's.

Cheers,
David.
Go to the top of the page
+Quote Post
Martin Kantola
post Jan 25 2009, 17:59
Post #96





Group: Members
Posts: 38
Joined: 21-January 09
Member No.: 65841



Thanks for the info David, I'm completely new to ABX but thanks to the available software it was easy to try it out. As for the procedure, what I did with the other samples was to take plenty of time to study the material first, and then take a test with exactly ten trials. The recent one I did differently, plunged straight into it and did 100 trials. Don't know if these methods are correct, but the importance of being able to find and focus on the difference is essential in audio if you ask me.

Anyway, the OT was about samples that work for testing 16 vs 24 bits. Might add some more material to the same little webpage at some point. Would be helpful to hear comments on how well the samples work for some of you guys.

Martin
Go to the top of the page
+Quote Post
MLXXX
post Jan 27 2009, 10:16
Post #97





Group: Members
Posts: 172
Joined: 25-February 08
From: Australia
Member No.: 51585



QUOTE (Martin Kantola @ Jan 24 2009, 12:11) *
My result for the "avd" dither was 8/10, as posted in the thread. And yes, I really want to see more people trying this too! Regardless of outcome. But so far many many posts, but very few ABX results.

You are welcome scrutinize my conversions, I'd be happy to correct any mistakes I might have made, but these are very basic operations on a DAW, honestly think I can handle them.

Martin


Martin, congrats for posting the test files, and for the persistence in doing the ABX trials; particularly the 100 trials of the version with 'a little high pass noise'!

I have not tried to ABX the 24bit original against the 16 bit dithered versions (with or without extra noise) because in the past I have found that 24 bit music dithered to 16 bits (with a noise shaped dither) sounds the same to me as the original 24 bits, at normal recording and playback levels.

The effort required for ABX testing can be considerable. In my own case, my ears/brain are most responsive the first time I am presented an AB comparison. If the passage is repeated, my hearing discrimination rapidly diminishes. So to commit myself to an ABX comparison of very similar files I have to set aside a fair stretch of time, to allow my hearing to regain its sensitivity to fine differences.

Before attempting a serious ABX I would like to be assured that the 16 bit dithered version has been derived using methodolgy to the satisfaction of HA Forum technical gurus, e.g. a dither conversion they have run themselves, and preferably that at least one person has successfully ABXd such an 'approved' dithered version. Perhaps that is laziness on my part!

No disrespect intended to yourself, but I can foresee reluctance of the HA community to accept any ABX test result unless all possible perceived doubt has been removed as to the reliability of the methodolgy for performing the dither.

[Out of interest, I did ABX tests of just 4 trials of the truncated version, and 4 trials of the 320kbps mp3 version, relative to the respective 24 bit originals. I found the truncated version relatively easy to distinguish. But the 320kpbs version needed a lot of concentration and 'refresher breaks'. The 4 correct answers I obtained in both cases merely corresponded to a 6.2% likelihood of not guessing. But all I was seeking was broad assurance I could distinguish the files. They were not what I'd call 'serious' ABX tests.]

Cheers

This post has been edited by MLXXX: Jan 28 2009, 12:56
Go to the top of the page
+Quote Post
krabapple
post Jan 27 2009, 18:57
Post #98





Group: Members
Posts: 2181
Joined: 18-December 03
Member No.: 10538



QUOTE (MLXXX @ Jan 27 2009, 04:16) *
The effort required for ABX testing can be considerable. In my own case, my ears/brain are most responsive the first time I am presented an AB comparison. If the passage is repeated, my hearing discrimination rapidly diminishes. So to commit myself to an ABX comparison of very similar files I have to set aside a fair stretch of time, to allow my hearing to regain its sensitivity to fine differences.


Is it the repetition of A/B comparison itself, or the repetition of the same music in a series of A/B comparisons, that you find fatigueing?

If it's the latter, I wonder if it might make discrimination easier, to prepare a batch of 16 different samples -- and use a different one for each of 16 trials. Then at least each A/B is 'fresh' to the ears.

For ease of preparation, the samples could be 16 different parts of the same track. If that's not a concern, they could be from different tracks (all prepared and vetted to HA standards as you have described)

This post has been edited by krabapple: Jan 27 2009, 18:57
Go to the top of the page
+Quote Post
MLXXX
post Jan 28 2009, 14:14
Post #99





Group: Members
Posts: 172
Joined: 25-February 08
From: Australia
Member No.: 51585



QUOTE (krabapple @ Jan 28 2009, 03:57) *
Is it the repetition of A/B comparison itself, or the repetition of the same music in a series of A/B comparisons, that you find fatigueing?

Having not tried the approach you suggest of setting up multiple different parts of the same track in advance I am not absolutely sure.

However I do find that once I have become desensitized by listening to one segment of a track too many times (this may be just 4 times), the use of a different segment only offers a temporary period of being able to discriminate. After that I may find I cannot discriminate any segments of any tracks! (unless of course a segment contains a much more marked difference).

Another factor is that I have found with experience that I cannot just rely on my memory of A and B when listening to X and Y. For example, at the beginning of a listening session I may perceive a lossy Dolby Digital version as somewhat thin -- not as full sounding as an uncompressed version. But later in the session the Dolby Digital may start to sound clearer than the uncompressed version (Dolby can have a "neat and tidy" sound), and the uncompressed version may sound thin in comparison.

So before commiting to a proposed answer I find now I have to take the precaution of replaying the A and B to make sure my hearing has not altered. This adds to the listening load per entered ABX response.

In the past, I found that on some occasions I would enter up to 5 incorrect answers in a row. Quite frustrating! This I eventually realized was because I was relying on my memory of the difference in sound of A and B (e.g. A is the "brighter" sound). During the course of the ABX trial, the fatigue had caused a juxtaposition in the relative perceived sound of A and B.

In the case of 'brightness", I could conjecture that what initially sounded "brighter" may have been brighter because of the presence of high frequency content towards the upper limit of my hearing. As time wore on, and with my ears being exposed to a great deal of high frequency content, my hearing in that range may have become desensitized, so that the character of "brightness" was lost.

These sorts of extraneous effects might normally go completely unnoticed. It's just the samples we try to ABX are sometimes very close to being indistinguishable, so that even minor changes in our hearing as a session progresses have the potential to lead us into error.

This post has been edited by MLXXX: Jan 28 2009, 17:42
Go to the top of the page
+Quote Post
AndyH-ha
post Jan 29 2009, 11:05
Post #100





Group: Members
Posts: 2204
Joined: 31-August 05
Member No.: 24222



This was asked more than once but there was no response. Usually, when there is some small, generally hard to detect difference between audio samples, it comes down to something specific, some nameable artifact or distortion, or whatever, at one or several points. For instance, a particular mp3 encoding may sound identical to the original except when a triangle is struck (or whatever) -- the rest of the track is not differentiable.

Is Martin’s claim for a difference here based solely on ABX results? Is there nothing that is consciously identifiable? My fairly casual 24 vs 16 comparisons have noticed nothing so far. As Martin observes, doing much ABX testing is a lot of work. Having a target might make it more tolerable. I don’t see that as invalidating the test, since it does not allow any way around the fact that I must still chose based only on the sound, regardless that I may be waiting for a particular sound, no?

I note that Axon identified some “harshness” at “the 8.5 mark.” does this mean at 8.5 seconds from the beginning? What does the “18/32 -“ at the beginning of that line signify?

Go to the top of the page
+Quote Post

10 Pages V  « < 2 3 4 5 6 > » 
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 24th July 2014 - 22:21