IPB

Welcome Guest ( Log In | Register )

> Upload forum rules

- No over 30 sec clips of copyrighted music. Cite properly and never more than necessary for the discussion.


- No copyrighted software without permission.


- Click here for complete Hydrogenaudio Terms of Service

5 Pages V   1 2 3 > »   
Reply to this topicStart new topic
DSD-2-PCM -- proof of concept, test sample and source code here
SebastianG
post Oct 8 2005, 23:55
Post #1





Group: Developer
Posts: 1318
Joined: 20-March 04
From: Göttingen (DE)
Member No.: 12875



The ZIP contains 4 files:
- dsd2pcm.jar (the converter written in Java, outputs raw 24/88 PCM, intel byte order)
- info.txt (contains some infos)
- test2822k.dsd (14 seconds, mono DSD, 5 megs)
- test44k.mp3 (conversion result)

have fun,
Sebi

Edit-2011: For the latest C/C++ source code see https://code.google.com/p/dsd2pcm/ It is released under the new BSD license. The attatchment to this post is still the first Java release.



This post has been edited by SebastianG: Dec 4 2011, 17:45
Attached File(s)
Attached File  dsd2pcm.zip ( 3.56MB ) Number of downloads: 2847
 
Go to the top of the page
+Quote Post
skamp
post Oct 9 2005, 01:27
Post #2





Group: Developer
Posts: 1443
Joined: 4-May 04
From: France
Member No.: 13875



Now you need to develop a DVD-ROM driver for reading SACD's and find a way to crack the copy protection schemes, and we're all set! You don't have a choice anymore.
wink.gif


--------------------
See my profile for measurements, tools and recommendations.
Go to the top of the page
+Quote Post
SebastianG
post Oct 9 2005, 10:32
Post #3





Group: Developer
Posts: 1318
Joined: 20-March 04
From: Göttingen (DE)
Member No.: 12875



QUOTE (skamp @ Oct 9 2005, 01:27 AM)
Now you need to develop a DVD-ROM driver for reading SACD's and find a way to crack the copy protection schemes, and we're all set! You don't have a choice anymore.
wink.gif
*


Intercepting the raw DSD stream between an SACD player's decoder and its DAC would probably be easier. (I've no experience on this subject.)

Sebi
Go to the top of the page
+Quote Post
SebastianG
post Oct 9 2005, 11:06
Post #4





Group: Developer
Posts: 1318
Joined: 20-March 04
From: Göttingen (DE)
Member No.: 12875



Oh, I noticed a small bug: The dsd file size must be divisible by 4. This is due to the 2nd stage wanting to "convert" 4 samples to one (4:1 decimator) -- the first stage turns each byte into a float sample (8:1 decimation).

Cheers,
Sebi
Go to the top of the page
+Quote Post
tarsier
post Dec 24 2006, 20:33
Post #5





Group: Members
Posts: 22
Joined: 26-December 05
Member No.: 26654



So is it theoretically possible to convert the PCM output back into a DSD stream so that it matches the original DSD stream? In other words is a DSD->PCM->DSD conversion theoretically lossless?
Go to the top of the page
+Quote Post
putanik
post Dec 24 2006, 22:54
Post #6





Group: Members
Posts: 21
Joined: 18-December 06
Member No.: 38778



QUOTE (tarsier @ Dec 24 2006, 13:33) *
So is it theoretically possible to convert the PCM output back into a DSD stream so that it matches the original DSD stream? In other words is a DSD->PCM->DSD conversion theoretically lossless?


see http://focus.ti.com/lit/ds/symlink/pcm1796.pdf - it does both PCM and DSD "good enough".
Go to the top of the page
+Quote Post
tarsier
post Dec 26 2006, 16:28
Post #7





Group: Members
Posts: 22
Joined: 26-December 05
Member No.: 26654



Thanks for the link. But right now, I'm only concerned with the theory at the moment. There are many people who claim that DSD is superior to PCM (they say it sounds better). But on the vast majority of converter chips that give you a PCM signal, the conversion is a 1-bit delta-sigma stream that is converted to PCM. DSD is just that 1-bit stream before the conversion.

In discussions I've had on this topic, I say that there isn't any difference between DSD and PCM if the conversion to PCM (in other words the delta-sigma stream conversion to pcm) is done with care. What I don't know is if theoretically and mathematically, is the conversion actually lossless. So is it theoretically possible (not just good enough for audio) to go from DSD to PCM and then back to DSD without losing any information? If so, it seems that the DSD/PCM discussions are moot, and it all boils down to how good the actual implementation is.

And is there a better forum to post this in? I just posted here due to the topic of DSD->PCM conversion.
Go to the top of the page
+Quote Post
Kees de Visser
post Dec 26 2006, 19:03
Post #8





Group: Members
Posts: 677
Joined: 22-May 05
From: France
Member No.: 22220



QUOTE (tarsier @ Dec 26 2006, 16:28) *
In discussions I've had on this topic, I say that there isn't any difference between DSD and PCM if the conversion to PCM (in other words the delta-sigma stream conversion to pcm) is done with care.
DSD and PCM exist in various formats so you'd have to be a bit more specific. "DSD" is usually 64Fs DSD but there are also 128Fs converters. When DSD is converted to PCM for editing/mixing purposes 24bit 352.8 kHz seems to be a popular choice (Pyramix workstation e.g.). Still, some recording engineers claim (no proof) that there is an audible difference (example). Another option is 8 bit 2.8 MHz PCM (Sonoma and Sadie workstation). My guess is that when the conversion involves sample rate and/or wordlength reduction, it can't be considered lossless. Most DSD users I know don't think the quality "loss" is something to worry about.
I gladly leave the final answer to the mathematical experts on this forum.
Go to the top of the page
+Quote Post
andyshedd
post Jan 8 2007, 22:44
Post #9





Group: Members
Posts: 7
Joined: 1-February 06
Member No.: 27484



I'm mildly curious how this method compares and contrasts to Sony's own SBM Direct conversion system for DSD, bit and sample rate levels aside--in terms of perceived sound quality. In my experience SBM Direct-encoded CDs are among the least fatiguing and irritating redbook cds that I have heard, especially with players that are prone to jitter. But if this system preserves more of the audio information of the original DSD recording then that benefit may be offset, assuming an appropriate D-to-A converter is used for playback of the converted recording.
Go to the top of the page
+Quote Post
frenchglen
post Jun 8 2009, 15:23
Post #10





Group: Members
Posts: 34
Joined: 23-July 07
Member No.: 45570



Is there any updated info about this program? I tried it, and it worked. Seems useful.

It would be interesting if it could support stereo, and support .DFF, which afaik is pretty much the same as .DSD, perhaps interchangeable. There's test DFF files at http://www.2l.no/hires/index.html.

Or even more useful, DSF files. with Sony DSD Direct software you can convert stereo PCM to DSF, but there's no program (I know of) to convert back to PCM.
Go to the top of the page
+Quote Post
LukeS
post Nov 23 2009, 15:56
Post #11





Group: Members
Posts: 12
Joined: 7-September 09
Member No.: 72985



I am bringing this back form years of being dead. I am working on hardware now to rip a SACD, I am capturing the raw unencrypted 6 mono DSD data streams that is sent to the DACs in my denon SACD player. I have some test data captured that I want to run through the DSD to PCM software but I am unable to read the sample DSD file provide with the program after it is converted to PCM. Once I get the test file confirmed I can move on to testing my captured data.

I really hope SebastianG is still around. Does anyone know what format the outputted PCM file from the program is in 16-bit, or 32-bit? If 32 bit what IEEE float (0.24, 16.8, 24.0). Is there an offset? Is the byte order little-endian or big endian? I used your program to convert the sample test2822k.dsd to a pcm and am trying to open it an Audacity and cooledit pro with no luck. I will keep messing with the options and hopefully stumble upon the right setting.

Thanks!
Luke

This post has been edited by LukeS: Nov 23 2009, 15:58
Go to the top of the page
+Quote Post
GeSomeone
post Nov 23 2009, 19:12
Post #12





Group: Members
Posts: 921
Joined: 22-October 01
From: the Netherlands
Member No.: 335



I think it's in the first post.
QUOTE (SebastianG @ Oct 9 2005, 00:55) *
outputs raw 24/88 PCM, intel byte order)

Go to the top of the page
+Quote Post
SebastianG
post Nov 23 2009, 20:31
Post #13





Group: Developer
Posts: 1318
Joined: 20-March 04
From: Göttingen (DE)
Member No.: 12875



QUOTE (GeSomeone @ Nov 23 2009, 19:12) *
I think it's in the first post.
QUOTE (SebastianG @ Oct 9 2005, 00:55) *
outputs raw 24/88 PCM, intel byte order)


Correct. The info.txt file says it all.
Input: raw mono DSD where the bytes' bits are processed from LSB to MSB. The file size must be a multiple of 4.
Output: raw 24bit PCM at 88.2 kHz in little endian format.

The decimation is done in two stages:
1. 8:1 (from 44100*64 to 44100*8 Hz)
2. 4:1 (from 44100*8 to 44100*2 Hz)

If you leave out the 2nd stage you basically have what's known as DXD (PCM at 352kHz sampling frequency) which seems to be popular as an intermediate/processing format.

If you test it and the result sounds too noisy chances are that the bit order of your file and the expected bit order doesn't match. If you know a little about hacking code it should be easy to change it. I already have a table in there that for reversing the bits in a byte, for example.

The implementation of the first stage is a little tricky. Instead of 8 MACs (multiply-and-accumulate) for 8 bits in a byte I just use one table lookup. The bitreversal is needed anyway to exploit the FIR filter's impulse response's symmetry and reduce the size of the tables. If someone wants to code something similar in C I can recommend this approach.

Cheers,
SG

This post has been edited by SebastianG: Nov 23 2009, 20:37
Go to the top of the page
+Quote Post
LukeS
post Nov 23 2009, 22:32
Post #14





Group: Members
Posts: 12
Joined: 7-September 09
Member No.: 72985



SebastianG,

Awesome job on the program, my problem was that I was setting 16 or 32 bit depth, not 24, Duh! I got it to work now with signed 24-bit, little-endian (intel).

I ripped 28 seconds (legal to post) of the Norah Jones - Come away with me - 04 - Feelin' The Same Way which I will attach. The output 88200Hz PCM has a hiss and static on it as you mentioned in your post. Is there a command line switch to output at the first stage 352kHz

I am much more of a hardware person then software and when I do software I do low level C not java so I may not be much use on the code front but I would love to take a look for fun. I will try an figure out how to extract the jar, look at the code, and see if I can tinker with it.
Attached File(s)
Attached File  Norah_Joens___04___Feelin__The_Same_Way.zip ( 6.44MB ) Number of downloads: 519
 
Go to the top of the page
+Quote Post
vincefalks
post Nov 24 2009, 00:17
Post #15





Group: Members
Posts: 37
Joined: 23-August 09
Member No.: 72571



I am not an expert but it would be cool to compare to other DSD to PCM converters out there (if possible). There's a DSD plugin for WMP which concerts stereo DSDIFF/DSF to 16-bit 44.1kHz.

As I understand it there's many options you can have when converting DSD to PCM and several different approaches, some good, some not as good.

This post has been edited by vincefalks: Nov 24 2009, 00:18
Go to the top of the page
+Quote Post
rpp3po
post Nov 24 2009, 02:48
Post #16





Group: Developer
Posts: 1126
Joined: 11-February 03
From: Germany
Member No.: 4961



What would be a higher quality approach than 1. converting to 32 bit samples 2. low passing 3. downsampling?

Of course, the choice of a specific low pass filter implies the usual trade-offs (delay, ringing, aliasing, ...). But your actual choice for DSD2PCM probably wouldn't look different than the one you would pick for a regular PCM2PCM decimation, for example 96 kHz -> 48 kHz, to the same target sample rate. Or not?

This post has been edited by rpp3po: Nov 24 2009, 04:15
Go to the top of the page
+Quote Post
SebastianG
post Nov 24 2009, 10:38
Post #17





Group: Developer
Posts: 1318
Joined: 20-March 04
From: Göttingen (DE)
Member No.: 12875



QUOTE (LukeS @ Nov 23 2009, 22:32) *
The output 88200Hz PCM has a hiss and static on it as you mentioned in your post. Is there a command line switch to output at the first stage 352kHz

No. That would not even help because the bit order affects the first stage. You could try to add a switch to enable/disable the "pre stage 1 bitreversal".

QUOTE (LukeS @ Nov 23 2009, 22:32) *
I will try an figure out how to extract the jar, look at the code, and see if I can tinker with it.

Just unzip it. .jar files are really .zip files with some added meta informations.

rpp3po is right. There's not a lot you can do differently in a mathematical sense than to pick other low pass filters. (Of course, the implementation might differ.) I even documented the properties of the two low pass filters. The paranoid user should be pleased with the 170 dB (IIRC) stopband rejection of the first stage. The lowpass filter's roll off starts at something around 25 kHz (IIRC). That's IMHO high enough because there's already an awful lot of noise just above it. (The DSD propaganda tells you that DSD has a "great dynamic range". That only applies to frequencies below 20kHz which is not surprizing).

Cheers,
SG

This post has been edited by SebastianG: Nov 24 2009, 10:41
Go to the top of the page
+Quote Post
2Bdecided
post Nov 24 2009, 12:40
Post #18


ReplayGain developer


Group: Developer
Posts: 5138
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



With the huge caveat that I made no attempt to do this properly or systematically in any way...

Interesting that if you perform the conversion badly (so that all the ultrasonic noise aliases back down into the audible band!) you hear noise that's anything-but-constant!

I know that exactly what you get is an artefact of whatever method you use to "wrongly" downsample - but even so it's interesting that this ultrasonic noise seems to vary so much - both in response to the music, and in some less predictable way.

If this was dither, it would be deemed to be very broken.


I know it's inaudible in normal playback, but if ultrasonic noise has any effect on replay equipment (assuming much of it reaches it - depends on the player) then having it vary is a bad thing - it's another potential cause of distortion which a supposedly "hi res" format can do without.

Cheers,
David.
Go to the top of the page
+Quote Post
vincefalks
post Nov 24 2009, 13:07
Post #19





Group: Members
Posts: 37
Joined: 23-August 09
Member No.: 72571



The noise is indeed a problem with the DSD format and, format-wise, something only improved by DSD128 (pushes it up twice as high) and furthermore with DXD (even less noise).

When converting to PCM, my opinion would be that one should have the option to be able to tweak such filter settings as they see fit. If ones objective would be to perform the most accurate "capture" (for lack of a better word) of what's from the original DSD signal (within the frequency range of your PCM output), you should be able to capture the whole thing, noise and all, if you wanted. Even if just for analysis and debugging.
Go to the top of the page
+Quote Post
rpp3po
post Nov 24 2009, 14:01
Post #20





Group: Developer
Posts: 1126
Joined: 11-February 03
From: Germany
Member No.: 4961



The problem with the "most accurate capture" is that DSD players do 'know' that terrible amounts of noise are there and their analog low-pass probably reaches far enough down to compensate somewhat. PCM doesn't have this problem, a good 96 kHz DAC will roughly output at least 40 kHz of perfect analog bandwidth. This would be a capture of the full digital DSD signal, but probably not an accurate capture of the equivalent DSD low passed 'experience'.

QUOTE (vincefalks @ Nov 24 2009, 13:07) *
The noise is indeed a problem with the DSD format and, format-wise, something only improved by DSD128 (pushes it up twice as high) and furthermore with DXD (even less noise).


Yes, solve the problem by wasting space for insanely high bitrates, to rescue this purely marketing motivated technology, instead of using proper PCM. DSD has not a single advantage over PCM other than DA converter costs. Good DSD DAC can be built pretty cheaply, kind of strange when you're targeting the higher end. But excellent PCM DACs are also available at commodity prices nowadays. Just accept it, the format is dead.
Go to the top of the page
+Quote Post
vincefalks
post Nov 24 2009, 14:25
Post #21





Group: Members
Posts: 37
Joined: 23-August 09
Member No.: 72571



QUOTE (rpp3po @ Nov 24 2009, 15:01) *
The problem with the "most accurate capture" is that DSD players do 'know' that terrible amounts of noise are there and their analog low-pass probably reaches far enough down to compensate somewhat. PCM doesn't have this problem, a good 96 kHz DAC will roughly output at least 40 kHz of perfect analog bandwidth. This would be a capture of the full digital DSD signal, but probably not an accurate capture of the equivalent DSD low passed 'experience'.

No, that's right. But to someone who wants as full a picture (mathemaatically) of the original DSD signal should have access to everything if they want. And then filter it out later if they want.

QUOTE
QUOTE (vincefalks @ Nov 24 2009, 13:07) *
The noise is indeed a problem with the DSD format and, format-wise, something only improved by DSD128 (pushes it up twice as high) and furthermore with DXD (even less noise).
Yes, solve the problem by wasting space for insanely high bitrates, to rescue this purely marketing motivated technology, instead of using proper PCM.

Well, I have no comment on that here smile.gif, but if you were to stay within the DSD "family", those are the "solutions" to the noise problem (although DXD is in its own new category, or you might say it's within the PCM "family" - it is PCM-*based* but with a relaxed anti-alias filter giving it properties/elements of the DSD encoding scheme).

QUOTE
DSD has not a single advantage over PCM other than DA converter costs. Good DSD DAC can be built pretty cheaply, kind of strange when you're targeting the higher end. But excellent PCM DACs are also available at commodity prices nowadays. Just accept it, the format is dead.

I have no comment about the value of DSD other than that I value it for reasons such as this: we have quite a lot of music mastered and released in the format, and some of it on non-hybrid discs. I happen to value this unique high quality catalog of music (and even on non-hybrid discs, there is the multichannel mix not available on the redbook layer) and thus am interested in converting this music to PCM in the purest way possible.

This post has been edited by vincefalks: Nov 24 2009, 14:29
Go to the top of the page
+Quote Post
saratoga
post Nov 24 2009, 15:46
Post #22





Group: Members
Posts: 4967
Joined: 2-September 02
Member No.: 3264



QUOTE (rpp3po @ Nov 24 2009, 08:01) *
DSD has not a single advantage over PCM other than DA converter costs. Good DSD DAC can be built pretty cheaply, kind of strange when you're targeting the higher end. But excellent PCM DACs are also available at commodity prices nowadays. Just accept it, the format is dead.


I don't even think the converter costs thing is true anymore.


QUOTE (vincefalks @ Nov 24 2009, 08:01) *
No, that's right. But to someone who wants as full a picture (mathemaatically) of the original DSD signal should have access to everything if they want. And then filter it out later if they want.


Well they've got the source code and are free to comment out or replace any of the filters they like.
Go to the top of the page
+Quote Post
Canar
post Nov 24 2009, 15:49
Post #23





Group: Super Moderator
Posts: 3361
Joined: 26-July 02
From: princegeorge.ca
Member No.: 2796



QUOTE (vincefalks @ Nov 24 2009, 08:25) *
I happen to value this unique high quality catalog of music (and even on non-hybrid discs, there is the multichannel mix not available on the redbook layer) and thus am interested in converting this music to PCM in the purest way possible.
I'd suggest keeping the originals as un-converted DSD and just decode to PCM on the fly using a foobar2000 component or something. I'm not sure that one exists currently, but having a library of lossless DSD material is probably preferable to the same material in PCM-decoded form. One less source of loss...


--------------------
You cannot ABX the rustling of jimmies.
No mouse? No problem.
Go to the top of the page
+Quote Post
rpp3po
post Nov 24 2009, 17:23
Post #24





Group: Developer
Posts: 1126
Joined: 11-February 03
From: Germany
Member No.: 4961



Raw DSD means saving about 3 GB per 70 minute stereo album. Kind of a waste when you can save 48kHz files, that should perfectly* preserve the 0-22 kHz band, and only need about 475 MB (FLAC). That's over 2.5 GB wasted, not for discarded but inaudible recorded information but plain quantization distortion, added by an inferior form of digitalization. Why save 2.5 GB of garbage per album? Has there ever been just one solid positive blind test for DSD?

* talk about differences of -110db or lower

This post has been edited by rpp3po: Nov 24 2009, 17:25
Go to the top of the page
+Quote Post
LukeS
post Nov 24 2009, 21:39
Post #25





Group: Members
Posts: 12
Joined: 7-September 09
Member No.: 72985



QUOTE (rpp3po @ Nov 24 2009, 08:01) *
The problem with the "most accurate capture" is that DSD players do 'know' that terrible amounts of noise are there and their analog low-pass probably reaches far enough down to compensate somewhat. PCM doesn't have this problem, a good 96 kHz DAC will roughly output at least 40 kHz of perfect analog bandwidth. This would be a capture of the full digital DSD signal, but probably not an accurate capture of the equivalent DSD low passed 'experience'.

My Denon player has a setting labled "SACD Filter" which can be set to 50kHz or 100kHz. I do not know what it is set to because I just got the player used a few days ago and it doesn't have a remote so I do not have access to the menu. I have ordered a remote and once it comes I will go through and turn off any audio post processing options I can. It the option is set to 50kHz, changing it to 100kHz may help with some of the noise I am getting.

I am very new to the SACD format but I am interested in picking up a second hand SACD player that outputs DSD over HDMI and trying to grab the DSD signals before they get encrypted and sent over HDMI, it may offer a less altered signal.


QUOTE (Canar @ Nov 24 2009, 09:49) *
I'd suggest keeping the originals as un-converted DSD and just decode to PCM on the fly using a foobar2000 component or something. I'm not sure that one exists currently, but having a library of lossless DSD material is probably preferable to the same material in PCM-decoded form. One less source of loss...

My plan exactly, I will keep the originals for archival purposes and make a copy that is converted it to a lossless PCM format like FLAC if a good conversion becomes available.

QUOTE (rpp3po @ Nov 24 2009, 11:23) *
Raw DSD means saving about 3 GB per 70 minute stereo album. Kind of a waste when you can save 48kHz files, that should perfectly* preserve the 0-22 kHz band, and only need about 475 MB (FLAC). That's over 2.5 GB wasted, not for discarded but inaudible recorded information but plain quantization distortion, added by an inferior form of digitalization. Why save 2.5 GB of garbage per album? Has there ever been just one solid positive blind test for DSD?

* talk about differences of -110db or lower

A couple things here...
48kHz files do not preserve 0-22kHz band perfectly, the brick wall effect means that the high pass filters have to cut into much lower frequencies for seamless playback.

Storage is cheap nowadays and will keep getting cheaper, I keep all my music on a NAS server. You can buy a 1.5TB drives for less then $100 each which is about 500 SACD albums depending how many tracks they have one them and over then next few years it will cost even less to store.

There is one important feature of SACDs that CD's do not offer, multi channel audio. Many albums do not have any other multichannel format like DVD-A, blu-ray so SACD is the only option.

Also the storage could be greatly reduced is a loss less container format was made to store the raw DSD channels.

This post has been edited by LukeS: Nov 24 2009, 21:50
Go to the top of the page
+Quote Post

5 Pages V   1 2 3 > » 
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 20th September 2014 - 01:49