IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
DSD / DST compress/decompress in Foobar, Plea to developers
Nalyd
post Apr 19 2014, 17:23
Post #1





Group: Members
Posts: 14
Joined: 27-September 09
Member No.: 73515



As DSD fans know, DST is simply losslessly-compressed DSD (just like FLAC is losslessly-compressed wav/PCM).

With the required plug-in, Foobar can play DSD files, as well as DST files (both ending in .dff). However, it cannot convert (compress/decompress) between the two.

There are already one or two programs that can compress DSD to DST (e.g. Philips ProTECH DSTEncoder). This is great for saving disc space. However, I am not aware of a single program that will decompress DST back to DSD.

We know Foobar can decompress DST on-the-fly because it plays back the files just fine. So, any chance a developer could develop this to make Foobar produce actual decompressed DSD (.dff) files?

Thanks in advance.
Go to the top of the page
+Quote Post
EpicForever
post Apr 19 2014, 18:57
Post #2





Group: Members
Posts: 706
Joined: 14-September 11
From: Szczecin, PL
Member No.: 93712



When I read about such topics and I see phrases like "Philips ProTECH DSTEncoder" then I think answer why something is missing is clear - patents, copyrights on intellectual property, etc.
OK, I am not an expert in this matter but I expect that if you will get any kind of official answer on your request it will be "foobar isn't about reverse engineering closed sources and breaking any copyrights".
Go to the top of the page
+Quote Post
kode54
post Apr 20 2014, 05:43
Post #3





Group: Admin
Posts: 4578
Joined: 15-December 02
Member No.: 4082



There is an open source reference implementation floating around somewhere, which was how I was able to produce a DST from a DSD file I had acquired from a free samples archive. However, both compression and decompression using this reference code is quite slow. Even decompression is sub real time on modern systems. Thankfully, each block is independently compressed, so multiple blocks may be decoded in parallel using multiple processor cores, which is what the SACD input component does.

EDIT: Here, have the reference decoder and encoder implementations, along with the included sample files. Beware of slow, also beware of patent encumbrance.

EDIT 2: foo_input_sacd manages to multithread decoding by using unique instances of the ebunch object from the reference decoder, and feeding each block, in turn, to a different thread, then waiting when all threads are occupied. It does make sure that it reads the output in a serial fashion, though. I'm not sure if the state data used by the encoder would allow for encoding every N blocks with one instance, without hurting the performance of the encoder.
Go to the top of the page
+Quote Post
Nalyd
post Apr 20 2014, 11:08
Post #4





Group: Members
Posts: 14
Joined: 27-September 09
Member No.: 73515



QUOTE (EpicForever @ Apr 19 2014, 18:57) *
When I read about such topics and I see phrases like "Philips ProTECH DSTEncoder" then I think answer why something is missing is clear - patents, copyrights on intellectual property, etc.
OK, I am not an expert in this matter but I expect that if you will get any kind of official answer on your request it will be "foobar isn't about reverse engineering closed sources and breaking any copyrights".


@ EpicForever: You seem to have missed my point (I cited ProTECH merely as an example of a program that can compress to DST, which is the complete opposite of what I need, not as something to be reverse-engineered.) My point was: Foobar already knows how to decompress DST on-the-fly anyway; is there a chance this could be used to create a file, instead of just on-the-fly audio.

QUOTE (kode54 @ Apr 20 2014, 05:43) *
foo_input_sacd manages to multithread decoding by using unique instances of the ebunch object from the reference decoder, and feeding each block, in turn, to a different thread, then waiting when all threads are occupied. It does make sure that it reads the output in a serial fashion, though. I'm not sure if the state data used by the encoder would allow for encoding every N blocks with one instance, without hurting the performance of the encoder.


@kode54: Are you saying that the on-the-fly DST decompression that Foobar does for the purpose of playback could not be adapted to create a file instead? (Thanks for the info and link - I'm not at all technical but will check it out.)

This post has been edited by Nalyd: Apr 20 2014, 11:54
Go to the top of the page
+Quote Post
kode54
post Apr 20 2014, 21:34
Post #5





Group: Admin
Posts: 4578
Joined: 15-December 02
Member No.: 4082



On the fly decompression would need to be adapted to write a file, yes. And it couldn't be through the input code already playing the file, or at least optimally, it wouldn't. It should be something like the converter.
Go to the top of the page
+Quote Post
Nalyd
post Apr 21 2014, 09:51
Post #6





Group: Members
Posts: 14
Joined: 27-September 09
Member No.: 73515



Thanks, kode54. Is there any chance a dff/dhf converter could be implemented? For those of us who create either DST or DoP files, there's currently no way to convert them back to plain dff/dhf; it's a one-way street. A converter would solve that issue...
Go to the top of the page
+Quote Post
Nalyd
post Apr 23 2014, 19:04
Post #7





Group: Members
Posts: 14
Joined: 27-September 09
Member No.: 73515



Any chance? If not, no worries, but thought I'd humbly ask...
Go to the top of the page
+Quote Post
kode54
post Apr 25 2014, 00:36
Post #8





Group: Admin
Posts: 4578
Joined: 15-December 02
Member No.: 4082



That depends on whether a developer wishes to step forward and make this happen.
Go to the top of the page
+Quote Post
Nalyd
post Apr 26 2014, 16:02
Post #9





Group: Members
Posts: 14
Joined: 27-September 09
Member No.: 73515



QUOTE (kode54 @ Apr 25 2014, 00:36) *
That depends on whether a developer wishes to step forward and make this happen.


Oh! Sorry, I thought you were one! Okay, thanks anyway for your responses.
Go to the top of the page
+Quote Post
db1989
post Apr 26 2014, 16:29
Post #10





Group: Super Moderator
Posts: 5275
Joined: 23-June 06
Member No.: 32180



kode54 is a developer, but that doesn’t mean he has to make everyone’s every wish into reality.
Go to the top of the page
+Quote Post
Nalyd
post May 2 2014, 09:55
Post #11





Group: Members
Posts: 14
Joined: 27-September 09
Member No.: 73515



QUOTE (db1989 @ Apr 26 2014, 16:29) *
kode54 is a developer, but that doesn’t mean he has to make everyone’s every wish into reality.


Oh I see; so his last response was a snark. Thanks for clarifying, and I certainly wasn't presuming that anyone had to make anyone's wish a reality.

This post has been edited by Nalyd: May 2 2014, 09:55
Go to the top of the page
+Quote Post
db1989
post May 2 2014, 18:46
Post #12





Group: Super Moderator
Posts: 5275
Joined: 23-June 06
Member No.: 32180



QUOTE (Nalyd @ May 2 2014, 09:55) *
Oh I see; so his last response was a snark. Thanks for clarifying
What? No. Don’t put words in his or my mouth. It wasn’t necessarily that at all. It’s a factually accurate statement. A plugin appearing will require someone developing it. Not explicitly saying ‘but it won’t be me’ isn’t confirmation of sarcasm. He’s not the only developer around, nor does his high productivity mean that he has to develop everything himself.
Go to the top of the page
+Quote Post
lvqcl
post May 2 2014, 19:21
Post #13





Group: Developer
Posts: 3329
Joined: 2-December 07
Member No.: 49183



BTW there are two plugins that can decode DST files:
* foo_input_dsdiff by kode54
* foo_input_sacd by manisiutkin
Go to the top of the page
+Quote Post
kode54
post May 3 2014, 02:54
Post #14





Group: Admin
Posts: 4578
Joined: 15-December 02
Member No.: 4082



Yes, but he doesn't want real time decoding, he wants offline conversion between raw DSD and DST compressed. And I wasn't necessarily refusing to do this, either. I just wasn't trying to get your hopes up by committing to doing it.

First, I have to look and see if it's possible to encode or decode DST blocks out of order with different instances of a DST encoder or decoder, and produce identical results. Then I can decide whether it would be beneficial to multithread the whole thing for a decent speed boost.

That's just converting one file. I haven't really stepped up to do a full converter utility. Certainly, it would be nice if it were something integrated with the stock converter component, or something which could be referenced from my component by the actual converter. I would hate to have to implement a dialog and conversion queue from scratch if I didn't need to.

EDIT: I have performed a quick test and determined that the reference DST encoder state is only specific to each individual block, and that the state going in and coming out is completely irrelevant, other than the fields containing the base information about the encoder mode, such as the block size and sample rate. This matches with what manisiutkin has determined and applied to foo_input_sacd for DST decoding. Thus, encoding may also be optimized by queueing each block of samples through multiple threads.
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 26th July 2014 - 12:53