IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
Porting Wavpack decoder to Dart
darkbyte
post Jan 30 2013, 11:51
Post #1





Group: Members
Posts: 152
Joined: 14-June 11
Member No.: 91517



Hi!

I'm trying to port the Wavpack decoder to Dart. I've used the Java version as the starting point and i've actually made it compilable very quickly. I'm trying to test it first with the standalone Dart VM and when that's working i will make it compatible with in-browser running.

When i started debugging the code i've realized that bit shifting in Dart is not working the same way as in Java because Dart have just one fixed point integer format. It uses a similar numeric system as Javascript. (there's a signed int type for generic integer handling, afaik its a 64bit signed integer, and num for double precision). The decoder fails very quickly in the beginning when it tries to calculate header's CRC. As i see there's a lot of bit manipulations in the code so i guess there's plenty of this kind of errors awaiting for me.

I'm wondering if there's a floating point version of the decoder available? Maybe that would be easier to port. I guess an even better way would be to understand the format and write a Dart decoder from scratch but that's a lot of work although it would be very interesting.

What do you think? smile.gif

Thanks!

Mod: i'll upload my current code to a code hosting site tonight if i won't forget it. tongue.gif

This post has been edited by darkbyte: Jan 30 2013, 12:03


--------------------
Wavpack -b450x1c
Go to the top of the page
+Quote Post
nu774
post Jan 30 2013, 12:32
Post #2





Group: Developer
Posts: 528
Joined: 22-November 10
From: Japan
Member No.: 85902



Maybe you had better read ECMA script spec, where semantics of bit-wise operator (such as logical/arithmetic shift) are defined. And when you are working with 32bit integer arithmetic, maybe you could do either of the following:
CODE
(some expression) | 0 /* convert to sint32 */
(other expression) >>> 0 /* convert to  uint32 */
Go to the top of the page
+Quote Post
darkbyte
post Jan 30 2013, 12:38
Post #3





Group: Members
Posts: 152
Joined: 14-June 11
Member No.: 91517



QUOTE (nu774 @ Jan 30 2013, 13:32) *
Maybe you had better read ECMA script spec, where semantics of bit-wise operator (such as logical/arithmetic shift) are defined. And when you are working with 32bit integer arithmetic, maybe you could do either of the following:
CODE
(some expression) | 0 /* convert to sint32 */
(other expression) >>> 0 /* convert to  uint32 */


I'm wondering it this works in Dart aswell. I'll give it a shot tonight. Thanks smile.gif


--------------------
Wavpack -b450x1c
Go to the top of the page
+Quote Post
soiaf
post Jan 30 2013, 21:51
Post #4





Group: Members (Donating)
Posts: 74
Joined: 13-May 05
From: Dublin, Ireland
Member No.: 22024



Sounds interesting, looking forward to seeing the end result!
If you download the Haxe decoder from the WavPack website, you'll find a file in the zip file called MyWavPack.js - this contains a WavPack decoder Javascript file generated from the various Haxe files - if Dart is using a similar method as Javascript for bit shifting then perhaps looking at this code may help.
Go to the top of the page
+Quote Post
darkbyte
post Jan 30 2013, 23:01
Post #5





Group: Members
Posts: 152
Joined: 14-June 11
Member No.: 91517



I couldn't make the bit shifting working in Dart as you suggested.
However i've found a more direct way to get signed fix point integers in Dart.
For eg. if you have a buffer with bytes loaded you can load a signed 32 bit integer this way using the "dart:scalarlist" package:

CODE
  static int getS32Int(List<int> buffer, int from) {
    final resultList = new Int8List(4)
    ..setRange(0, 4, buffer, from);
    return resultList.asByteArray().getInt32(0);
  }


On the sample Wavpack file i'm working with i've already got till the read_entropy_vars phase of WavPackOpenInputFile(). smile.gif
So i'm making progress but i have to debug the Dart and Java code side by side to validate the calculations.


--------------------
Wavpack -b450x1c
Go to the top of the page
+Quote Post
bryant
post Feb 1 2013, 04:34
Post #6


WavPack Developer


Group: Developer (Donating)
Posts: 1291
Joined: 3-January 02
From: San Francisco CA
Member No.: 900



This is very cool...please keep us posted on your progress (and feel free to post any questions that come up)!

David
Go to the top of the page
+Quote Post
darkbyte
post Jun 25 2013, 22:57
Post #7





Group: Members
Posts: 152
Joined: 14-June 11
Member No.: 91517



To revive this topic, i've uploaded my latest attempt here before i completely lose it: wv4dart.zip
Sadly i don't had time to go on with it. And i felt that without understanding the inner workings of the codec it's not a trivial task to find out what's going wrong. It's a very simple CLI app for now which can be run using the standalone Dart VM or with DartEditor. No prompt or help yet but it works similarly to the example of the Java version.

I think it's worth to give it another try. Especially that now Dart supports SIMD operations and i'm interested how big speed boost can be achieved with that.

bryant, is there any documentation which details the inner workings of the coded (biased on the decoder)? So far i found your math theory and the file format documentation. Is there any "noobish" documentation which explains the structure of the decoder? DSP is a very new territory for me. unsure.gif

Thanks! smile.gif

This post has been edited by darkbyte: Jun 25 2013, 23:01


--------------------
Wavpack -b450x1c
Go to the top of the page
+Quote Post
saratoga
post Jun 26 2013, 01:27
Post #8





Group: Members
Posts: 4967
Joined: 2-September 02
Member No.: 3264



https://github.com/FFmpeg/FFmpeg/blob/maste...codec/wavpack.c

The ffmpeg decoder is quite easy to understand if you need another reference.
Go to the top of the page
+Quote Post
darkbyte
post Jun 26 2013, 11:44
Post #9





Group: Members
Posts: 152
Joined: 14-June 11
Member No.: 91517



QUOTE (saratoga @ Jun 26 2013, 02:27) *
The ffmpeg decoder is quite easy to understand if you need another reference.


Thanks saratoga! smile.gif


--------------------
Wavpack -b450x1c
Go to the top of the page
+Quote Post
soiaf
post Jun 26 2013, 16:50
Post #10





Group: Members (Donating)
Posts: 74
Joined: 13-May 05
From: Dublin, Ireland
Member No.: 22024



Thanks for posting that. I tried out your code (needed to convert scalarlist to typed_data to get it to run) - if I find some time over the weekend I'll have a look at your code and see if I can determine where it's going wrong, would certainly be nice to get it up and running!
Go to the top of the page
+Quote Post
darkbyte
post Jun 26 2013, 16:59
Post #11





Group: Members
Posts: 152
Joined: 14-June 11
Member No.: 91517



Maybe we should move the code to a repository then to make the joint development even easier. smile.gif
I'll create a Github repo soon.

Mod: here it is: https://github.com/nistvan86/wv4dart/

I know that the layout of the packaging is not following the Dart conventions at all but it was just a quick hack. We can refactor it later.

The main problem is the conversion of the bit shift operators and the nearly 1:1 move to the different type system of Dart. As far i remember the median calculation is the first noticable part where calculations go wrong. Some variables are growing infinitely on the sample file i was testing with. The decoder did decode the silent part of the file without failing. laugh.gif

This post has been edited by darkbyte: Jun 26 2013, 17:25


--------------------
Wavpack -b450x1c
Go to the top of the page
+Quote Post
soiaf
post Jun 26 2013, 17:21
Post #12





Group: Members (Donating)
Posts: 74
Joined: 13-May 05
From: Dublin, Ireland
Member No.: 22024




Another good resource for learning about audio codecs is the audio formats reference document available here:

http://sourceforge.net/projects/audiotools...ts%20reference/
Go to the top of the page
+Quote Post
darkbyte
post Jun 26 2013, 18:06
Post #13





Group: Members
Posts: 152
Joined: 14-June 11
Member No.: 91517



QUOTE (soiaf @ Jun 26 2013, 18:21) *
Another good resource for learning about audio codecs is the audio formats reference document available here:

http://sourceforge.net/projects/audiotools...ts%20reference/


Amazing! ohmy.gif Thanks for sharing, this seems very helpful.


--------------------
Wavpack -b450x1c
Go to the top of the page
+Quote Post
darkbyte
post Jun 30 2013, 19:25
Post #14





Group: Members
Posts: 152
Joined: 14-June 11
Member No.: 91517



soiaf fixed the decoder this weekend so decoding is now possible. Merged his modifications to my repository so feel free to check it out.
It's still very slow (a 7 minute wavpack required eta. 10 minutes to decode) and some distortions are present in the resulting WAV file, but we have a starting point smile.gif

Thanks soiaf!


--------------------
Wavpack -b450x1c
Go to the top of the page
+Quote Post
soiaf
post Jun 30 2013, 20:58
Post #15





Group: Members (Donating)
Posts: 74
Joined: 13-May 05
From: Dublin, Ireland
Member No.: 22024



No problem, glad I could help out.
I'll have a look again sometime over the next few days and see if I can fix a couple more bugs! smile.gif
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 20th September 2014 - 06:06