IPB

Welcome Guest ( Log In | Register )

3 Pages V   1 2 3 >  
Reply to this topicStart new topic
TAK 2.2.0 - Alpha release
TBeck
post Jun 6 2011, 15:03
Post #1


TAK Developer


Group: Developer
Posts: 1095
Joined: 1-April 06
Member No.: 29051



Alpha release 1 of TAK 2.2.0 ((T)om's lossless (A)udio (K)ompressor)

It consists of:

- TAK Applications 2.2.0 Alpha 1 b in "\Applications".
- TAK Winamp plugin 2.2.0 Alpha 1 in "\WinAmp".
- TAK Decoding library 2.2.0 Alpha 1 in "\Deco_Lib".

The final release will additionally contain the SDK.

Download:

Download link removed. TAK 2.2.0 Final has been released.

What's new

This release brings support for multi-channel audio and speed optimizations for encoder and decoder.

New features:

- Support for multi-channel audio. While the stream format supports up to 16 channels, the codec currently is restricted to a maximum of 6 channels.
- Support for the "Wave Format Extensible" file format.

Improvements:

- Encoding speed improvements of up to 10 percent for my primary file set. Most of it is achieved by a modification of the algorithm which selects the optimal predictor order for each subframe. It will now often use less predictors than before, what may on average result in about 0.01 percent worse compression. You will only notice an speed advantage, if your files benefit from high predictor orders.
- Decoding speed improvements of up to 18 percent for my primary file set. Some of it is attributed to the above-noted modification of the encoder's predictor order selection algorithm. Therefore it will only take effect when decoding files encoded with this version and only, if they benefit from high predictor orders. Additionally SSSE3-instructions can be used for predictor counts of 32 and more. This affects the presets p3, p4 and sometimes p2, but only, if a particular file benefits from high predictor orders.

Known issues:

- If you use pipe decoding and the application reading the pipe is beeing terminated before the whole file has been read, TAKC may get into an endless loop and has to be manually killed with the task manager. I don't think this is a big issue but i will try to fix it in one of the next versions. BTW: Big thanks to shnutils for testing the pipe decoding!
- There seem to be some compatibility issues with pipe decoding to some other applications ("crc1632.exe" has been reported). I will try to fix it in the next release.

Alpha testing

This alpha release has already gone through extensive testing performed by my automatic scripts. Nevertheless there may be bugs left. Especially because i had to write a lot of new code for the support of multi-channel audio. This also involved a lot of minor modifications of the existing code. Therefore i would like you to verify the proper function of the codec: Compress -> Decompress -> Compare resulting wave with the original file, either by a binary compare or by the use MD5-check sums.

Certainly i am very interested into efficiency comparisons of the new multi-channel audio codec and other compressors.

Some remarks:

The most time consuming part of the new codec is it's channel decorrelation mechanism. The strongest presets sometimes check any possible channel combination. Principially you have n * (n - 1) (n = channel count) possible combinations, if you count "A predicts B" and "B predicts A" as two combinations.

Some figures:

2 channels -> 2 combinations
4 channels -> 12 combinations
5 channels -> 20 combinations
6 channels -> 30 combinations
8 channels -> 56 combinations

This rapid increase is the reason, why the codec currently is restricted to a maximum of 6 channels. I have to find and evaluate more heuristics for a fast estimation of optimal pairings which doesn't require a full evaluation of all possibilities.

Some are alreaday working. Most of them rely on the presence of a speaker assignment mask in the source wave file. If present, some faster presets will only test those combinations, which were most useful in my evaluations. A low frequency channel will never be evaluated.

But this only works, if the speaker assignment is known. Therefore the encoding time of the same audio data may differ considerably dependent on the presence of a speaker mask in the original wave file.

Im my evaluations the new codec often did beat Mpeg4Als -7 compressionwise, if a particular file provided good opportunities for channel decorrelation. Unfortunately for some files there are zero opportunities. My test corpus is still too small to make any generalized statement regarding the new codecs efficiency.

Therefore i am very interested into compression results and comparisons with other codecs.

Thanks for testing and have fun

Thomas

This post has been edited by TBeck: Jul 10 2011, 23:53
Go to the top of the page
+Quote Post
_mē_
post Jun 6 2011, 17:17
Post #2





Group: Members
Posts: 231
Joined: 6-April 09
Member No.: 68706



I get "Wave file not supported" on all files.
Sample:
http://www.hydrogenaudio.org/forums/index....showtopic=88969
BTW flood protection sucks.
Come on, what a problem with users writing 2 simple posts within a minute?
Go to the top of the page
+Quote Post
TBeck
post Jun 6 2011, 17:31
Post #3


TAK Developer


Group: Developer
Posts: 1095
Joined: 1-April 06
Member No.: 29051



QUOTE (_mē_ @ Jun 6 2011, 18:17) *
I get "Wave file not supported" on all files.
Sample:
http://www.hydrogenaudio.org/forums/index....showtopic=88969

Hm, works for me. I tried encoding with both Tak and Takc and decoding with Tak. Binary compare of the resulting wave file is also ok.

edit: Here is TAK's file info result:

CODE
=== d2.tak ====================================================

  File size:                     2.37 MB
  Header size:                   0.13 KB
    Unused:                      0.00 KB
  Compression:                  36.01 %
  Samples per channel:         384000
  File duration:                 8.00 sec
  Frame duration:                 125 ms
  Seek table:              Not available
  Audio format:            PCM, 48000 Hz, 24 Bits, 6 Channels
  Encoder:                 V 2.2.0, -p2
  Codec:                   4 Integer 24 bit MC (TAK 2.2)
  Wave file meta data:     Header 68, Footer 0 Bytes
  MD5:                     Not available
  APEv2-Tag:               No
  Status:                  Ok


This post has been edited by TBeck: Jun 6 2011, 17:37
Go to the top of the page
+Quote Post
lvqcl
post Jun 6 2011, 17:52
Post #4





Group: Developer
Posts: 3325
Joined: 2-December 07
Member No.: 49183



QUOTE
Hm, works for me.

Works here too.
Go to the top of the page
+Quote Post
TBeck
post Jun 6 2011, 17:54
Post #5


TAK Developer


Group: Developer
Posts: 1095
Joined: 1-April 06
Member No.: 29051



Although there is no hint, i just looked at my wave file reader and found some unfortunate limitation. It seems to insist on the wValidBitsPerSample value to be zero or equal to wBitsPerSample. That's not how it is meant to work. This could be a source of your problem. Maybe the editor you are using to create the sample file is modifying this field in the way TAK expects.

edit: Addendum

I can post a quick fix for you to try.

This post has been edited by TBeck: Jun 6 2011, 18:08
Go to the top of the page
+Quote Post
_mē_
post Jun 6 2011, 18:06
Post #6





Group: Members
Posts: 231
Joined: 6-April 09
Member No.: 68706



The file was created with sox. I just tried Tak instead of Takc with the same result. I use Windows XP x64 if that matters.
I can only repeat my suggestion from long ago:
Make Takc more verbose. "Wave file not supported" is not really helpful.
Go to the top of the page
+Quote Post
lvqcl
post Jun 6 2011, 18:09
Post #7





Group: Developer
Posts: 3325
Joined: 2-December 07
Member No.: 49183



QUOTE
The file was created with sox.

What version? 14.3.2 or earlier?

Added: sox 14.3.1 and earlier writes dwChannelMask = 0.

This post has been edited by lvqcl: Jun 6 2011, 18:15
Go to the top of the page
+Quote Post
TBeck
post Jun 6 2011, 18:14
Post #8


TAK Developer


Group: Developer
Posts: 1095
Joined: 1-April 06
Member No.: 29051



QUOTE (_mē_ @ Jun 6 2011, 19:06) *
The file was created with sox. I just tried Tak instead of Takc with the same result. I use Windows XP x64 if that matters.

Can you please try this TAK.exe: Attachment removed

This post has been edited by TBeck: Jun 6 2011, 20:03
Go to the top of the page
+Quote Post
TBeck
post Jun 6 2011, 18:28
Post #9


TAK Developer


Group: Developer
Posts: 1095
Joined: 1-April 06
Member No.: 29051



QUOTE (_mē_ @ Jun 6 2011, 19:06) *
I can only repeat my suggestion from long ago:
Make Takc more verbose. "Wave file not supported" is not really helpful.

Well, since this is possibly a bug, i am not sure if a more informative message would have been created.

Nevertheless i will think about it. I don't want to break the current design of the internal error handling, but maybe i can add a function to request extended error info.

QUOTE (lvqcl @ Jun 6 2011, 19:09) *
QUOTE
The file was created with sox.

Added: sox 14.3.1 and earlier writes dwChannelMask = 0.

Thanks for the hint. But this should(!) be no problem.
Go to the top of the page
+Quote Post
lvqcl
post Jun 6 2011, 18:40
Post #10





Group: Developer
Posts: 3325
Joined: 2-December 07
Member No.: 49183



I decoded d2.flac to 2 wav files with SoX 14.3.1 and 14.3.2.

They have different RIFF chunk sizes: 0x0069783c for 14.3.1 and 0x00697848 for 14.3.2.

takc.exe can encode the latter file, but not the former.
Go to the top of the page
+Quote Post
TBeck
post Jun 6 2011, 18:57
Post #11


TAK Developer


Group: Developer
Posts: 1095
Joined: 1-April 06
Member No.: 29051



QUOTE (lvqcl @ Jun 6 2011, 19:40) *
I decoded d2.flac to 2 wav files with SoX 14.3.1 and 14.3.2.

They have different RIFF chunk sizes: 0x0069783c for 14.3.1 and 0x00697848 for 14.3.2.

takc.exe can encode the latter file, but not the former.

A difference of 12 bytes? I am having some difficulties to interpret this. Well, i am really not the master of sound file formats...

Could you please send me the maybe first 512 bytes of both files?
Go to the top of the page
+Quote Post
lvqcl
post Jun 6 2011, 19:09
Post #12





Group: Developer
Posts: 3325
Joined: 2-December 07
Member No.: 49183



--

This post has been edited by lvqcl: Jun 6 2011, 19:26
Go to the top of the page
+Quote Post
TBeck
post Jun 6 2011, 19:10
Post #13


TAK Developer


Group: Developer
Posts: 1095
Joined: 1-April 06
Member No.: 29051



QUOTE (TBeck @ Jun 6 2011, 19:57) *
QUOTE (lvqcl @ Jun 6 2011, 19:40) *
I decoded d2.flac to 2 wav files with SoX 14.3.1 and 14.3.2.

They have different RIFF chunk sizes: 0x0069783c for 14.3.1 and 0x00697848 for 14.3.2.

takc.exe can encode the latter file, but not the former.

A difference of 12 bytes? I am having some difficulties to interpret this. Well, i am really not the master of sound file formats...

Could you please send me the maybe first 512 bytes of both files?

Sorry, no longer needed! I had some trouble downloading 14.3.1 from sourceforge, but finally it worked. Now i can reproduce the failure.

edit: I was too slow! Thank you.

This post has been edited by TBeck: Jun 6 2011, 19:11
Go to the top of the page
+Quote Post
_mē_
post Jun 6 2011, 19:13
Post #14





Group: Members
Posts: 231
Joined: 6-April 09
Member No.: 68706



Sorry, my bad, seems to be a flake bug, the flac doesn't decompress correctly!
I'm going to replace it in a minute with a wavpack file which I just verified to be OK. The latest build shows the same problem.
Sox 14.3.2 BTW.
ADDED: I can confirm that both versions of tak compress the decompressed flac correctly.
ADDED: The wavpack compressed file is uploaded. Same place as before, http://www.hydrogenaudio.org/forums/index....showtopic=88969

This post has been edited by _mē_: Jun 6 2011, 19:20
Go to the top of the page
+Quote Post
TBeck
post Jun 6 2011, 19:34
Post #15


TAK Developer


Group: Developer
Posts: 1095
Joined: 1-April 06
Member No.: 29051



QUOTE (_mē_ @ Jun 6 2011, 20:13) *
Sorry, my bad, seems to be a flake bug, the flac doesn't decompress correctly!
I'm going to replace it in a minute with a wavpack file which I just verified to be OK. The latest build shows the same problem.
Sox 14.3.2 BTW.
ADDED: I can confirm that both versions of tak compress the decompressed flac correctly.
ADDED: The wavpack compressed file is uploaded. Same place as before, http://www.hydrogenaudio.org/forums/index....showtopic=88969

Thank you. I can reproduce the failure.

QUOTE (lvqcl @ Jun 6 2011, 19:40) *
They have different RIFF chunk sizes: 0x0069783c for 14.3.1 and 0x00697848 for 14.3.2.

takc.exe can encode the latter file, but not the former.

Often i may be a bit too restrictive when reading foreign file formats...

My wave reader starts like this:

CODE
  if not FFile.Seek (0) then begin
    Result := frr_IoError;
    EXIT;
  end;
  if FFile.Length < SizeOf (Chunk) then begin
    Result := frr_InvalidType;
    EXIT;
  end;
  if not FFile.ReadBuffer (Chunk, SizeOf (Chunk)) then begin
    Result := frr_IoError;
    EXIT;
  end;
  if Chunk.Id <> WaveRiffId then begin
    Result := frr_InvalidType;
    EXIT;
  end;
  if Chunk.nBytes <> FFile.Length - SizeOf (Chunk) then begin
    Result := frr_StructError;
    EXIT;
  end;

It's the last check which is causing the trouble. The reader insits, that nothing comes after the RIFF-chunk and it's data. Could it be that Sox is adding some meta data? I am a bit irritated, because i never before received a report about such a failure.
Go to the top of the page
+Quote Post
lvqcl
post Jun 6 2011, 19:51
Post #16





Group: Developer
Posts: 3325
Joined: 2-December 07
Member No.: 49183



From SoX changelog:

"Fix bug where FACT chunk was not accounted for in RIFF length calculation. Patch by David Bryant."

Probably that's why SoX 14.3.2 works fine but 14.3.1 doesn't.
Go to the top of the page
+Quote Post
TBeck
post Jun 6 2011, 20:00
Post #17


TAK Developer


Group: Developer
Posts: 1095
Joined: 1-April 06
Member No.: 29051



QUOTE (lvqcl @ Jun 6 2011, 20:51) *
From SoX changelog:

"Fix bug where FACT chunk was not accounted for in RIFF length calculation. Patch by David Bryant."

Probably that's why SoX 14.3.2 works fine but 14.3.1 doesn't.

Thank you so much! This explains everything. After inspecting the file with a hex editor, i just wanted to post the suspicion, that the file is invalid.

Then everything is ok (for now...).

Thank you all for helping me to resolve this issue.
Go to the top of the page
+Quote Post
[JAZ]
post Jun 6 2011, 20:05
Post #18





Group: Members
Posts: 1751
Joined: 24-June 02
From: Catalunya(Spain)
Member No.: 2383



Indeed, that last check shouldn't be made.

Microsoft's Wave format is a RIFF based format, and the RIFF format is just a union of labeled chunks of data.
I don't know if it is usual nowadays, but wave editors used to add an INFO FACT chunk adding their name to the files. This is just one of many things that can be also in the file. And then, there is the people that insist in adding tags to wave files...

You have to assert that the chunk's size of the "data" chunk is not bigger than the current position minus file length, but only that. Read up to chunk size, and if you are not at the file end, then there's something else that you don't care.


So:

Open
Read 8 bytes-> 4bytes Id and 4 bytes size (in little endian, in the case of RIFF files). Verify that it's RIFF and the chunk size is smaller than the file size.
Read next 4 bytes if it's WAVE, continue.
Read 8 byes -> 4 bytes ID and 4 bytes size. Verify that the ID is "fmt " (there's a space in there, yes). The chunk size can also be an indication of which waveformat you have. Read up to fmt's chunk size.
Read 8 bytes -> 4 bytes ID and 4 byte size. loop while this chunk ID isn't "data", skipping the chunk size.
Read wave data up to this chunk size.
Close.

This is a basic Wave file reader. Something simpler is not a wave reader, just a reader that happens to read most wave files.

This post has been edited by [JAZ]: Jun 6 2011, 20:08
Go to the top of the page
+Quote Post
lvqcl
post Jun 6 2011, 20:11
Post #19





Group: Developer
Posts: 3325
Joined: 2-December 07
Member No.: 49183



QUOTE
Read up to chunk size, and if you are not at the file end, then there's something else that you don't care.


...but please don't break -ihs switch wink.gif
Go to the top of the page
+Quote Post
TBeck
post Jun 6 2011, 20:14
Post #20


TAK Developer


Group: Developer
Posts: 1095
Joined: 1-April 06
Member No.: 29051



QUOTE ([JAZ] @ Jun 6 2011, 21:05) *

Indeed, that last check shouldn't be made.

Microsoft's Wave format is a RIFF based format, and the RIFF format is just a union of labeled chunks of data.
I don't know if it is usual nowadays, but wave editors used to add an INFO chunk adding their name to the files. This is just one of many things that can be also in the file. And then, there is the people that insist in adding tags to wave files...

You are right.

I think my motivation for including this check was that i wanted to make sure, that i don't unintentionally ignore additional wave data chunks possibly added by applications trying to circumvent the file formats size limitations. Maybe this is a non-issue, but as i wrote above, i am (obviously) not an expert for audio file formats.

I will modify the wave reader for the next release.
Go to the top of the page
+Quote Post
TBeck
post Jun 6 2011, 20:16
Post #21


TAK Developer


Group: Developer
Posts: 1095
Joined: 1-April 06
Member No.: 29051



QUOTE (lvqcl @ Jun 6 2011, 21:11) *
QUOTE
Read up to chunk size, and if you are not at the file end, then there's something else that you don't care.


...but please don't break -ihs switch wink.gif

How can you say something so horrible?! I never would (intentionally...)!
Go to the top of the page
+Quote Post
Destroid
post Jun 7 2011, 00:18
Post #22





Group: Members
Posts: 545
Joined: 4-June 02
Member No.: 2220



I managed to finish a preview of a TAK benchmark due to a conversion issue.* This short test only covers TAK and not other multichannel encoders. The end result is that TAK 2.2.0 completed the test successfully without any errors, also noted encoding speed-ups with stereo material. Machine used: Athlon XP 3000+, 2GB, WD1.5GB SATA, WinXP SP3, NTFS.*
CODE
Source files:
DTS source 6ch 16bit 48KHz 3.29GB 102:19.125 RG -5.61dB
CD audio 1 2ch 16bit 44.1KHz 297MB 29:26.392 RG -9.75dB
CD audio 2 2ch 16bit 44.1KHz 304MB 30:07.258 RG -10.71dB


encoder / file enc timer301.exe (enc / dec)
(-p2 setting) ratio enc / dec speed (Kernel) (User) (Process)
-------------- ------ ----------------- ----------------------------------
TAK 2.2.0 DTS 40.30% 15.49x / 59.00x (2% / 13%) (95% / 75%) (98% / 88%)
TAK 2.2.0 CD 1 67.12% 115.10x / 197.68x (6% / 14%) (85% / 83%) (91% / 98%)
TAK 2.2.0 CD 2 68.96% 114.53x / 191.90x (5% / 12%) (86% / 84%) (91% / 96%)
TAK 2.1.0 CD 1 67.12% 110.49x / 192.62x (5% / 14%) (86% / 80%) (91% / 94%)
TAK 2.1.0 CD 2 68.96% 106.71x / 191.48x (5% / 12%) (84% / 83%) (90% / 95%)


bitcompared decoded TAK files with original WAV files = No differences.

*conversion issue: TAK reports Wave file not supported on WAV files with durations >02:04:16 (converted DTS->WAV using FB2K v1.1.6, foo_input_dts.dll v0.3.0 & specs above). Sizes of resulting WAV files are shown to be 4.04GB and 4.10GB even though every media player reports a durations of 02:04:16 for both those files. Anyone give me hint at fixing this (possibly) WAV header issue?


--------------------
"Something bothering you, Mister Spock?"
Go to the top of the page
+Quote Post
lvqcl
post Jun 7 2011, 01:27
Post #23





Group: Developer
Posts: 3325
Joined: 2-December 07
Member No.: 49183



QUOTE
Anyone give me hint at fixing this (possibly) WAV header issue?


1) Convert DTS to W64.
2) Convert W64 to WAV, encode to TAK with (the above-mentioned) -ihs switch.
3) Compare W64 and TAK.

Added: according to takc help, -ihs works only in pipe mode...
takc.exe -e -ihs -p2 - out.tak < in.wav


This post has been edited by lvqcl: Jun 7 2011, 13:23
Go to the top of the page
+Quote Post
TBeck
post Jun 7 2011, 02:57
Post #24


TAK Developer


Group: Developer
Posts: 1095
Joined: 1-April 06
Member No.: 29051



QUOTE (Destroid @ Jun 7 2011, 01:18) *
I managed to finish a preview of a TAK benchmark due to a conversion issue.* This short test only covers TAK and not other multichannel encoders. The end result is that TAK 2.2.0 completed the test successfully without any errors, also noted encoding speed-ups with stereo material. Machine used: Athlon XP 3000+, 2GB, WD1.5GB SATA, WinXP SP3, NTFS.*
...
bitcompared decoded TAK files with original WAV files = No differences.

Thank you! That's good news.

QUOTE (Destroid @ Jun 7 2011, 01:18) *
*conversion issue: TAK reports Wave file not supported on WAV files with durations >02:04:16 (converted DTS->WAV using FB2K v1.1.6, foo_input_dts.dll v0.3.0 & specs above). Sizes of resulting WAV files are shown to be 4.04GB and 4.10GB even though every media player reports a durations of 02:04:16 for both those files. Anyone give me hint at fixing this (possibly) WAV header issue?

Well, add about half a second to 02:04:16 and your files exceed the 4 GB limit of the wave file format. There is little i can do besides adding support for something like the Wave64-Format.
Go to the top of the page
+Quote Post
TBeck
post Jun 7 2011, 03:13
Post #25


TAK Developer


Group: Developer
Posts: 1095
Joined: 1-April 06
Member No.: 29051



QUOTE (lvqcl @ Jun 7 2011, 02:27) *
takc.exe -e -ihs -p2 - out.tak < in.wav

Good idea! Currently i have no time to look at the source to check if this works for files larger than 4 GB. TAK's stream functions definitely support more than 4 GB, but i am not sure, if i have somewhere coded some restriction. Anyway, you will not be able to decode such a file while TAK only supports the standard wave format as output format...
Go to the top of the page
+Quote Post

3 Pages V   1 2 3 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 23rd July 2014 - 10:11