IPB

Welcome Guest ( Log In | Register )

2 Pages V   1 2 >  
Reply to this topicStart new topic
TAK 2.1.0 - Beta release 3, Now with multi-core-encoding
TBeck
post Dec 11 2010, 21:45
Post #1


TAK Developer


Group: Developer
Posts: 1098
Joined: 1-April 06
Member No.: 29051



Beta release 3 of TAK 2.1.0 ((T)om's lossless (A)udio (K)ompressor)

It consists of:

- TAK Applications 2.1.0 Beta 3 b
- TAK Winamp plugin 2.1.0 Beta 3
- TAK Decoding library 2.1.0 Beta 3

The final release will additionally contain the SDK.

Download:

Download link removed.

The final version has been released: TAK 2.1.0

What's new in Beta 3 b

This release brings only some minor fixes and modifications.

Bug fixes:

- Some part of the encoder comes in to versions: One for Single core, one for Multi core encoding. If you specify -tn1 on the command line, the single core version is beeing used, otherwises the multi core version. But if you didn't specify -tn# at all, the previos version of Takc was using the multi core version with only one thread. On systems with more than 1 core this setting will often be faster than the true single core version. Therefore speed comparisons of 'takc -tn1' vs. 'takc -tn2 (or more)' were valid, but 'takc' vs. 'takc -tn2 (or more)' were not. On my system the difference is about 3 percent. This is also relevant for comparisons of V2.0 and V2.1 Beta 3. If you have more than one core and didn't specify -tn1, the speed advantage of for instance the new SSSE3 optimizations has been overestimated.

Modifications:

- Added the -cpu# switch to the command line version, which lets you control some cpu optimizations.
- Some additions to the application's ReadMe.

What's new in Beta 3

This release brings speed optimizations (new in Beta 2) and multi core support (new in Beta 3) for the encoder and adds a new user selectable codec, which significantly improves the compression efficiency of LossyWav-processed files. Files compressed with this codec can not be decoded by earlier versions of Tak, Takc, in_tak and tak_deco_lib! The default codec remains unchanged und is therefore backwards compatible to TAK V2.0.0.

Improvements:

- Encoding speed improvements of about 10 to 20 percent (depends on preset and cpu) for cpus with the SSSE3 instruction set. Since SSSE3 (note the three 'S') isn't supported by AMD, only intel users will benefit from those optimizations.
- The encoder now creates up to four threads to utilize multiple cpu cores. Specify the thread number in the General Options dialog of the GUI-version or with the -tn option of the command line version. By default only one thread is created. You will only notice a speed up, if the encoding speed isn't already limited by the performance of your drives.
- New additional codec that improves the compression efficiency of LossyWav-processed files by up to about 2 percent (relative to the original file size) for the quality setting -q5.0 (less or more for other settings). It supports any block size that is an integer multiple of 256 samples. Please don't specify the -fsl512 option at the command line. While this was required for the standard codec, it will severily hurt the performance of the new dedicated LossyWav-codec. Another advantage of the new codec: You will not loose much compression if LossyWav deceides to remove no bits, as can happen with for instance some low amplitude files with little signal complexity. Simply specify -cLW at the command line to activate the new codec. Earlier it wasn't advantegous to use presets higher than -p2m when encoding LossyWav-Files. That's no longer true, you may even benefit from -p4m.

Modifications:

- The file info function now also shows the name of the codec used to compress the file. The new codec is called "3 LossyWav (TAK 2.1)".
- Moved the verify-option from the details-dialog to the general compression options dialog.
- All dialogs with an Add-files-option locked the source folder until the dialog was closed. Hopefully this is no longer the case (new in Beta 3).

Known issues:

- If you use pipe decoding and the application reading the pipe is beeing terminated before the whole file has been read, TAKC may get into an endless loop and has to be manually killed with the task manager. I don't think this is a big issue but i will try to fix it in one of the next versions. BTW: Big thanks to shnutils for testing the pipe decoding!
- There seem to be some compatibility issues with pipe decoding to some other applications ("crc1632.exe" has been reported). I will try to fix it in the next release.

Results

Below i will present some test results illustrating the compression efficiency and speed improvements of V2.1.

All test have been conducted with a Pentium Dual Core E5200, 2.5 GHz, 45 nm. Disk IO was disabled respectively minimized by a cache. Speed values are beeing expressed as multiple of real time, compression values as the proportion compressed / uncompressed file size. The test corpus consisted of 46 files.

Speed results for the encoder optimizations (SSSE3)

CODE
        V 2.0     V 2.1     Improvement

-p0     360,19    388,27      7,80%
-p1     276,29    319,08     15,49%
-p2     192,64    229,08     18,92%
-p3      90,60    107,72     18,90%
-p4      50,30     57,32     13,96%
-p4m     22,28     25,58     14,81%

You can find some user reports in the Beta 2 thread.

Speed results for the multi core encoder

CODE
      1 Thread  2 Threads   Improvement

-p0     388,27    736,27     89,63%
-p1     319,08    613,12     92,15%
-p2     229,08    445,95     94,67%
-p3     107,72    212,32     97,10%
-p4      57,32    113,73     98,41%
-p4m     25,58     50,43     97,15%


Results for the LossyWav-codec

First a comparison of different codecs and LossyWav quality settings.

Compression efficiency for various LossyWav quality settings

CODE
         FLAC 1.2.1 TAK 2.0    TAK 2.1    Advantage over
         -8         -p2m       -p4m       FLAC       TAK 2.0

-q0.0    20,61      19,07      17,25       3,36       1,82
-q2.5    27,43      25,95      23,93       3,50       2,02
-q5.0    33,26      31,78      29,62       3,64       2,16
-q7.5    38,79      37,28      35,03       3,76       2,25

Sometimes LossyWav deceides not to remove any or only very few bits from a file. Then it can happen, that the LossyWav-Mode of the codec is less efficient then the standard mode. To test this i used a worst case scenario.
I compressed my test corpus with TAK's standard (-cStd) and LossyWav (-cLW) codec, but without prior processing with LossyWav.

Compression efficiency for unprocessed files

CODE
         TAK 2.1    TAK 2.1
         -cStd      -cLW       Loss

-p0      58,74      58,99      -0,25
-p1      57,84      57,73       0,11
-p2      56,90      57,00      -0,10
-p3      56,36      56,44      -0,08
-p4      56,02      56,06      -0,04
-p4m     55,88      55,97      -0,09

Since the presets of the 2 codecs are constructed slightly different, they are not directly comparable. But i think it is safe to say, that the average loss is usually not bigger than about 0.1 percent.

Encoding and decoding speed are close to the standard codec, therefore i conducted no tests.

Beta testing

The beta version has already gone through extensive testing performed by my automatic scripts. But i had no oppurtunity to test encoding with more than 2 cores. Bugs are possible! If you want to help, please make sure to first compress, then decompress and finally compare the decompressed files with the original files. It may not be sufficient to use -v (Verify) and -md5 (MD5-creation and validation) to reveal multi core encoder errrors!

Please try the beta release and report any bugs in this thread.

I would also be happy about tests of compression efficiency and speed.

This will be the last beta release unless bug fixes are necessary.

Thanks for testing and have fun

Thomas

This post has been edited by TBeck: Jan 8 2011, 21:09
Go to the top of the page
+Quote Post
Bugs.Bunny
post Dec 12 2010, 00:34
Post #2





Group: Members
Posts: 39
Joined: 21-August 08
Member No.: 57350



Tested the Beta 3 with my Intel Core i7 920 (4 cores Hyperthreading -> 8 CPUs in Task-Manager)

4 threads p4m
Compression: 51.83 %
Duration: 43.87 sec
Speed: 77.66 * real time
The Task-Manager showed a CPU usage of ~52% while encoding.

Decompress:
Duration: 15.51 sec
Speed: 219.71 * real time

Bit compared the wav files (original vs encoded->decoded) and they where all 100% identical.
Go to the top of the page
+Quote Post
alvaro84
post Dec 12 2010, 10:26
Post #3





Group: Members
Posts: 128
Joined: 9-August 06
Member No.: 33830



I've just tested the new mulithreaded encoder with the 2 (65nm Conroe 4M) cores I have. I stored the source WAVs on a HDD and read them several times during the test so I'm pretty sure they were cached (it's a ~450MB test corpus, March over sky from dBu music, a Touhou remix album), the target was an SSD. My results are quite interesting:

1*1THD: Total encoding time: 1:04.771, 40.62x realtime (Single thread, single instance)
1*2THD: Total encoding time: 0:41.215, 63.83x realtime (Multithreaded, single instance - used the built-in multithreaded encoder)
2*2THD: Total encoding time: 0:46.160, 56.99x realtime (Multithreaded, two instances - used both the built-in and the foobar threading)
2*1THD: Total encoding time: 0:37.846, 69.52x realtime (Single thread, two instances - used -tn1 and two concurrent encoders at once)

It seems that in this particular case the old, external threading worked better than the new, built-in support. Note that it was not an I/O bound case, nothing depended on HDD head movement.

This post has been edited by alvaro84: Dec 12 2010, 10:30
Go to the top of the page
+Quote Post
Nowings69
post Dec 12 2010, 16:32
Post #4





Group: Members
Posts: 95
Joined: 22-December 09
From: nicyoume
Member No.: 76223



E7200 (2 threds SIMD SSE4.1)

TEST.wav uses P2 test mode with tak.exe

TAK 2.0.0(MMX)
C: 56.94%
D: 8.81sec
S: 241.81x


TAK 2.1.0(NONE)
C: 56.94%
D: 19.98sec
S: 107.17x


TAK 2.1.0(MMX)
C: 56.94%
D: 8.77sec
S: 242.79x


TAK 2.1.0(SSSE3)
C: 56.94%
D: 7.58sec
S: 281.13x

TAK 2.1.0(SSSE3 -threds2)
C: 56.94%
D: 3.94sec
S: 540.70x


If takc.exe with fb2k
TAK 2.1.0(SSSE3) is same as tak.exe
But SSSE3 with 2 threds is 440x-480x

Anyway TAK's logo is nothing
I was going to design it like this


but this is too cheap!




This post has been edited by Nowings69: Dec 12 2010, 16:57
Go to the top of the page
+Quote Post
Steve Forte Rio
post Dec 12 2010, 19:04
Post #5





Group: Members
Posts: 456
Joined: 4-October 08
From: Ukraine
Member No.: 59301



Great update!

And now, can you add the switch for choosing what CPU's encoder must use?
It can be useful for dual core processors with hyperthreading technology (like Core i) - to prevent using of two threads of one physical CPU


Here is some strange results for my Core i3 530 (2,93 Ghz, 2 cores, 4 threads):

CODE
TAK 2.1.0 beta 3 -p4m -ihs

HT-on, -tn1  - 31.04x realtime

HT-on, -tn4  - 61.63x realtime

HT-on, -tn2  - 42.42x realtime

HT-off, -tn2  - 57.35x realtime


Look how a big difference is there between results for enabled and disabled Hyper Threading...

This post has been edited by Steve Forte Rio: Dec 12 2010, 19:44
Go to the top of the page
+Quote Post
jc3213
post Dec 13 2010, 05:47
Post #6





Group: Members
Posts: 11
Joined: 28-January 10
Member No.: 77608



i3 350M 2.26G Hyper-Threading running TAK 2.1.0 beta 3 with SSSE3

======================================================================================

Thread 4, Preset P2

CYNTHIA HARRELL - I AM THE WIND.wav 61.69% 466*

Compression: 61.69 %
Duration: 0.60 sec
Speed: 463.38 * real time

------------------------------------------------------------------------------------------------------

Thread 4, Preset P4M + MD5

CYNTHIA HARRELL - I AM THE WIND.wav 60.81% 56*

Compression: 60.81 %
Duration: 4.94 sec
Speed: 56.28 * real time

===================================================================================

Thread 1 , Preset P2

CYNTHIA HARRELL - I AM THE WIND.wav 61.69% 191*

Compression: 61.69 %
Duration: 1.45 sec
Speed: 191.06 * real time

------------------------------------------------------------------------------------------------------

Thread 1 , Preset P4M + MD5

CYNTHIA HARRELL - I AM THE WIND.wav 60.81% 23*

Compression: 60.81 %
Duration: 11.89 sec
Speed: 23.36 * real time

===================================================================================

This is amazing. Thanks for your hard working Thomas, this one is really awesome.

BUT,I wonder if the final release can identify the Multi-threading and SSSE3 automatically,and is able to save a user preset profile?It is very annoying because I have to change the OPTION + PRESET everytime I run TAK.Please make these possible.

Sorry for my bad English.

This post has been edited by jc3213: Dec 13 2010, 05:53
Go to the top of the page
+Quote Post
TBeck
post Dec 13 2010, 17:01
Post #7


TAK Developer


Group: Developer
Posts: 1098
Joined: 1-April 06
Member No.: 29051



Thank you all for testing!

QUOTE (Bugs.Bunny @ Dec 12 2010, 00:34) *
4 threads p4m
The Task-Manager showed a CPU usage of ~52% while encoding.

Seems as if -p4m here is already too fast to take advantage of more than 2 threads. Maybe it's time to make -p4m a bit stronger (and slower)...

QUOTE (alvaro84 @ Dec 12 2010, 10:26) *
1*2THD: Total encoding time: 0:41.215, 63.83x realtime (Multithreaded, single instance - used the built-in multithreaded encoder)
2*1THD: Total encoding time: 0:37.846, 69.52x realtime (Single thread, two instances - used -tn1 and two concurrent encoders at once)

It seems that in this particular case the old, external threading worked better than the new, built-in support. Note that it was not an I/O bound case, nothing depended on HDD head movement.

QUOTE (Nowings69 @ Dec 12 2010, 16:32) *
If takc.exe with fb2k
TAK 2.1.0(SSSE3) is same as tak.exe
But SSSE3 with 2 threds is 440x-480x

Well, that's not optimal. But i am unsure, if i can improve it. There can be so much interaction if 2 programs (TAK and FOOBAR) are running simultaneously. I may try one or two modifications, but not for this release.

QUOTE (Nowings69 @ Dec 12 2010, 16:32) *
TAK 2.1.0(NONE)
S: 107.17x

TAK 2.1.0(MMX)
S: 242.79x

Interesting. It's been quite some time since i evaluated the speed of NONE. NONE is using plain pascal code, which is not even really optimized. It mainly serves as a reference to check the assembler optimizations against.

QUOTE (Nowings69 @ Dec 12 2010, 16:32) *
Anyway TAK's logo is nothing.

Which logo? TAK still lacks one...

QUOTE (Steve Forte Rio @ Dec 12 2010, 19:04) *
And now, can you add the switch for choosing what CPU's encoder must use?
It can be useful for dual core processors with hyperthreading technology (like Core i) - to prevent using of two threads of one physical CPU
...
Look how a big difference is there between results for enabled and disabled Hyper Threading...

I am just working on it, but i think you will have to wait until V2.1.1 for a solution. That because a lot of testing on different systems may be required.

QUOTE (jc3213 @ Dec 13 2010, 05:47) *
BUT,I wonder if the final release can identify the Multi-threading and SSSE3 automatically,and is able to save a user preset profile?It is very annoying because I have to change the OPTION + PRESET everytime I run TAK.Please make these possible.

If available, SSSE3 is selected automatically. I am unsure, if the same should be done with multi-threading. If and how many threads are advantegous may depend on several system and setup specific factors, which TAK can't determine.

Currently i don't intend to implement a configuration file. Maybe later. Can you possibly use foobar for encoding?
Go to the top of the page
+Quote Post
Steve Forte Rio
post Dec 13 2010, 20:01
Post #8





Group: Members
Posts: 456
Joined: 4-October 08
From: Ukraine
Member No.: 59301



Sorry, can't find the answer here.

is it possible to disable SSSE3 optimization for encoding?

This post has been edited by Steve Forte Rio: Dec 13 2010, 20:02
Go to the top of the page
+Quote Post
alvaro84
post Dec 13 2010, 21:23
Post #9





Group: Members
Posts: 128
Joined: 9-August 06
Member No.: 33830



QUOTE (TBeck @ Dec 13 2010, 17:01) *
Well, that's not optimal. But i am unsure, if i can improve it. There can be so much interaction if 2 programs (TAK and FOOBAR) are running simultaneously. I may try one or two modifications, but not for this release.


Now that you say - it makes perfect sense. Maybe I should test it outside foobar? But this is the 'real life' way I use TAK unsure.gif
Go to the top of the page
+Quote Post
TBeck
post Dec 13 2010, 21:32
Post #10


TAK Developer


Group: Developer
Posts: 1098
Joined: 1-April 06
Member No.: 29051



QUOTE (Steve Forte Rio @ Dec 13 2010, 20:01) *
Sorry, can't find the answer here.

is it possible to disable SSSE3 optimization for encoding?

If you are referring to the command line version: Good question! And i have to apologize... There is no switch to control cpu optimizations. It must have vanished sometime. If it ever existed, i am not really sure. But surely i could add such a switch.

Well, i have to to take stock of myself (a new item from my dictionary), to deceide, if i want to release another beta, which accomplishes some of the more recent user demands. Unfortunately too many betas may create a bad impression for irregular readers. I have to think about it.
Go to the top of the page
+Quote Post
TBeck
post Dec 13 2010, 21:45
Post #11


TAK Developer


Group: Developer
Posts: 1098
Joined: 1-April 06
Member No.: 29051



QUOTE (alvaro84 @ Dec 13 2010, 21:23) *
QUOTE (TBeck @ Dec 13 2010, 17:01) *
Well, that's not optimal. But i am unsure, if i can improve it. There can be so much interaction if 2 programs (TAK and FOOBAR) are running simultaneously. I may try one or two modifications, but not for this release.

Now that you say - it makes perfect sense. Maybe I should test it outside foobar? But this is the 'real life' way I use TAK unsure.gif

I am not sure, but possibly your setting already provides optimum performance. TAK itself most likely will never process multiple files simultaneously, but maybe exactly this leads to maximum performance on many systems. Probably a bit disappointing for you. Possibly TAK's multi-core-encoding is only advantegous for less sophisticated users (i am sure, there are many, i myself can count me among them when using some other applications), who don't know how to setup foobar for optimum performance.
Go to the top of the page
+Quote Post
Bugs.Bunny
post Dec 13 2010, 21:47
Post #12





Group: Members
Posts: 39
Joined: 21-August 08
Member No.: 57350



QUOTE (TBeck @ Dec 13 2010, 18:01) *
Seems as if -p4m here is already too fast to take advantage of more than 2 threads. Maybe it's time to make -p4m a bit stronger (and slower)...

A bit stronger -p4m would be interesting. But also it would be interesting to have an option for 8 threads. Since my CPU has got 4 hyperthreading cores = 8 threads it would be interesting to see how much 8 threads would raise the CPU usage on my system.
I've seen some multithreading applications that nearly fill my system to 100%.

Btw. I've got two SSDs installed in my PC: an Intel SLC drive (System) and a Vertex MLC (Data). For my encoding/decoding test I did read the files from one drive and write them to the other. The drives should not be a limiting factor...

Tried P2 and P0 with 4 threads now also:
P2
Compression: 52.82 %
Duration: 5.45 sec
Speed: 625.49 * real time

P0
Compression: 54.15 %
Duration: 3.67 sec
Speed: 929.00 * real time
Go to the top of the page
+Quote Post
Nowings69
post Dec 14 2010, 00:27
Post #13





Group: Members
Posts: 95
Joined: 22-December 09
From: nicyoume
Member No.: 76223



Which logo? TAK still lacks one...

This one?


Thanks always nice one to all

Go to the top of the page
+Quote Post
alvaro84
post Dec 14 2010, 01:48
Post #14





Group: Members
Posts: 128
Joined: 9-August 06
Member No.: 33830



QUOTE (TBeck @ Dec 13 2010, 21:45) *
I am not sure, but possibly your setting already provides optimum performance. TAK itself most likely will never process multiple files simultaneously, but maybe exactly this leads to maximum performance on many systems. Probably a bit disappointing for you. Possibly TAK's multi-core-encoding is only advantegous for less sophisticated users (i am sure, there are many, i myself can count me among them when using some other applications), who don't know how to setup foobar for optimum performance.


I think that processing one single file via multiple threads could be better in the aforementioned 'real life' circumstances. The way I measured was optimized to show the speed differences in the runs, but real media is rarely cached, especially when there's more of it. Processing two (or more!) files on HDDs simultaneously would lead more head movement and therefore could be much slower. I experienced that mere 2 encoders reading and writing concurrently a single hard drive can be a serious bottleneck, and it can be effectively alleviated by processing the same file in parallel so the encoder can do linear reads/writes. I'll do a test on uncached data if someone don't outrun me (which can be quite easy as I won't possibly turn on my home PC until Friday evening and even then I may won't run any tests in a bad sleep deprivation I'll have by then.)

It seems there's room to improve my test methods too biggrin.gif
We need to eliminate the drives as limiting factors to measure what the encoder is capable of - yet we need to have that bottleneck to see the usefulness of the changes you just made. Two different things.

(Yup, I'd really like to go all SSD, they're cool and silent, but even if terabyte SSDs exist (do they?), they cost an arm and a leg sad.gif So HDDs are still a reality.)

This post has been edited by alvaro84: Dec 14 2010, 01:53
Go to the top of the page
+Quote Post
hellokeith
post Dec 14 2010, 08:46
Post #15





Group: Members
Posts: 288
Joined: 14-August 06
Member No.: 34027



Thomas,

Anything specifically you would like to see tested on 4 cores (no HT)? I have no idea about tak switches, but I'm willing to carry out any test you seek, with your instruction. I've many albums in lossless format I can convert to wav and then test with tak.
Go to the top of the page
+Quote Post
jc3213
post Dec 14 2010, 12:38
Post #16





Group: Members
Posts: 11
Joined: 28-January 10
Member No.: 77608



Thanks for the answer,but I'd prefer application rather than foobar/winamp cause I don't know whether the parameters is correct,because I don't know where to find a complete manual about TAKC parameters.(sorry)

EDIT:X264 encoder is able to determine multi-thread if use the parament "--threads auto" ,and the number of threads can be managed by using "--threads X"(X = number of the thread)

EDIT2:Maybe you can add two command line parameters: "-mmx" to enable MMX support,and "-ssse3" to enable SSSE3 support.

Moderation: removed unnecessary large quote.

This post has been edited by Synthetic Soul: Dec 14 2010, 13:08
Go to the top of the page
+Quote Post
jc3213
post Dec 14 2010, 13:24
Post #17





Group: Members
Posts: 11
Joined: 28-January 10
Member No.: 77608



Phenom II x4 905e 2.2GHz, running application.

=========================================================

Thread 4 Preset P2

Priere -プリエール-.wav 59.80% 290*

Compression: 59.80 %
Duration: 1.31 sec
Speed: 288.66 * real time

--------------------------------------------------------------------------

Thread 1 Preset P2

Priere -プリエール-.wav 59.80% 52*

Compression: 59.80 %
Duration: 7.23 sec
Speed: 52.16 * real time

=========================================================

Thread 4 P4M

Priere -プリエール-.wav 58.97% 66*

Compression: 58.97 %
Duration: 5.69 sec
Speed: 66.27 * real time

--------------------------------------------------------------------------

Thread 1 P4M

Priere -プリエール-.wav 58.97% 8*

Compression: 58.97 %
Duration: 47.75 sec
Speed: 7.89 * real time

=========================================================

My friend's AMD quad-core processor gives more than 400% boost up @ oreset 2 and more than 800% boost up @ preset 4m.Is there any thing wrong??
Go to the top of the page
+Quote Post
TBeck
post Dec 14 2010, 17:18
Post #18


TAK Developer


Group: Developer
Posts: 1098
Joined: 1-April 06
Member No.: 29051



QUOTE (TBeck @ Dec 13 2010, 17:01) *
QUOTE (Steve Forte Rio @ Dec 12 2010, 19:04) *
And now, can you add the switch for choosing what CPU's encoder must use?
It can be useful for dual core processors with hyperthreading technology (like Core i) - to prevent using of two threads of one physical CPU
...
Look how a big difference is there between results for enabled and disabled Hyper Threading...

I am just working on it, but i think you will have to wait until V2.1.1 for a solution. That because a lot of testing on different systems may be required.

After accomplishing the code i thought a bit more about the issue (wrong order: code first then ask...). And i deceided not to add an option to let the user select cores respectively to let the codec automatically select only physical cores.

Let me explain why:

First: It's no surprise that Hyperthreading doesn't work well for the codec. Hyperthreading can only be advantegous, if the threads are doing different things, or more specifically using different execution units of the physical core (for instance Integer-, floating point or MMX-calculations). But the codec threads will most of the time require the same execution ports. So you only get a penalty for the overhead of the hyperthreading management.

You could use Window's SetProcessAffinityMask() function to restrict TAK to only physical cores (that's what my new code would do). But this is generally considered as bad practice. One reasonable example: Two different applications (for instance TAK and foobar,) are using the same method. Both then would be bound to the same physical cpu's although they possibly are using different execution units and would benefit from hyperthreading.

Ok, a quite sophisticated user who knows what he is doing would choose a proper configuration to optimize his system for this situation. But what about the average user? I am sure that as soon as i add a '-UseOnlyPhysicalCpus'-switch you will find recommendations to tune TAK for maximum performance by applying this switch. And sometimes this may be bad advice.

Therefore i am currently quite reluctant to implement such a switch.

QUOTE (TBeck @ Dec 13 2010, 21:32) *
QUOTE (Steve Forte Rio @ Dec 13 2010, 20:01) *
Sorry, can't find the answer here.

is it possible to disable SSSE3 optimization for encoding?

If you are referring to the command line version: Good question! And i have to apologize... There is no switch to control cpu optimizations. It must have vanished sometime. If it ever existed, i am not really sure. But surely i could add such a switch.

Well, i have to to take stock of myself (a new item from my dictionary), to deceide, if i want to release another beta, which accomplishes some of the more recent user demands. Unfortunately too many betas may create a bad impression for irregular readers. I have to think about it.

I have added a switch to control the cpu optimizations.

By doing so i found a (non-fatal) bug in the command line version:

Some part of the encoder comes in to versions: One for Single core, one for Multi core encoding. If you specify -tn1, the single core version is beeing used, otherwises the multi core version. But if you don't specify -tn# at all, Takc will use the multi core version with only one thread.

On systems with more than 1 core this setting will often be faster than the true single core version. Therefore speed comparisons of 'takc -tn1' vs. 'takc -tn2 (or more)' are valid, but 'takc' vs. 'takc -tn2 (or more)' are not. On my system the difference is about 3 percent.

This is also relevant for comparisons of V2.0 and V2.1. If you have more than one core and don't specify -tn1, the speed advantage of for instance the new SSSE3 optimizations may be overestimated.

Probably i will soon release a Beta 3b.

QUOTE (Bugs.Bunny @ Dec 13 2010, 21:47) *
A bit stronger -p4m would be interesting. But also it would be interesting to have an option for 8 threads. Since my CPU has got 4 hyperthreading cores = 8 threads it would be interesting to see how much 8 threads would raise the CPU usage on my system.
I've seen some multithreading applications that nearly fill my system to 100%.

As i explained above, hyperthreading isn't advantegous for the encoder. And you can only achieve such a high cpu usage, if neither disk io nor memory access are limiting factors.

QUOTE (alvaro84 @ Dec 14 2010, 01:48) *
I'll do a test on uncached data if someone don't outrun me (which can be quite easy as I won't possibly turn on my home PC until Friday evening and even then I may won't run any tests in a bad sleep deprivation I'll have by then.)

Please take care of yourself rolleyes.gif

QUOTE (hellokeith @ Dec 14 2010, 08:46) *
Anything specifically you would like to see tested on 4 cores (no HT)? I have no idea about tak switches, but I'm willing to carry out any test you seek, with your instruction. I've many albums in lossless format I can convert to wav and then test with tak.

Thank you for the offer. Please wait for the Beta 3b release.
Go to the top of the page
+Quote Post
TBeck
post Dec 15 2010, 01:03
Post #19


TAK Developer


Group: Developer
Posts: 1098
Joined: 1-April 06
Member No.: 29051



The beta 3 b has been released. Please see the first post for details.
Go to the top of the page
+Quote Post
Nowings69
post Dec 15 2010, 15:24
Post #20





Group: Members
Posts: 95
Joined: 22-December 09
From: nicyoume
Member No.: 76223



E7200 (2 threads SIMD SSE4.1)

TEST.wav uses takc.exe -p2 -ihs with fb2k

TAK2.0.0

Compression: 64.30 %
Duration: 18.46 sec
Speed: 211.88 * real time


TAK2.1.0 beta 3 b(NONE)

Compression: 64.30 %
Duration: 38.09 sec
Speed: 102.66 * real time


TAK2.1.0 beta 3 b(MMX)

Compression: 64.30 %
Duration: 18.25 sec
Speed: 214.34 * real time


TAK2.1.0 beta 3 b(SSSE3)

Compression: 64.30 %
Duration: 15.76 sec
Speed: 248.10 * real time


TAK2.1.0 beta 3 b(SSSE3 -threads 2)

Compression: 64.30 %
Duration: 10.96 sec
Speed: 356.71 * real time


TAK2.1.0 beta 3 b(default)

Compression: 64.30 %
Duration: 16.14 sec
Speed: 242.32 * real time


TAK2.1.0 beta 3 b(default -threads 2)

Compression: 64.30 %
Duration: 9.76 sec
Speed: 400.70 * real time

This post has been edited by Nowings69: Dec 15 2010, 16:18
Go to the top of the page
+Quote Post
TBeck
post Dec 16 2010, 01:02
Post #21


TAK Developer


Group: Developer
Posts: 1098
Joined: 1-April 06
Member No.: 29051



Thanks again for testing the beta.

I would like to release TAK 2.1 final til sunday. Then my mind will be free to deal with something else. Well, not necessarily with something totally different, but with another aspect.

Therefore i am asking again for bug reports.

BTW: Has anyone tried the new LossyWav-codec? No need to try it, if you don't use LossyWav at all. I am just curious, if this codec is of any value for somebody.
Go to the top of the page
+Quote Post
hellokeith
post Dec 16 2010, 08:35
Post #22





Group: Members
Posts: 288
Joined: 14-August 06
Member No.: 34027



QUOTE (TBeck @ Dec 14 2010, 10:18) *
QUOTE (hellokeith @ Dec 14 2010, 08:46) *
Anything specifically you would like to see tested on 4 cores (no HT)? I have no idea about tak switches, but I'm willing to carry out any test you seek, with your instruction. I've many albums in lossless format I can convert to wav and then test with tak.

Thank you for the offer. Please wait for the Beta 3b release.


QUOTE (TBeck)
The beta 3 b has been released. Please see the first post for details.


Thomas,

Do you have something specific you wanted tested?
Go to the top of the page
+Quote Post
Nowings69
post Dec 16 2010, 10:54
Post #23





Group: Members
Posts: 95
Joined: 22-December 09
From: nicyoume
Member No.: 76223



QUOTE (TBeck @ Dec 16 2010, 09:02) *
BTW: Has anyone tried the new LossyWav-codec? No need to try it, if you don't use LossyWav at all. I am just curious, if this codec is of any value for somebody.


I usually choose LossyFLAC for LPCM with Video in Matroska
and it is better for portable(e.g.ROCKBOX) too.
Because I dont know how to play it with LossyTAK right now.
Go to the top of the page
+Quote Post
jc3213
post Dec 16 2010, 16:12
Post #24





Group: Members
Posts: 11
Joined: 28-January 10
Member No.: 77608



QUOTE (TBeck @ Dec 16 2010, 09:02) *
Thanks again for testing the beta.

I would like to release TAK 2.1 final til sunday. Then my mind will be free to deal with something else. Well, not necessarily with something totally different, but with another aspect.

Therefore i am asking again for bug reports.

BTW: Has anyone tried the new LossyWav-codec? No need to try it, if you don't use LossyWav at all. I am just curious, if this codec is of any value for somebody.

Well, I'd prefer MP3 rather than other LossyCodec, because my portable devices don't support any Lossless Codecs or other LossyCodec besides LossyWav.I don't think it's nessessary to have a lossyTAK,cause no handheld devices support it. This is only my personal opinion.
Go to the top of the page
+Quote Post
TBeck
post Dec 16 2010, 23:32
Post #25


TAK Developer


Group: Developer
Posts: 1098
Joined: 1-April 06
Member No.: 29051



QUOTE (hellokeith @ Dec 16 2010, 08:35) *
Do you have something specific you wanted tested?

Thank you! Regarding the speed optimizations anything sensible has been evaluated. As i wrote in the initial post, verification of the proper codec function is always useful: "If you want to help, please make sure to first compress, then decompress and finally compare the decompressed files with the original files. It may not be sufficient to use -v (Verify) and -md5 (MD5-creation and validation) to reveal multi core encoder errrors!" Maybe you could do this with your favourite preset.

There are usually more and more interesting testing opportunities when i introduce new codec features which have to be tuned for optimum performance.

QUOTE (Nowings69 @ Dec 16 2010, 10:54) *
I usually choose LossyFLAC for LPCM with Video in Matroska
and it is better for portable(e.g.ROCKBOX) too.
Because I dont know how to play it with LossyTAK right now.

QUOTE (jc3213 @ Dec 16 2010, 16:12) *
Well, I'd prefer MP3 rather than other LossyCodec, because my portable devices don't support any Lossless Codecs or other LossyCodec besides LossyWav.I don't think it's nessessary to have a lossyTAK,cause no handheld devices support it. This is only my personal opinion.

I was aware of those restrictions, but thought, there would be some users using LossyWav only for archiving purposes, where the lack of hardware support wouldn't matter.

I think i will remove the dedicated LossyWav codec from TAK 2.1. It can easily be added again when there is some demand. But currently i don't want to add such a quite complex feature that has not been tested by anyone else but me. Nevertheless the development of the codec was a quite interesting task for me.
Go to the top of the page
+Quote Post

2 Pages V   1 2 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 25th October 2014 - 19:43