IPB

Welcome Guest ( Log In | Register )

5 Pages V  « < 2 3 4 5 >  
Reply to this topicStart new topic
Apple Lossless Audio Codec is now open source (Apache license)
tuffy
post Nov 9 2011, 17:09
Post #76





Group: Members
Posts: 121
Joined: 20-August 07
Member No.: 46367



QUOTE (Wombat @ Nov 9 2011, 09:12) *
QUOTE (jukkap @ Nov 9 2011, 09:05) *

I've found out that Apple's ALAC encoder is extremely fast encoder. I compiled Apple ALAC with Intel C++ max optimized. I'll do better ALAC vs. FLAC comparison later.

The Apple's reference ALAC encoder reads/writes the files in too short blocks and it slows down dramatically its performance.

Appleīs default compression must be pretty lousy likle between -3 and -4 for flac

FLAC has a lot of tunable parameters in each frame. At the higher compression presets, it tries every combination of parameters per frame and writes out the smallest frame it can find. This is why it parallelizes really well. An encoder can pass off all those searches to their own threads and run them all simultaneously.

ALAC, on the other hand, has very few tunable parameters. Files compatible with iTunes support only a handful of channel correlation possibilities, and LPC parameter counts of 4 or 8. Everything else is fixed, relying on its adaptiveness to take up the slack I suppose. So because it's not trying out so many options, it compresses relatively quickly. But that also means it can't compress as well as FLAC. And that same adaptiveness means its can't decompress as quickly as FLAC.
Go to the top of the page
+Quote Post
Destroid
post Nov 10 2011, 09:55
Post #77





Group: Members
Posts: 555
Joined: 4-June 02
Member No.: 2220



So now the capsule summary of ALAC appears to be faster but weaker compression and not as easily decoded compared to FLAC (I guess from a -5 perspective).

Maybe some more tests can show its how well ALAC deals with mono vs. stereo as well as the single instruments vs. CD albums and that I usually try, but the question is what kind of binaries are there to try?


--------------------
"Something bothering you, Mister Spock?"
Go to the top of the page
+Quote Post
lvqcl
post Nov 10 2011, 16:57
Post #78





Group: Developer
Posts: 3467
Joined: 2-December 07
Member No.: 49183



Some time ago I tested several lossless encoders:

FLAC 1.2.1: 187x realtime at -4 compression option, 147x at -5.
iTunes x64 10.5: ~100x realtime
refalac 0.04: 77x realtime (and 130x in fast mode)
FFmpeg ALAC: 124x realtime

So current ALAC encoders are slower than FLAC.
Go to the top of the page
+Quote Post
Wombat
post Nov 10 2011, 17:09
Post #79





Group: Members
Posts: 1118
Joined: 7-October 01
Member No.: 235



QUOTE (Destroid @ Nov 10 2011, 10:55) *
So now the capsule summary of ALAC appears to be faster but weaker compression and not as easily decoded compared to FLAC (I guess from a -5 perspective).

Maybe some more tests can show its how well ALAC deals with mono vs. stereo as well as the single instruments vs. CD albums and that I usually try, but the question is what kind of binaries are there to try?

ALAC isnīt faster as flac -4 here and compresses worse but i didnīt do exact tests and i wonīt because i donīt have much interest in ALAC. Decoding of ALAC is always much slower.
On my system the foobar "Decoding Speed Test" plugin, decoding in RAM is ~150x for ALAC and ~600x for flac -8, maybe someone can check against.

Now Gregory Chudovīs compressors are completely different beasts. His libalac can compress to around the same size or slightly better as flac 1.21 but at an immense speed penalty. 15x on a C2@3400.
Since libalac has the ability to use a compression value from 1 to 10 someone may find a sweet spot.
Modern graphics cards and OSīs should all be capable of using OpenCL and here FlacCL from Gregory can encode ~500x speed with higher compression as flac 1.21. For non-OpenCL systems his libflake archieves the same compression at ~50x speed here.
I think with both compressors Mr. Chudov maxed pretty much out the possible.

Go to the top of the page
+Quote Post
lvqcl
post Nov 10 2011, 18:06
Post #80





Group: Developer
Posts: 3467
Joined: 2-December 07
Member No.: 49183



Results of my lossless encoding tests:
Encoding speed (x realtime) vs. compression ratio (encoded file size / original file size):

Go to the top of the page
+Quote Post
Wombat
post Nov 10 2011, 19:31
Post #81





Group: Members
Posts: 1118
Joined: 7-October 01
Member No.: 235



Nice graph, thanks. To be really complete and fair it should contain Cuetools libflake or FlacCL imho since it has Cueetools ALAC.

Edit: with fair i mean this graph suggests ALAC can compress better as flac but in reality it only does because of the effort of the programmer. It should only be fair to use the corresponding flac encoder coming from the same synapses.

This post has been edited by Wombat: Nov 10 2011, 19:44
Go to the top of the page
+Quote Post
googlebot
post Nov 10 2011, 22:27
Post #82





Group: Members
Posts: 698
Joined: 6-March 10
Member No.: 78779



So about how much difference in storage costs are we talking here, for a 10000 track collection, with the presented encoding efficiencies? $0.50? The playback ecosystem is much more important.

This post has been edited by googlebot: Nov 10 2011, 22:29
Go to the top of the page
+Quote Post
Wombat
post Nov 10 2011, 22:37
Post #83





Group: Members
Posts: 1118
Joined: 7-October 01
Member No.: 235



QUOTE (googlebot @ Nov 10 2011, 23:27) *
So about how much difference in storage costs are we talking here, for a 10000 track collection, with the presented encoding efficiencies? $0.50? The playback ecosystem is much more important.

Oh, sorry. i thought if we talk about ALAC we can go into detail. I wonder what hydrogenaudio stands for after all. It isnīt about storage cost i hope.
Go to the top of the page
+Quote Post
bennie
post Nov 16 2011, 18:20
Post #84





Group: Members
Posts: 4
Joined: 16-November 11
Member No.: 95215



Hi all,

First post after lurking for a while sad.gif Thanks for this lovely and very helpful community!

Does anyone know the correct value of the ALAC coefficients in order to produce bit-identical ALAC/CAF files with this official tool compared with the ones produced with iTunes? This seems to have a little impact on filesize as well (i.e. "max coded framesize" is smaller with the official tool). I searched the internet but have not found any documentation about this. Hope someone who is more knowledgeable has some clues?
Go to the top of the page
+Quote Post
Destroid
post Nov 21 2011, 10:21
Post #85





Group: Members
Posts: 555
Joined: 4-June 02
Member No.: 2220



@lvqcl - I saw your post in the FB2K 1.1.10 beta thread
QUOTE (lvqcl @ Nov 19 2011, 20:22) *
foo_input_alac 1.0.7: 121x realtime ALAC decoding speed
fb2k 1.1.10b1: 221x realtime.

and ffmpeg x64 (libavcodec 53.28.0): 167x realtime.

I am surprised at the native ALAC performance being better than originally expected. Any guess to why native ALAC did so well (other than the brilliant FB2K programmers' prowess wink.gif )?

edit: found answer to one of my own questions, my bad.

This post has been edited by Destroid: Nov 21 2011, 10:26


--------------------
"Something bothering you, Mister Spock?"
Go to the top of the page
+Quote Post
lvqcl
post Nov 21 2011, 15:56
Post #86





Group: Developer
Posts: 3467
Joined: 2-December 07
Member No.: 49183



BTW, refalac 0.12 decoding speed: 198x realtime.
Go to the top of the page
+Quote Post
Peter
post Nov 22 2011, 09:19
Post #87


foobar2000 developer


Group: Admin
Posts: 3313
Joined: 30-September 01
Member No.: 84



Sadly this new decoder has multiple security issues:
- Missing input range checks lead to various out of range reads (no more harmful than just crashing the application), some of these might be triggerable on legit undamaged files, since functions such as dyn_get_32bit() will read more bytes from the input buffer than they really care about.
- "Partial frame" feature will happily cause arbitrary number of samples to be read from the file, possibly greater than the caller expects, writing past the buffer allocated by the caller. This one may be exploitable.

At least I see that iTunes itself cannot be too easily crashed with these, I guess they outsource decoding to external processes where these behaviors are acceptable-enough.
Go to the top of the page
+Quote Post
Justin Ruggles
post Nov 22 2011, 13:54
Post #88





Group: Developer
Posts: 165
Joined: 3-June 06
From: Raleigh, NC
Member No.: 31393



Concerning what Apple had to gain from open sourcing their ALAC code... people finding bugs like this their software is ultimately a benefit to Apple. All software has bugs. Many pieces of software have some potential security vulnerabilities. Getting the public to find these for you for free saves money.

This post has been edited by db1989: Nov 22 2011, 14:09
Reason for edit: removing unnecessary full quote of above post
Go to the top of the page
+Quote Post
neovibe
post Dec 9 2011, 13:59
Post #89





Group: Members
Posts: 22
Joined: 16-September 08
Member No.: 58347



Hi all,

I am a big user of CUETools + libalac (to convert from FLAC to my apple ecosystem) and although I can't quite keep up to your technical knowledge, I'd like to know if it is foreseen that programs using libalac would move to the code made available by apple.

I am not aware of the differences (regarding the resulting file) but for the sake of future compatibility and general... harmonization, would you all agree that the apple code will be the only one used from here on?

Or is no one even looking into implementing this one instead of the reverse engineered options?

And again, no matter the future, thank you all that have taken the time to make something that makes the life of thousands of people easier without asking nothing in return.
(lame thanks I know, but it's due)

cheers everyone.
Go to the top of the page
+Quote Post
tuffy
post Dec 9 2011, 14:36
Post #90





Group: Members
Posts: 121
Joined: 20-August 07
Member No.: 46367



QUOTE (neovibe @ Dec 9 2011, 06:59) *
I am not aware of the differences (regarding the resulting file) but for the sake of future compatibility and general... harmonization, would you all agree that the apple code will be the only one used from here on?

Or is no one even looking into implementing this one instead of the reverse engineered options?

I doubt anyone would use this code as-is. It's not like libflac or libwavpack, with lots of documentation and easy to hook into other programs. Even Apple is probably using a slightly different implementation in iTunes. But it is useful as a reference to ensure that other encoders and decoders are handling files in a compatible way. And in lieu of any formal standard, that will have to do.
Go to the top of the page
+Quote Post
Wombat
post Dec 9 2011, 14:58
Post #91





Group: Members
Posts: 1118
Joined: 7-October 01
Member No.: 235



QUOTE (neovibe @ Dec 9 2011, 14:59) *
Hi all,

I am a big user of CUETools + libalac (to convert from FLAC to my apple ecosystem) and although I can't quite keep up to your technical knowledge, I'd like to know if it is foreseen that programs using libalac would move to the code made available by apple.

I am not aware of the differences (regarding the resulting file) but for the sake of future compatibility and general... harmonization, would you all agree that the apple code will be the only one used from here on?

Or is no one even looking into implementing this one instead of the reverse engineered options?


If i was the coder of libalac and did it with better compression in mind as the apple default i wouldnīt change anything. I am not aware of any non-compliant files it creates.
I may even ask you, as user to find any file that doesnīt work like it should and correct the code. Since this didnīt happen afaik nothing needs to be changed imho.

Help with your usage to improve the existing code with finding problem samples but donīt ask the coder to throw away his code.
Go to the top of the page
+Quote Post
neovibe
post Dec 9 2011, 15:59
Post #92





Group: Members
Posts: 22
Joined: 16-September 08
Member No.: 58347



QUOTE (Wombat @ Dec 9 2011, 13:58) *
I may even ask you, as user to find any file that doesnīt work like it should and correct the code. Since this didnīt happen afaik nothing needs to be changed imho.

Help with your usage to improve the existing code with finding problem samples but donīt ask the coder to throw away his code.


I am not proposing anyone to throw away anything. Thats exactly what I'm asking: is there any real technical advantage in using the apple code at all?

This post has been edited by neovibe: Dec 9 2011, 15:59
Go to the top of the page
+Quote Post
Wombat
post Dec 9 2011, 16:34
Post #93





Group: Members
Posts: 1118
Joined: 7-October 01
Member No.: 235



QUOTE (neovibe @ Dec 9 2011, 16:59) *
QUOTE (Wombat @ Dec 9 2011, 13:58) *
I may even ask you, as user to find any file that doesnīt work like it should and correct the code. Since this didnīt happen afaik nothing needs to be changed imho.

Help with your usage to improve the existing code with finding problem samples but donīt ask the coder to throw away his code.


I am not proposing anyone to throw away anything. Thats exactly what I'm asking: is there any real technical advantage in using the apple code at all?

In another thread i already asked Gregory Chudov, the Cuetools developer if there are new insights from this apple code. There arenīt for compressing.
On the decoding side there seems to be a faster implementation used in foobar already.
Go to the top of the page
+Quote Post
Peter
post Dec 10 2011, 15:36
Post #94


foobar2000 developer


Group: Admin
Posts: 3313
Joined: 30-September 01
Member No.: 84



I've published my bug-fixed version of Apple's library:
http://perkele.cc/software/ALAC
Go to the top of the page
+Quote Post
nu774
post Dec 10 2011, 16:26
Post #95





Group: Developer
Posts: 564
Joined: 22-November 10
From: Japan
Member No.: 85902



QUOTE (Peter @ Dec 10 2011, 23:36) *
I've published my bug-fixed version of Apple's library:
http://perkele.cc/software/ALAC

That's nice, thank you for opening it...
QUOTE
* MSVC/Windows compatibility fixes (MSVC does not like "static inline").

I was just using ugly -D inline=_inline to shut them up (C99 "inline" is not usable in MSVC, but it does support MSVC specific "_inline" keyword for C) .
It seems compiling as C++ (with -Tp) works, too.

By the way, mp4aac.[ch] (not in the original distribution) is included intentionally? Seems like spoontypes.h is missing.
Go to the top of the page
+Quote Post
Peter
post Dec 10 2011, 16:29
Post #96


foobar2000 developer


Group: Admin
Posts: 3313
Joined: 30-September 01
Member No.: 84



QUOTE (nu774 @ Dec 10 2011, 16:26) *
By the way, mp4aac.[ch] (not in the original distribution) is included intentionally? Seems like spoontypes.h is missing.

Oops, it wasn't meant to go in there. Thanks for pointing, fixed now.
Go to the top of the page
+Quote Post
kode54
post Dec 10 2011, 21:21
Post #97





Group: Admin
Posts: 4691
Joined: 15-December 02
Member No.: 4082



To save 5 seconds of Googling, here's a reference article that may be useful for applying that compiler intrinsic change for GCC as well:

How to use MSVC intrinsics to get the equivalent of this gcc code, only in reverse. Handy bit mangling C version included as well, probably faster than doing a for loop over the entire number, especially considering all the branches that would involve.
Go to the top of the page
+Quote Post
nu774
post Dec 21 2011, 06:34
Post #98





Group: Developer
Posts: 564
Joined: 22-November 10
From: Japan
Member No.: 85902



Have found another bug in Apple implementation.
http://alac.macosforge.org/trac/ticket/8

Apparently they are not using this implementation themselves, since encoder of CoreAudioToolbox doesn't suffer from these channel layout bugs, and returns proper magic cookies.
Go to the top of the page
+Quote Post
jukkap
post Dec 21 2011, 11:41
Post #99





Group: Members
Posts: 26
Joined: 14-July 11
Member No.: 92296



QUOTE (Peter @ Dec 10 2011, 16:36) *
I've published my bug-fixed version of Apple's library:
http://perkele.cc/software/ALAC


That's great work ! Thanks.

(I am now complete-testing the new fixed version)

This post has been edited by jukkap: Dec 21 2011, 11:41
Go to the top of the page
+Quote Post
k.eight.a
post Sep 30 2012, 16:21
Post #100





Group: Members
Posts: 434
Joined: 31-October 03
From: Europe, CZ
Member No.: 9571



I've read the whole thread. I thought I'd be able to find a command line decoder that benefits from the officialy open-sourced Apple Lossless Audio Codec.
If I understand correctly, so far there is none.


--------------------
Sorry for my poor English, I'm trying to get better... ;)
"The greatest trick the Devil ever pulled, was convincing the world he didn't exist."
Go to the top of the page
+Quote Post

5 Pages V  « < 2 3 4 5 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 21st December 2014 - 23:39