IPB

Welcome Guest ( Log In | Register )

2 Pages V   1 2 >  
Reply to this topicStart new topic
Potential serious bug in Apple Lossless codec in iTunes 10
Noble Tiger
post Oct 6 2010, 06:52
Post #1





Group: Members
Posts: 6
Joined: 6-October 10
From: Ontario, Canada
Member No.: 84374



Hi everyone,

I'm using iTunes 10.0.1.22 on Windows Vista Home Premium SP1 64-bit.

I noticed a potential bug in the Apple Lossless Codec in iTunes 10 where if you convert a WAV or AIFF file to Apple Lossless format in iTunes, then convert it back to it's original file type (WAV or AIFF respectively) in iTunes, the resulting files are not the same size in bytes or in time length as the original WAV or AIFF files. The files are usually 5KB or so smaller and can be about 1 second shorter than the original files. I did extensive testing with this using various WAV and AIFF files both generated by iTunes (e.g. ripping a CD track) and by other programs and every time the Apple Lossless codec in iTunes does not convert back to the original file's length or file size. Apple Lossless should convert back exactly, but it isn't. In previous versions of iTunes it did, but not in 10.0.1.22. This may also effect the accuracy Apple Lossess (ALAC) file playback in iTunes and conversion to other formats within iTunes as well.

I think it's a bug, but can anyone else reproduce this on their systems? or is it just on my system?

To test in iTunes:

Convert a WAV to Apple Lossess then back to WAV and compare the two WAV's
Convert an AIFF to Apple Lossless then back to AIFF and compare the two AIFF's
Go to the top of the page
+Quote Post
Engelsstaub
post Oct 6 2010, 07:44
Post #2





Group: Members
Posts: 556
Joined: 16-February 10
Member No.: 78200



I just tried this and the file sizes are exactly the same (Win7 64.)

I didn't actually rip the track with iTunes (same version as yours,) but I decompressed a FLAC file to WAV with dBpoweramp. From thence I copied it into the iTunes library and created an Apple Lossless version from within. I decompressed that (ALAC) file within Explorer to WAV with dBpa (I'm not aware of any way to do this from within iTunes.)

My file-sizes were exactly the same. Ditto for the track-lengths. I don't know what to tell you. Maybe I should try ripping to WAV in iTunes? I never use the iTunes ripper anymore nor do I do conversions within the app. I use dBpoweramp and EAC for all of that.

IDK... I'll try it with a CD in the iTunes ripper to WAV and get back to you.

(EDIT) I was able to reproduce your error once I figured out how to switch the settings between "Create Apple Lossless version" and "Create WAV version." (I never saw the option to convert to WAV on left-click because I never had it set to rip directly to WAV.)

The original file/song was 41.48 Mb 4 min. 6 sec. for the original wav and 41.47 Mb 4 min. 6 sec. for the one converted to ALAC and back to WAV within iTunes. That IS strange...

...so it's not just your system.

This post has been edited by Engelsstaub: Oct 6 2010, 08:16


--------------------
The Loudness War is over. Now it's a hopeless occupation.
Go to the top of the page
+Quote Post
greynol
post Oct 6 2010, 08:36
Post #3





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



QUOTE (Noble Tiger @ Oct 5 2010, 22:52) *
The files are usually 5KB or so smaller and can be about 1 second shorter than the original files.

5KB is less than three hundredths of a second of CD-quality audio stored as raw PCM data (read: WAV or AIFF).

QUOTE (Noble Tiger @ Oct 5 2010, 22:52) *
Convert a WAV to Apple Lossess then back to WAV and compare the two WAV's
Convert an AIFF to Apple Lossless then back to AIFF and compare the two AIFF's

Please provide two 30 second (or less) clips of a wave file: one before and one after conversion. Post them to our uploads sub-forum:
http://www.hydrogenaudio.org/forums/index.php?showforum=35


--------------------
YOUR EYES CANNOT HEAR!!!!!!!!!!!
Go to the top of the page
+Quote Post
greynol
post Oct 6 2010, 08:45
Post #4





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



QUOTE (Engelsstaub @ Oct 5 2010, 23:44) *
The original file/song was 41.48 Mb 4 min. 6 sec. for the original wav and 41.47 Mb 4 min. 6 sec. for the one converted to ALAC and back to WAV within iTunes. That IS strange...

These methods are far too crude to determine whether a conversion is lossless. Because a wave file is nothing more than a container there can be differences in file size which will not affect the audio data in any way whatsoever. Seeing that you are using Windows, I know of a few options to bit-compare the raw PCM data such as foobar2000 or EAC. Similarly, you can generate md5 hashes of the PCM data using shntool at the command line and compare them.

This post has been edited by greynol: Oct 6 2010, 09:07


--------------------
YOUR EYES CANNOT HEAR!!!!!!!!!!!
Go to the top of the page
+Quote Post
Engelsstaub
post Oct 6 2010, 09:15
Post #5





Group: Members
Posts: 556
Joined: 16-February 10
Member No.: 78200



I didn't know that. I just assumed that the file size would be the same if the data was the same within the container. I'm as terrified of the command line as I am of editing the registry; I'll only do it if I follow a tutorial verbatim. (I've only had a personal computer outside of work for about four years.)

I do have EAC and a portable Foobar 2000. I can give it a try just for the sake of learning something smile.gif It's just "Compare WAVs" in EAC, right? I'll have a go at it tomorrow and get back to this...


--------------------
The Loudness War is over. Now it's a hopeless occupation.
Go to the top of the page
+Quote Post
Robertina
post Oct 6 2010, 09:19
Post #6





Group: Members
Posts: 1306
Joined: 4-January 09
Member No.: 65169



QUOTE (Engelsstaub @ Oct 5 2010, 21:15) *
I do have EAC and a portable Foobar 2000. I can give it a try just for the sake of learning something   smile.gif

For foobar2000 you would need Peter's Binary Comparator. After installation select the two files in question and choose from foobar's context menu -> Utilities / Bit-Compare tracks.
Go to the top of the page
+Quote Post
Engelsstaub
post Oct 6 2010, 10:36
Post #7





Group: Members
Posts: 556
Joined: 16-February 10
Member No.: 78200



This is what I got with F2K (and now I know how to do something new...so thanks for that, Robertina.)

CODE
Differences found in 1 out of 1 track pairs.

Comparing:
"...\Desktop\02 Cross My Heart And Hope To Die.wav"
"...\Desktop\original.wav"
Length mismatch : 4:06.503039 vs 4:06.573333, 10870784 vs 10873884 samples



(I modified the log to exclude the full directory and changed the file name of the original WAV ripped with iTunes so that I could differentiate between the two.)


EAC:





This post has been edited by Engelsstaub: Oct 6 2010, 10:48


--------------------
The Loudness War is over. Now it's a hopeless occupation.
Go to the top of the page
+Quote Post
andy o
post Oct 6 2010, 14:29
Post #8





Group: Members
Posts: 1291
Joined: 14-April 09
Member No.: 68950



QUOTE (Engelsstaub @ Oct 6 2010, 02:36) *
This is what I got with F2K (and now I know how to do something new...so thanks for that, Robertina.)

So what about the first WAV conversion you made without iTunes? Seems to suggest the "problem" is in iTunes putting the audio in a wav container, not of the ALAC encoder.
Go to the top of the page
+Quote Post
mixminus1
post Oct 6 2010, 15:18
Post #9





Group: Members
Posts: 687
Joined: 23-February 05
Member No.: 20097



Confirmed under Windows XP SP3: a 52 second WAV converted to ALAC in iTunes 10.0.1.22 and then converted back to *either* WAV or AAC and then WAV in iTunes is 2,570 samples shorter (WAVs checked in foobar2000).

I also tried the AAC->WAV conversion in fb2k and the result was identical.

Since Englesstaub doesn't experience this issue when converting from iTunes-compressed ALAC to WAV using dBPA, it would indeed appear to be an issue with the ALAC *de*coder in iTunes 10.0.1.22.

I won't have access to my Mac until later this evening - anybody on a Mac before then that can try to replicate this under OS X?


--------------------
"Not sure what the question is, but the answer is probably no."
Go to the top of the page
+Quote Post
twostar
post Oct 6 2010, 15:36
Post #10





Group: Members
Posts: 487
Joined: 5-August 02
From: Manila
Member No.: 2939



Same thing here using iTunes version 10.0.0.68 and Win7 Starter.

Here's the log from foobar2000's bit comparator.

CODE
Differences found in 1 out of 1 track pairs.

Comparing:
"C:\Users\ac25\Desktop\Foo Fighters - Greatest Hits\itunes.wav"
"C:\Users\ac25\Desktop\Foo Fighters - Greatest Hits\orig.wav"
Length mismatch : 2:14.304218 vs 2:14.306667, 5922816 vs 5922924 samples


This post has been edited by twostar: Oct 6 2010, 15:36
Go to the top of the page
+Quote Post
mixminus1
post Oct 6 2010, 15:58
Post #11





Group: Members
Posts: 687
Joined: 23-February 05
Member No.: 20097



It's also interesting that there doesn't seem to be any consistency with regards to the length of the truncation - files of different lengths are producing different percentages of truncated samples.

Just for the helluvit, I created four WAVs in Cool Edit (1 kHz tone) of exactly 30, 60, 90, and 120 seconds in length - here are the number of truncated samples for each WAV:

- 30 sec: 4088
- 60 sec: 4080
- 90 sec: 4072
- 120 sec: 4064

Interesting, no? A WAV cut randomly at around 1:48 lost 1,400 samples.

Very odd bug...

edit: grammar

This post has been edited by mixminus1: Oct 6 2010, 15:59


--------------------
"Not sure what the question is, but the answer is probably no."
Go to the top of the page
+Quote Post
tuffy
post Oct 6 2010, 16:05
Post #12





Group: Members
Posts: 111
Joined: 20-August 07
Member No.: 46367



It's worth noting that the default block size for ALAC is 4096 samples. So assuming you're using 44100Hz:

(44100 * 30) % 4096 = 4088
(44100 * 60) % 4096 = 4080
(44100 * 90) % 4096 = 4072
(44100 * 120) % 4096 = 4064

So it looks like you're losing exactly 1 ALAC frame each time for whatever reason.
Go to the top of the page
+Quote Post
twostar
post Oct 6 2010, 16:14
Post #13





Group: Members
Posts: 487
Joined: 5-August 02
From: Manila
Member No.: 2939



I wonder if other pre-iTunes 10 versions are affected as well.
Go to the top of the page
+Quote Post
mixminus1
post Oct 6 2010, 16:39
Post #14





Group: Members
Posts: 687
Joined: 23-February 05
Member No.: 20097



QUOTE (tuffy @ Oct 6 2010, 08:05) *
It's worth noting that the default block size for ALAC is 4096 samples. So assuming you're using 44100Hz:

(44100 * 30) % 4096 = 4088
(44100 * 60) % 4096 = 4080
(44100 * 90) % 4096 = 4072
(44100 * 120) % 4096 = 4064

So it looks like you're losing exactly 1 ALAC frame each time for whatever reason.

Ah, that's right, 4096, just like FLAC (@ 44.1 kHz). wink.gif

Yep, looks like you nailed it: doing a modulo 4096 on all the file lengths posted thus far checks out perfectly, so it looks like the ALAC decoder is indeed truncating one full frame.


--------------------
"Not sure what the question is, but the answer is probably no."
Go to the top of the page
+Quote Post
Corby
post Oct 6 2010, 17:14
Post #15





Group: Members
Posts: 7
Joined: 21-January 02
Member No.: 1103



Same on Mac version of iTunes.
Foobar (running on Wine): "Length mismatch : 4:15.626667 vs 4:15.605261, 11273136 vs 11272192 samples"
No differences when comparing original WAV and ALAC M4A.

Edit: no differences when decoding ALAC M4A to WAV using XLD. So it must be something somewhere inside iTunes WAV handling...

This post has been edited by Corby: Oct 6 2010, 17:18
Go to the top of the page
+Quote Post
Noble Tiger
post Oct 6 2010, 19:19
Post #16





Group: Members
Posts: 6
Joined: 6-October 10
From: Ontario, Canada
Member No.: 84374



It looks like it is a bug in iTunes 10. I didn't notice this in iTunes 9 but I'm also not 100% sure that I tested it thoroughly.

I also can convert the ALAC file back to WAV accurately using dBpoweramp Music Converter, so it must be the iTunes ALAC decoder that's the problem. This problem also occurs with AIFF files converted to ALAC and back, and I would theorize that this would also affect conversion of ALAC to AAC or ALAC to MP3 as well within iTunes. Which is good to be aware of.

You'll notice that sometimes when you convert the WAV or AIFF to ALAC in iTunes 10, if you take a look at the time length for the files, the ALAC file will often be listed as 1 second shorter than the WAV or AIFF (but not always), so that could mean that even playback of ALAC in iTunes 10 could be truncated by a full frame at the end (If this bug is related to the ALAC decoder).

Most likely this problem is related to Quicktime I would guess, as Quicktime does the background encoding/decoding for iTunes doesn't it? Can anyone duplicate this problem using Quicktime only? (I don't have Quicktime Pro myself or I'd try it).

I've reported the bug to Apple and I hope they release an update to either iTunes or Quicktime soon to address this.

This post has been edited by Noble Tiger: Oct 6 2010, 19:24
Go to the top of the page
+Quote Post
C.R.Helmrich
post Oct 6 2010, 20:09
Post #17





Group: Developer
Posts: 686
Joined: 6-December 08
From: Erlangen Germany
Member No.: 64012



QUOTE (mixminus1 @ Oct 6 2010, 16:58) *
Just for the helluvit, I created four WAVs in Cool Edit (1 kHz tone) of exactly 30, 60, 90, and 120 seconds in length - ...

Interesting indeed. Just for the helluvit wink.gif , would you mind trying the same thing with an 11.61 second long 44.1-kHz file? That's 125*4096+1, so there should be 1 sample missing in the decoded WAV.

Chris


--------------------
If I don't reply to your reply, it means I agree with you.
Go to the top of the page
+Quote Post
mixminus1
post Oct 6 2010, 20:28
Post #18





Group: Members
Posts: 687
Joined: 23-February 05
Member No.: 20097



Et viola:

CODE
Differences found in 1 out of 1 track pairs.

Comparing:
"C:\...\tone.1kHz.512001_samples.wav"
"C:\...\tone.1kHz.512001_samples_ALAC.wav"
Length mismatch : 0:11.610000 vs 0:11.609977, 512001 vs 512000 samples


--------------------
"Not sure what the question is, but the answer is probably no."
Go to the top of the page
+Quote Post
Engelsstaub
post Oct 7 2010, 05:50
Post #19





Group: Members
Posts: 556
Joined: 16-February 10
Member No.: 78200



QUOTE (andy o @ Oct 6 2010, 08:29) *
QUOTE (Engelsstaub @ Oct 6 2010, 02:36) *
This is what I got with F2K (and now I know how to do something new...so thanks for that, Robertina.)

So what about the first WAV conversion you made without iTunes? Seems to suggest the "problem" is in iTunes putting the audio in a wav container, not of the ALAC encoder.


I would have to agree.

(...but my knowledge is limited on much of this and I guess I've been making a few assumptions until now. I would have never came across this "problem" had the OP not brought it up. That's the thing about not being a programmer or even working with a CLI once in a while: everything is GUI and I just trust the software I come to like. I always thought of iTunes (however "Apple" or bloated) as a decent ripper and such.

It would never occur to me to go testing the integrity of a dBpoweramp conversion. I hold Spoon's software in the highest esteem.)

...at this point I've become more a "follower" of the thread as you guys appear a bit more technically adept on its subject matter. smile.gif




--------------------
The Loudness War is over. Now it's a hopeless occupation.
Go to the top of the page
+Quote Post
greynol
post Oct 7 2010, 05:54
Post #20





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



The problem with 3rd party encoders/decoders is that ALAC is proprietary and has to be reverse-engineered.

That Apple has botched their very own decoder is quite disconcerting!

This post has been edited by greynol: Oct 7 2010, 05:55


--------------------
YOUR EYES CANNOT HEAR!!!!!!!!!!!
Go to the top of the page
+Quote Post
Engelsstaub
post Oct 7 2010, 06:34
Post #21





Group: Members
Posts: 556
Joined: 16-February 10
Member No.: 78200



QUOTE (greynol @ Oct 6 2010, 23:54) *
The problem with 3rd party encoders/decoders is that ALAC is proprietary and has to be reverse-engineered.

That Apple has botched their very own decoder is quite disconcerting!


Agreed. I was thinking the exact same thing. My last post was getting long-winded enough though.

I don't know why they don't just open the source-code. They could stipulate that it must carry the name Apple Lossless or an acronym like ALAC to make a proper attribution to it's original developers. They don't make a dime off of the format that I'm aware of. And as you've said, it seems now the reverse-engineers know more about what they're doing with it then they do.


--------------------
The Loudness War is over. Now it's a hopeless occupation.
Go to the top of the page
+Quote Post
greynol
post Oct 7 2010, 06:42
Post #22





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



Well, no, I'm not ruling out the possibility that there aren't problems with 3rd party implementations of Apple lossless; rather, I simply steer clear of it.

This post has been edited by greynol: Oct 7 2010, 06:42


--------------------
YOUR EYES CANNOT HEAR!!!!!!!!!!!
Go to the top of the page
+Quote Post
Corby
post Oct 7 2010, 06:46
Post #23





Group: Members
Posts: 7
Joined: 21-January 02
Member No.: 1103



QUOTE (greynol @ Oct 7 2010, 06:54) *
The problem with 3rd party encoders/decoders is that ALAC is proprietary and has to be reverse-engineered.

That Apple has botched their very own decoder is quite disconcerting!


XLD is also using Apple's decoder and writes the WAVs that are identical to original WAV. That probably means that decoder is just fine, and that bug is in iTunes WAV handling routines.
Go to the top of the page
+Quote Post
Engelsstaub
post Oct 7 2010, 06:52
Post #24





Group: Members
Posts: 556
Joined: 16-February 10
Member No.: 78200



If I didn't currently have an iPod I'd likely steer clear of it as well. It has no real practicality for anything other than Apple software and the iPod.

For a while I thought I was being silly about archiving CDs in FLAC and converting some to ALAC for iTunes/iPod. It appears I had a good reason for retaining the FLACs anyway, although I didn't exactly know it at the time. laugh.gif

We always hear people say things like "lossless is lossless." This may be a good lesson in why we should consider one format "better" than another for reasons that were demonstrated here. For me, it just makes me an even bigger fan of FLAC and it's developers.


--------------------
The Loudness War is over. Now it's a hopeless occupation.
Go to the top of the page
+Quote Post
greynol
post Oct 7 2010, 06:57
Post #25





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



I'm part of the camp that thinks lossless on portables is silly.

@Corby, apologies for prompting you to repeat yourself. I wasn't following along carefully enough.


--------------------
YOUR EYES CANNOT HEAR!!!!!!!!!!!
Go to the top of the page
+Quote Post

2 Pages V   1 2 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 22nd July 2014 - 19:58