IPB

Welcome Guest ( Log In | Register )

2 Pages V   1 2 >  
Reply to this topicStart new topic
Optimized MPEG ALS encoders, speed optimizations
Garf
post Jan 9 2006, 18:19
Post #1


Server Admin


Group: Admin
Posts: 4883
Joined: 24-September 01
Member No.: 13



I spent some time with the MPEG ALS reference code, and applied several code and algorithmic optimizations, as well as builds with different compilers. The result seems to be a speedup between 0 and 200%, depending on the exact settings used.

I did not use the included lpc_adapt.o, but instead wrote my own algorithms for adaptive determination of the LPC order. It seems to work well, but it is slightly inferior in ratio compared to the shipped one, but it compensates that a bit by being faster.

If you are playing with ALS the following settings seem interesting from a speed/ratio point of view:

"Fast": -o5
"Default":
"High": -a -o30
"Extra High": -a -b -o50

Settings like -t, -p and -z1, -z2 and z3 don't really provide a seizable gain for large increases in encoding and decoding time. However, for maximal compression -z3 seems to be best.

Binaries can be downloaded at

http://sjeng.org/ftp/lossless/

[Edit: there's a single binary per arechitecture now]
For Intel machines mp4als_intel should be best. I did not benchmark on AMD's yet.

Please note that patents may apply to MPEG ALS, and that you may need a patent license for legal use of this program.

This post has been edited by Garf: Mar 5 2006, 10:48
Go to the top of the page
+Quote Post
tedgo
post Jan 9 2006, 20:29
Post #2





Group: Members
Posts: 1089
Joined: 16-April 04
From: Bavaria, Germany
Member No.: 13548



Good news.
So mpeg4als could be usable one day wink.gif

I tested your otimized mp4als_intel.exe on my old machine and compared it with the "original" mp4alsRM16 (made the test with the same track as the flac-test, so don't wonder about the "bad" overall compresion ratio wink.gif)
-p doesn't work on my machine. The *.exe refuses to work with that switch.

CODE
Testsystem:

CPU:      1.5GHz P4 "Williamette"
RAM:      512 MB SD-RAM PC133
HDD:      WD 80GB 7.200upm
OS:      WinXP SP2

=================================================================================


MODE        SPEED x
   ENCODE      DECODE    RATIO

MPEG-4 ALS RM16
-o5             14,60  42,25   63,29%
Default         10,05  31,91   62,94%
-p               2,14  24,79   62,80%
-a -o30         11,98  22,81   62,85%
-a -b -o50       7,52  12,77   62,29%
-7               0,31   4,73   61,83%
-7 -p            0,17   4,65   61,82%
-z1              0,43   0,43   62,05%
-z1 -p           0,40   0,40   61,99%
-z2              0,23   0,23   61,53%
-z2 -p           0,22   0,22   61,51%
-z3              0,14   0,14   61,31%
-z3 -p           0,14   0,14   61,29%

MPEG-4 ALS optimized by Garf (mp4als_intel.exe)
-o5             33,33  48,00   63,29%
Default         24,49  36,81   62,93%
-p              CRASH  -----   ------
-a -o30         16,35  25,21   62,85%
-a -b -o50      10,33  15,31   62,29%
-7               0,69  10,42   61,96%
-7 -p           CRASH  -----   ------
-z1              0,46   0,45   62,05%
-z1 -p          CRAsH  -----   ------
-z2              0,23   0,23   61,53%
-z2 -p          CRASH  -----   ------
-z3              0,14   0,14   61,31%
-z3 -p          CRASH  -----   ------


This post has been edited by tedgo: Jan 9 2006, 20:43
Go to the top of the page
+Quote Post
Garf
post Jan 9 2006, 20:35
Post #3


Server Admin


Group: Admin
Posts: 4883
Joined: 24-September 01
Member No.: 13



Do the other binaries exhibit the same behaviour?

I didn't touch the -z and -p stuff.
Go to the top of the page
+Quote Post
tedgo
post Jan 9 2006, 20:37
Post #4





Group: Members
Posts: 1089
Joined: 16-April 04
From: Bavaria, Germany
Member No.: 13548



I only tested the mp4als_intel until now.
Wait a minute...

EDIT:
No, only mp4als_intel refuses to work with -p
mp4als_sse2 and mp4als_gen are working.
Will make a test with this two versions later at night.

EDIT2: (added new tests with mp4als_gen and mp4als_sse2, but without -z-switches, because they are too slow for realtime decoding, at least on my system...)
CODE
Testsystem:

CPU:   1.5GHz P4 "Williamette"
RAM:   512 MB SD-RAM PC133
HDD:   WD 80GB 7.200upm
OS:    WinXP SP2

=================================================================================


MODE               SPEED x
              ENCODE   DECODE   RATIO

MPEG-4 ALS RM16
-o5            14,60     42,25   63,29%
Default        10,05     31,91   62,94%
-p              2,14     24,79   62,80%
-a -o30        11,98     22,81   62,85%
-a -b -o50      7,52     12,77   62,29%
-7              0,31      4,73   61,83%

MPEG-4 ALS optimized by Garf (mp4als_intel.exe)
-o5            33,33     48,00   63,29%
Default        24,49     36,81   62,93%
-p             CRASH     -----   ------
-a -o30        16,35     25,21   62,85%
-a -b -o50     10,33     15,31   62,29%
-7              0,69     10,42   61,96%

MPEG-4 ALS optimized by Garf (mp4als_sse2.exe)
-o5            33,90     45,80   63,29%
Default        25,64     35,93   62,93%
-p              2,55     27,78   62,80%
-a -o30        16,71     25,75   62,85%
-a -b -o50     11,09     16,13   62,28%
-7              0,73     10,79   61,96%

MPEG-4 ALS optimized by Garf (mp4als_gen.exe)
-o5            33,71     45,80   63,29%
Default        25,21     35,50   62,93%
-p              2,54     27,65   62,80%
-a -o30        16,53     25,21   62,85%
-a -b -o50     10,89     15,92   62,29%
-7              0,72     10,75   61,96%

Very effective speed-up with all settings compared to the "original" mp4alsRM16.
Now i could die for a directshow-decoder and/or a foobar2000-plugin, because compression ratio and speed is near by my so far prefered wavpack wink.gif

Very excellent work!!!
Its a real speed-up of the codec.
Is it still specs-compliant or have you changed anything that could break the specs?

This post has been edited by tedgo: Jan 10 2006, 00:32
Go to the top of the page
+Quote Post
xmixahlx
post Jan 10 2006, 01:22
Post #5





Group: Members
Posts: 1394
Joined: 20-December 01
From: seattle
Member No.: 693



i haven't been following the MPEG4-ALS stuff, but anyways - is the source available? and are your changes available?

i.e. the possibility of a *nix binary...


later

p.s. - how is the european comma-instead-of-a-period not confusing?


--------------------
RareWares/Debian :: http://www.rarewares.org/debian.html
Go to the top of the page
+Quote Post
Shade[ST]
post Jan 10 2006, 01:30
Post #6





Group: Members
Posts: 1189
Joined: 19-May 05
From: Montreal, Canada
Member No.: 22144



Is it me, or from the benchmarks is Garf's version slower than the ref?
Go to the top of the page
+Quote Post
sh1leshk4
post Jan 10 2006, 01:53
Post #7





Group: Members
Posts: 403
Joined: 2-October 04
Member No.: 17436



The numbers probably indicate encode speed time against realtime.
Like 14x, 33x, so on and so forth; not in seconds or minutes.


~probably...
Go to the top of the page
+Quote Post
Speek
post Jan 10 2006, 01:59
Post #8





Group: Members
Posts: 394
Joined: 31-October 01
Member No.: 386



QUOTE (xmixahlx @ Jan 10 2006, 01:22 AM)
i haven't been following the MPEG4-ALS stuff, but anyways - is the source available?


Yes, the reference codec is available from this page:
http://www.nue.tu-berlin.de/forschung/proj...ess/mp4als.html
Go to the top of the page
+Quote Post
skamp
post Jan 10 2006, 04:38
Post #9





Group: Developer
Posts: 1412
Joined: 4-May 04
From: France
Member No.: 13875



Would you mind posting the modified source code as well? I'd like to try to build it under linux...


--------------------
See my profile for measurements, tools and recommendations.
Go to the top of the page
+Quote Post
tedgo
post Jan 10 2006, 10:24
Post #10





Group: Members
Posts: 1089
Joined: 16-April 04
From: Bavaria, Germany
Member No.: 13548



@Shade[ST] and sh1leshk4
The numbers on speed means "encode speed time against realtime" not "seconds".
See the header of the table (Sorry if its been mistakable wink.gif)
I used this way because a one minute song and a ten minute song would be both encoded with Garf's optimization at about 33 times realtime with -o5, while it took about 1,8 seconds for a one minute song but about 18 seconds for a ten minute song.

So "-o5" or "Default" with Garf's version is about twice as fast as the reference in encoding and about 10% faster in decoding.

Btw. would be great if mpeg4als would make the way into nero to make it usable in mp4. With hardware-support it could be THE lossless audio standard in the future.
I'm awaiting mpeg4sls now to close the gap between aac and als wink.gif

This post has been edited by tedgo: Jan 10 2006, 10:53
Go to the top of the page
+Quote Post
Garf
post Jan 10 2006, 10:59
Post #11


Server Admin


Group: Admin
Posts: 4883
Joined: 24-September 01
Member No.: 13



QUOTE (tedgo @ Jan 9 2006, 09:37 PM)
Very effective speed-up with all settings compared to the "original" mp4alsRM16.
Now i could die for a directshow-decoder and/or a foobar2000-plugin, because compression ratio and speed is near by my so far prefered wavpack  wink.gif

Very excellent work!!!
Its a real speed-up of the codec.
Is it still specs-compliant or have you changed anything that could break the specs?
*


It's fully spec-compliant.

I guess the real wait is for MP4 embedding to be finalized, so you can have ALS album images + chapters in MP4 files.
Go to the top of the page
+Quote Post
Garf
post Jan 10 2006, 11:02
Post #12


Server Admin


Group: Admin
Posts: 4883
Joined: 24-September 01
Member No.: 13



QUOTE (xmixahlx @ Jan 10 2006, 02:22 AM)
i haven't been following the MPEG4-ALS stuff, but anyways - is the source available?


Yes. Source for the reference encoder is available, minus adaptive LPC (which isn't needed for a working encoder, but is needed for making a good one).

QUOTE
and are your changes available?


Not yet. I will have to see what I do with this.

This post has been edited by Garf: Jan 10 2006, 11:02
Go to the top of the page
+Quote Post
Garf
post Jan 10 2006, 11:06
Post #13


Server Admin


Group: Admin
Posts: 4883
Joined: 24-September 01
Member No.: 13



QUOTE (tedgo @ Jan 10 2006, 11:24 AM)
Btw. would be great if mpeg4als would make the way into nero to make it usable in mp4. With hardware-support it could be THE lossless audio standard in the future.
I'm awaiting mpeg4sls now to close the gap between aac and als wink.gif
*


Well I did this to learn some things about lossless codecs, don't read anything into this related to what my employer may or may not do.

ALS is nice, but it's also patented, and there are alternatives that are not. We will have to see how things pan out in the industry.
Go to the top of the page
+Quote Post
Garf
post Jan 10 2006, 11:11
Post #14


Server Admin


Group: Admin
Posts: 4883
Joined: 24-September 01
Member No.: 13



By the way, quite a bit of encode and decode speedup should be possible with some assembler coding of the critical loops. The compiler doesn't seem do be doing a very good job there.

Also, ALS should gain massively in speed on 64 bit machines. I think I can compile a 64 bit version later this week.

This post has been edited by Garf: Jan 10 2006, 11:11
Go to the top of the page
+Quote Post
tedgo
post Jan 10 2006, 12:39
Post #15





Group: Members
Posts: 1089
Joined: 16-April 04
From: Bavaria, Germany
Member No.: 13548



QUOTE (Garf @ Jan 10 2006, 12:06 PM)
Well I did this to learn some things about lossless codecs, don't read anything into this related to what my employer may or may not do.

ALS is nice, but it's also patented, and there are alternatives that are not. We will have to see how things pan out in the industry.
*

Would be too bad if mpeg4als wouldn't find the way into an application like nero, only because its patented.
Aac and mpeg4 part 10 are also patented (correct me if i'm wrong) and did the way into nero. And mpeg4als is developed as the lossless audio standard for the mp4-specs. So it offers itself as the lossless audio solution for mp4. A very interesting solution for audio storage or video soundtrack in mp4 on the forthcoming blue ray disc or hd-dvd.
Nero has made the first step, now its time to go the next one biggrin.gif wink.gif

This post has been edited by tedgo: Jan 10 2006, 12:40
Go to the top of the page
+Quote Post
Garf
post Jan 10 2006, 12:49
Post #16


Server Admin


Group: Admin
Posts: 4883
Joined: 24-September 01
Member No.: 13



Yes, but HE-AAC and AVC don't really have free alternatives that are as good, and they are already accepted by the industry.

The industry currently seems to have adopted FLAC as far as lossless storage goes. Whether they are willing to switch to ALS, for, well, marginal benefits, but extra fees, remains to be seen.

There's also the issue of MPEG SLS being up-and-coming, too.

Encoding a video soundtrack in ALS seems very silly to me. What's wrong with AAC? It's not like you store the video losslessy smile.gif

This post has been edited by Garf: Jan 10 2006, 12:50
Go to the top of the page
+Quote Post
tedgo
post Jan 10 2006, 12:59
Post #17





Group: Members
Posts: 1089
Joined: 16-April 04
From: Bavaria, Germany
Member No.: 13548



I think flac is only adopted by the industry because it is very popular, but flac has still one of the worst compression ratios around, even with -8.
With a lossless codec like mpeg4als, specified for the widely used mp4-container and its compatibility "guaranteed" through the specs, preferences of industry could change it in the future.
And what about drm? Its hated by the users, but loved by the industry...
You'll see, it will not last long until the first online musicstore will sell drm'ed lossless files (for far too much money...) wink.gif
Maybe the real musicstore? After all real networks are one of the partners improved the old lpac-codec to develop mpeg4als...

Btw. video soundtrack in a lossless format does make sense!
Think of own recordings, not only dvd-rips. Or music-videos.
I used wavpack and xvid quant1 or x264 lossless in mkv so far, but would immediately change to nero avc and mpeg4als in mp4.

This post has been edited by tedgo: Jan 10 2006, 13:28
Go to the top of the page
+Quote Post
skamp
post Jan 10 2006, 13:36
Post #18





Group: Developer
Posts: 1412
Joined: 4-May 04
From: France
Member No.: 13875



QUOTE (tedgo @ Jan 10 2006, 12:59 PM)
You'll see, it will not last long until the first online musicstore will sell drm'ed lossless files (for far too much money...) wink.gif
*

It's already the case.


--------------------
See my profile for measurements, tools and recommendations.
Go to the top of the page
+Quote Post
SebastianG
post Jan 10 2006, 13:38
Post #19





Group: Developer
Posts: 1317
Joined: 20-March 04
From: Göttingen (DE)
Member No.: 12875



QUOTE (Garf @ Jan 10 2006, 11:06 AM)
ALS is nice, but it's also patented [...]
*

Is it ? What parts ?
It's basically linear prediction + residual coding only, isn't it ? Some small differences compared to FLAC are:
- coding of LPC filter via non-uniformly quantized reflection coefficients instead of direct lpc coeffs (already done on LPC10 I suppose)
- entropy coding (is arithmetic coding mandatory??)
I don't know anything about ALS' interchannel prediction. Is it smart / patented ?


Sebi

This post has been edited by SebastianG: Jan 10 2006, 13:39
Go to the top of the page
+Quote Post
Lyx
post Jan 10 2006, 13:49
Post #20





Group: Members
Posts: 3353
Joined: 6-July 03
From: Sachsen (DE)
Member No.: 7609



QUOTE (tedgo @ Jan 10 2006, 12:59 PM)
I think flac is only adopted by the industry because it is very popular, but flac has still one of the worst compression ratios around, even with -8.
*

On average, this "worst" ratio is just 4% behind monkey's audio. You and many ha.org users may care about such small difference - but in the real world, 4% is insignificant.


--------------------
I am arrogant and I can afford it because I deliver.
Go to the top of the page
+Quote Post
tedgo
post Jan 10 2006, 13:50
Post #21





Group: Members
Posts: 1089
Joined: 16-April 04
From: Bavaria, Germany
Member No.: 13548



@skamp
Yes, but only in the unpopular wma lsl format and only available in the U.S.
I think mpeg4als in mp4 would be more accepted, only because its not from microsoft... wink.gif
Although i don't understand the reservations against wma lsl! It often has a better compression ratio as the popular flac (but is much slower at least on my machine).
But i would still prefer mpeg4als, because it is muxable (or will be) in mp4-videos.

@Lyx
4% may be significant if you have a music library with thousands of lossless songs.

This post has been edited by tedgo: Jan 10 2006, 13:53
Go to the top of the page
+Quote Post
Lyx
post Jan 10 2006, 13:59
Post #22





Group: Members
Posts: 3353
Joined: 6-July 03
From: Sachsen (DE)
Member No.: 7609



You didn't really read what i wrote, right? The industry is not about you - it cares about normal users, not geeks. All the major lossless formats are in a marging of 4% difference in compression ratio: for normal users, this means that all major lossless compressors are *equal* in regards to ratio.

Now, if you are a big company, and you only have two criterias to base your choice on - compression-ratio and marketshare(userbase) - and asuming compression is almost equal, but marketshare differs greatly.... when what are you going to base your decision on, hmm?


--------------------
I am arrogant and I can afford it because I deliver.
Go to the top of the page
+Quote Post
Garf
post Jan 10 2006, 14:03
Post #23


Server Admin


Group: Admin
Posts: 4883
Joined: 24-September 01
Member No.: 13



QUOTE (SebastianG @ Jan 10 2006, 02:38 PM)
QUOTE (Garf @ Jan 10 2006, 11:06 AM)
ALS is nice, but it's also patented [...]
*

Is it ? What parts ?
It's basically linear prediction + residual coding only, isn't it ? Some small differences compared to FLAC are:
- coding of LPC filter via non-uniformly quantized reflection coefficients instead of direct lpc coeffs (already done on LPC10 I suppose)
- entropy coding (is arithmetic coding mandatory??)
I don't know anything about ALS' interchannel prediction. Is it smart / patented ?

Sebi
*



Ha, I thought about this too! The thing is, there's not much to patent in the default mode, but all the extra's (LTP, MCC, BMGC, the exact backwards prediction method used, progressive prediction) probably are patented to some extent.

So, while you could perhaps make a good encoder that is patentfree, a compliant decoder would still have to support the above.

This post has been edited by Garf: Jan 10 2006, 14:07
Go to the top of the page
+Quote Post
tedgo
post Jan 10 2006, 14:47
Post #24





Group: Members
Posts: 1089
Joined: 16-April 04
From: Bavaria, Germany
Member No.: 13548



@Lyx
What should i have read over in your short post?
QUOTE
On average, this "worst" ratio is just 4% behind monkey's audio. You and many ha.org users may care about such small difference - but in the real world, 4% is insignificant.

The real world is big. If you opined the industry with "the real world" you should write it... wink.gif

This discussion would be unnecessary if you'd really read what i wrote:
QUOTE
I think flac is only adopted by the industry because it is very popular, but flac has still one of the worst compression ratios around, even with -8.
With a lossless codec like mpeg4als, specified for the widely used mp4-container and its compatibility "guaranteed" through the specs, preferences of industry could change it in the future.
And what about drm? Its hated by the users, but loved by the industry...
You'll see, it will not last long until the first online musicstore will sell drm'ed lossless files (for far too much money...)
Maybe the real musicstore? After all real networks are one of the partners improved the old lpac-codec to develop mpeg4als...

The compression ratio was just one point which may be (with the other points) of interest for users and therefore the industry in the future.
So, why you get excited? blink.gif

But its enough off-topic, back to the topic's "roots" wink.gif

This post has been edited by tedgo: Jan 10 2006, 14:49
Go to the top of the page
+Quote Post
Garf
post Jan 10 2006, 15:47
Post #25


Server Admin


Group: Admin
Posts: 4883
Joined: 24-September 01
Member No.: 13



64 bit build: http://sjeng.org/ftp/lossless/mp4als_x64.exe

64 bit speedups on AMD X2 4400

Encoding

Default: 85x --> 110x
-7: 2.2x --> 2.5x
-z1: 1.4x --> 3.6x(!!)

Decoding

Default: 105x --> 160x
-a -o50: 70x --> 96x
-7: 43x --> 52x
-z1: 1.4x --> 3.6x

This post has been edited by Garf: Jan 10 2006, 15:58
Go to the top of the page
+Quote Post

2 Pages V   1 2 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 26th July 2014 - 02:07