IPB

Welcome Guest ( Log In | Register )

Nine different codecs 100-pass recompression test
bernhold
post Mar 23 2013, 19:12
Post #1





Group: Members
Posts: 41
Joined: 22-March 13
Member No.: 107330



Hi everyone!

I lately discovered this forum and enjoyed reading the listening tests. I decided to run a listening test myself. Have you ever wondered how different codecs are affected by re-encoding / re-compressing? Of course, recompressing audio is a bad idea, but sometimes can't be avoided. To clear things up, I did a test with the following encoders:
  • WMA Professional 10 (wmapro)
  • WMA 9.2 (wma)
  • Musepack (mpc)
  • Fraunhofer Mp3 surround encoder (mp3s)
  • LAME (mp3)
  • Quicktime AAC (qaac)
  • Nero AAC (nero)
  • Vorbis OGG (vorbis)
  • Opus (opus)

Quality settings: Low (~96 kbps) and high (~256 kbps)
Bitrate modes: CBR, ABR and VBR

I encoded the original sample with the respective encoder, decoded it back to WAV and encoded it again, for 100 times. Then I listened to the results to determine which encoder produced the best results.

RESULTS

AAC is the clear winner by far. It is virtually unaffected by the number of passes. All other codecs had degraded sound quality increasing with the number of encoding passes, especially at low bitrates.

At low bitrates, AAC was the only codec providing satisfactory results. All other encoders fall way behind and produce audible compression artifacts such as cracking noises, muffled sound and hissing. At high bitrates, LAME and Musepack can compete with AAC, but all other encoders fall way behind.

It's interesting to see how much encoders profit from an increased bitrate when recompressing many times. For AAC, as the clear winner, it didn't matter. That being said, Musepack placed 9th with low bitrate settings, but at high bitrate, it was almost as good as AAC and placed 4th. This is similar to LAME, which produced loud cracking noises at low bitrates and placed 8th, but sounded almost perfect at high bitrates and placed 3rd.

Other codecs were mainly unaffected by bitrate, such as WMA, the Fraunhofer MP3s encoder, Opus and OGG Vorbis. These codecs were mainly affected by the number of recompression passes.

In general, WMA and the Fraunhofer MP3s codec were the most disappointing. WMA produced loud hissing and cracking noises, while the Fraunhofer encoder sounded bland and muffling, discarding brilliance and detail. The only reason Fraunhofer placed decent is that it doesn't produce loud cracking or hissing noises, which to my ears is even worse than just muffled or dull sound. Of course, that's purely subjective.

Some encoders did not only degrade sound quality, but also had some other quirks. For example, the LAME encoder lowers the volume with every encoding pass. The 100th pass was virtually inaudible. I had to normalize the audio to hear anything at all. Other encoders produced erroneous files and garbage. The Fraunhofer encoder added silence to the beginning and end of each file and repeated parts of the sample at the end. After 100 passes, it created a 12 seconds file (the original file was 7 seconds). Winamp and Foobar2000 even reported a length of 1:02 minutes for the Fraunhofer file, however the playback ended after 12 seconds. The Vorbis encoder did a similar thing, which resulted in a reported length of 2 seconds, while the playback ended at 7 seconds. I can't really say if I did something fundamentally wrong or if it's the encoders fault, but in the end, the Fraunhofer and Vorbis encoders produced corrupted files. For the listening test, I tried to fix all errors like added silence or corrupted files, since I wanted to judge the sound quality only.

You can view the complete test on my homepage. There, I also have attached the test audio samples so you can hear them in your browser. I also visualized the waveform of each sample, it's very interesting to see.

http://bernholdtech.blogspot.de/2013/03/Ni...ssion-test.html

For example, this is the original file:



This is after 100 re-encodings with Nero AAC:



And this is after 100 re-encodings with OGG Vorbis:



This is after 100 re-encodings with WMA (Windows Media Audio):



This post has been edited by bernhold: Mar 23 2013, 19:38
Go to the top of the page
+Quote Post
 
Start new topic
Replies
romor
post Mar 23 2013, 22:04
Post #2





Group: Members
Posts: 680
Joined: 16-January 09
Member No.: 65630



I run your files through Python (from yesterday's waveform thread), as it also colors waveform on spectral intensity.

Here is result: http://db.tt/q9gXzysF


--------------------
scripts: http://goo.gl/M1qVLQ
Go to the top of the page
+Quote Post

Posts in this topic
- bernhold   Nine different codecs 100-pass recompression test   Mar 23 2013, 19:12
- - zerowalker   Interesting. Though i am surprised that Vorbis did...   Mar 23 2013, 19:35
|- - Arnold B. Krueger   QUOTE (zerowalker @ Mar 23 2013, 14:35) I...   Mar 25 2013, 14:42
|- - Mach-X   QUOTE (zerowalker @ Mar 23 2013, 13:35) I...   Mar 26 2013, 07:09
|- - eahm   QUOTE (Mach-X @ Mar 25 2013, 23:09) ...   Mar 26 2013, 17:17
|- - lvqcl   QUOTE (Mach-X @ Mar 26 2013, 10:09) ...   Mar 26 2013, 17:40
|- - Mach-X   QUOTE (lvqcl @ Mar 26 2013, 11:40) QUOTE ...   Mar 27 2013, 05:39
|- - lvqcl   QUOTE (Mach-X @ Mar 27 2013, 08:39) ...   Mar 27 2013, 17:07
- - bernhold   I read somewhere (Wikipedia, I think) that the imp...   Mar 23 2013, 19:37
|- - alter4   QUOTE (bernhold @ Mar 23 2013, 21:37) I r...   Mar 23 2013, 20:21
- - saratoga   You probably need to manually adjust the encoders ...   Mar 23 2013, 19:42
|- - bernhold   QUOTE (saratoga @ Mar 23 2013, 19:42) You...   Mar 23 2013, 20:20
- - me7   Wow, you uploaded all files playable within the br...   Mar 23 2013, 19:46
- - hlloyge   I was always wondering about this, but neverh had ...   Mar 23 2013, 20:17
- - zima   Hm, reencoding 100 times seems like a bit of an ov...   Mar 23 2013, 21:42
- - bernhold   Yes, 100 times is not practical But there's ...   Mar 23 2013, 21:53
- - romor   I run your files through Python (from yesterday...   Mar 23 2013, 22:04
|- - dhromed   This test is very comprehensive! Good job. QU...   Mar 24 2013, 03:16
- - zima   QUOTE (bernhold @ Mar 23 2013, 21:53) I e...   Mar 23 2013, 22:10
- - dgauze   It seems as though this test would be more suited ...   Mar 23 2013, 22:56
- - saratoga   After 100 passes, rounding error probably starts t...   Mar 24 2013, 02:19
- - kjoonlee   Does this belong under "Listening Tests...   Mar 24 2013, 04:16
- - greynol   No this does not belong in listening tests and wil...   Mar 24 2013, 06:16
- - Mach-X   While perhaps it doesnt belong in listening tests,...   Mar 24 2013, 07:48
- - greynol   Read my post again. You will not see any mention ...   Mar 24 2013, 11:16
- - [JAZ]   @greynol: Could you clarify why this infringes the...   Mar 24 2013, 12:07
|- - romor   QUOTE ([JAZ] @ Mar 24 2013, 13:07...   Mar 24 2013, 16:57
- - IgorC   Don't take me wrong. It's clear than the...   Mar 24 2013, 15:53
- - greynol   Sound quality of lossy codecs is determined though...   Mar 24 2013, 15:58
|- - 2Bdecided   QUOTE (greynol @ Mar 24 2013, 14:58) Soun...   Mar 26 2013, 18:25
|- - Porcus   QUOTE (2Bdecided @ Mar 26 2013, 18:25) e....   Mar 26 2013, 23:44
- - db1989   What would be interesting IMO and, I think, much m...   Mar 24 2013, 18:34
- - romor   Maybe also, something similar to transcoding test ...   Mar 24 2013, 18:48
- - 2Bdecided   Anyone else think there's an error in here? e...   Mar 26 2013, 18:22
- - hankwang   About the noise in the Vorbis sample: I experience...   Mar 27 2013, 12:22
- - Primius   If codec A was better than codec B after 100 itera...   Mar 27 2013, 14:49
|- - greynol   QUOTE (Primius @ Mar 27 2013, 06:49) If c...   Mar 27 2013, 15:00
|- - db1989   QUOTE (Primius @ Mar 27 2013, 13:49) If c...   Mar 27 2013, 15:01
- - Mach-X   db1989 and,greynol 100%, and greynol I hadn't ...   Mar 27 2013, 16:40
- - Mach-X   I see the word "attempt" in there, but n...   Mar 27 2013, 17:24
|- - Nick.C   QUOTE (Mach-X @ Mar 27 2013, 16:24) ...   Mar 27 2013, 19:11
|- - saratoga   QUOTE (Mach-X @ Mar 27 2013, 11:24) ...   Mar 27 2013, 19:47
- - Mach-X   Precisely the point I was getting at. At the bitra...   Mar 28 2013, 02:14
- - Spikey   QUOTE Anyway, in case it hasnít already been said ...   Apr 9 2013, 18:39


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 31st October 2014 - 06:49