IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
LAME 3.98.2 VBR bitrate test, all -V settings in 0.5 step increments
Alex B
post Nov 24 2008, 18:31
Post #1





Group: Members
Posts: 1303
Joined: 14-September 05
From: Helsinki, Finland
Member No.: 24472



Over three years ago I tested the LAME 3.97b1 VBR bitrates (link). For that test I gathered a set of 25 various "reference" tracks. The tracks were specifically selected so that they could more or less represent an average large media library of pop/rock/jazz etc genres. Since then the same set has served well in many occasions. During the preparations of the new public 128 kbps VBR MP3 listening test I gathered another set of classical tracks.

Now I have tested all LAME 3.98.2 VBR settings (in 0.5 step increments) using these two reference sets. For this test I encoded 1050 MP3 files (21x50). I analyzed the files with EncSpot Pro v.2.1 and prosessed the exported data with Excel.

Here are the results:

Some general details: (the frequency values are kHz)


Bitrates "Various":


Bitrates "Classical":


Average Bitrates:




The test tracks:


EDIT:
The classical track #4 has a slightly incorrect name in the table. It should be "Berlin Radio Symphony Orchestra - Mahler, Symphony No 8, 2. Ewiger Wonnebrand".


The source data is available here: http://www.hydrogenaudio.org/forums/index....st&p=601002 (Excel sheet)

This post has been edited by Alex B: Dec 3 2008, 11:03


--------------------
http://listening-tests.freetzi.com
Go to the top of the page
+Quote Post
guruboolez
post Nov 24 2008, 19:06
Post #2





Group: Members (Donating)
Posts: 3474
Joined: 7-November 01
From: Strasbourg (France)
Member No.: 420



Excellent and useful work. This table should appear on the wiki.

On my own classical music test set (bigger: 150 tracks from different discs / 16 hours of music), results on the few settings I tested before are very similar:
-V5: 133.55 kbps (+2.7 kbps)
-V4.5: 145.36kbps (+2.8 kbps)
Go to the top of the page
+Quote Post
Alexxander
post Nov 24 2008, 19:15
Post #3





Group: Members
Posts: 463
Joined: 15-November 04
Member No.: 18143



Thanks for sharing your helpfull information. The data presented together with my personal listening experience is that V3 is a very nice spot concerning quality versus quantity (lame 3.97 and 3.98).
Go to the top of the page
+Quote Post
DigitalDictator
post Nov 24 2008, 19:42
Post #4





Group: Members
Posts: 313
Joined: 9-August 02
From: SoFo
Member No.: 3002



I wonder what the plot (or chart) would look like adding the -Y switch above -V3
Go to the top of the page
+Quote Post
melomaniac
post Nov 24 2008, 19:49
Post #5





Group: Members
Posts: 43
Joined: 1-August 08
From: Brussels
Member No.: 56565



Thank you for this interesting bitrate table. Very useful and helpful for the community.
Nice presentation too. smile.gif
Go to the top of the page
+Quote Post
Alex B
post Nov 24 2008, 20:17
Post #6





Group: Members
Posts: 1303
Joined: 14-September 05
From: Helsinki, Finland
Member No.: 24472



QUOTE (guruboolez @ Nov 24 2008, 20:06) *
On my own classical music test set (bigger: 150 tracks from different discs / 16 hours of music), results on the few settings I tested before are very similar:
-V5: 133.55 kbps (+2.7 kbps)
-V4.5: 145.36kbps (+2.8 kbps)

Interestingly the average values that EncSpot reported directly are even closer to your values:
Classical -V5 133.1 kbps
Classical -V4.5 144.7kbps

I noticed that the average values calculated by Excel always differed slightly from the EncSpot averages. Probably that is caused by rounding. EncSpot may internally use greater precision than it shows for individual tracks. I could have used the EncSpot averages, but it would have looked odd if the average values could not have been verified by calculating them from the given track values.

In any case this kind of a test can only give a rough estimation, so the exact precision does not matter that much.

This post has been edited by Alex B: Nov 24 2008, 20:20


--------------------
http://listening-tests.freetzi.com
Go to the top of the page
+Quote Post
Alex B
post Nov 24 2008, 20:51
Post #7





Group: Members
Posts: 1303
Joined: 14-September 05
From: Helsinki, Finland
Member No.: 24472



QUOTE (DigitalDictator @ Nov 24 2008, 20:42) *
I wonder what the plot (or chart) would look like adding the -Y switch above -V3

It would be more useful to test that with tracks that really exploit the SFB21 bitrate bloat problem. I avoided the most extreme tracks when I gathered the test sets. Perhaps someone who has a nice media library of Metal/Industrial/etc genres could do the comparison.


--------------------
http://listening-tests.freetzi.com
Go to the top of the page
+Quote Post
/mnt
post Nov 24 2008, 22:14
Post #8





Group: Members
Posts: 697
Joined: 22-April 06
Member No.: 29877



QUOTE (Alex B @ Nov 24 2008, 20:51) *
QUOTE (DigitalDictator @ Nov 24 2008, 20:42) *
I wonder what the plot (or chart) would look like adding the -Y switch above -V3

It would be more useful to test that with tracks that really exploit the SFB21 bitrate bloat problem. I avoided the most extreme tracks when I gathered the test sets. Perhaps someone who has a nice media library of Metal/Industrial/etc genres could do the comparison.

I have plenty of tracks that suffer from the sfb21 bloat problem.


--------------------
"I never thought I'd see this much candy in one mission!"
Go to the top of the page
+Quote Post
senab
post Nov 24 2008, 22:21
Post #9





Group: Members
Posts: 247
Joined: 4-August 05
From: Birmingham, UK
Member No.: 23690



QUOTE (Alex B @ Nov 24 2008, 20:17) *
I noticed that the average values calculated by Excel always differed slightly from the EncSpot averages. Probably that is caused by rounding. EncSpot may internally use greater precision than it shows for individual tracks. I could have used the EncSpot averages, but it would have looked odd if the average values could not have been verified by calculating them from the given track values.


I'd guess that EncSpot takes into account padding or maybe the id3 buffer space, and just calculates the bitrate on actual audio data. I'm probably wrong though laugh.gif


--------------------
::.. www.senab.co.uk
Go to the top of the page
+Quote Post
kornchild2002
post Nov 24 2008, 23:25
Post #10





Group: Members
Posts: 2080
Joined: 8-April 05
From: Cincinnati, OH
Member No.: 21277



I saw that the Evanescence track suffered from the bitrate bloat. Both /mnt and I have plenty of tracks that Lame does this with. I can suggest nearly anything by Lamb Of God, All That Remains, Hatebreed, or In Flames. There are also some Nine Inch Nails tracks off of [With_Teeth] and Year Zero that get bloated.
Go to the top of the page
+Quote Post
/mnt
post Nov 24 2008, 23:37
Post #11





Group: Members
Posts: 697
Joined: 22-April 06
Member No.: 29877



And for some reason Powerslave by Iron Maiden really suffers from sbf21 bloating, like it was a very brutal Death Metal album.

This post has been edited by /mnt: Nov 25 2008, 00:02


--------------------
"I never thought I'd see this much candy in one mission!"
Go to the top of the page
+Quote Post
Alex B
post Nov 25 2008, 00:12
Post #12





Group: Members
Posts: 1303
Joined: 14-September 05
From: Helsinki, Finland
Member No.: 24472



Actually I really would not like to start testing these things again. It was a bit exhausting experience. So could someone else test the -Y switch?

In general it might be a good idea to publish a separate metal/industrial/punk/etc (you name it) table in which the -V0 bitrates would probably be near 300 kbps.

Regarding to the bitrate diffrences between the EncSpot calculated and Excel calculated average values I dont understand how the difference can be more than 0.5 kbps if it is caused by rounding. A difference of 0.5 kbps could be possible in the worst case scenario when all tracks happen to be internally something like xxx.499 kbps when EncSpot displays & exports rounded integer numbers (which I used in my Excel table). Maybe EncSpot is buggy afterall.

This post has been edited by Alex B: Nov 25 2008, 00:25


--------------------
http://listening-tests.freetzi.com
Go to the top of the page
+Quote Post
CiTay
post Nov 25 2008, 00:20
Post #13


Administrator


Group: Admin
Posts: 2378
Joined: 22-September 01
Member No.: 3



I updated the bitrate estimates on the LAME page a bit, according to your results.
Go to the top of the page
+Quote Post
/mnt
post Nov 25 2008, 01:27
Post #14





Group: Members
Posts: 697
Joined: 22-April 06
Member No.: 29877



I have made a table of 29 sfb21 bloated tracks at V3 - V0 and the -Y switch aswell.



A zipped Excel document version can be found here.

This post has been edited by /mnt: Nov 25 2008, 02:14


--------------------
"I never thought I'd see this much candy in one mission!"
Go to the top of the page
+Quote Post
skelly831
post Nov 25 2008, 07:01
Post #15





Group: Members (Donating)
Posts: 782
Joined: 11-April 05
From: México
Member No.: 21361



Great info guys! the ~60kbps savings with -Y are impressive, thanks for taking the time to make these tests smile.gif


--------------------
we was young an' full of beans
Go to the top of the page
+Quote Post
tosse
post Nov 25 2008, 08:45
Post #16





Group: Members
Posts: 6
Joined: 9-May 06
From: Sweden
Member No.: 30612



QUOTE (Alex B @ Nov 24 2008, 21:17) *
I noticed that the average values calculated by Excel always differed slightly from the EncSpot averages. Probably that is caused by rounding. EncSpot may internally use greater precision than it shows for individual tracks. I could have used the EncSpot averages, but it would have looked odd if the average values could not have been verified by calculating them from the given track values.

Perhaps encspot uses a weighted average, weighted on the track durations that is?
Go to the top of the page
+Quote Post
Alex B
post Nov 25 2008, 12:59
Post #17





Group: Members
Posts: 1303
Joined: 14-September 05
From: Helsinki, Finland
Member No.: 24472



Thanks tosse. I did some tests and I think you are right. I should have thought about that.

When EncSpot calculates the track specific average bitrates it naturally takes the amount of each differently sized frame into account. Logically and practically it is correct to do the same when the average bitrate of a bunch of files is calculated.

If there is a 30 s 80 kbps file and a 2 min 320 kbps file EncSpot correctly reports that the average bitrate is 272 kbps instead of 200 kbps.

I'll add the EncSpot calculated average bitrates to my test results.

EDIT

On a second thought, for the purpose of this test, I am not interested about the mathematically correct disk space usage / audio duration ratio of my complete test file sets.

I am trying find out how the encoder behaves with various kinds of source material. In that sense each source track should probably be weighted equally. For instance, I may have a difficult to encode shorter track that represents a certain type of source material and an easy to encode longer track that represents another type of source material.

In any case this is not going to be an exact science.

This post has been edited by Alex B: Nov 25 2008, 14:01


--------------------
http://listening-tests.freetzi.com
Go to the top of the page
+Quote Post
Pio2001
post May 31 2009, 19:31
Post #18


Moderator


Group: Super Moderator
Posts: 3936
Joined: 29-September 01
Member No.: 73



Hello,
A bit surprised by the low target bitrate of the -V5 setting in the wiki, which is 130 kbps, I also ran a small test in order to check Lame 3.98.2 bitrates.

I chose 10 tracks from various styles. It was before finding this thread, and I got results very similar to yours, exept for a small difference for the -V2 setting.

Our results are indeed a bit different from what is given in the wiki.

CODE
    V6      V5      V4      V3      V2
1   110.4   120.4   142.9   156.8   170.7
2   114.9   131.5   150.9   164.4   181.1
3   112.5   124.6   154.8   174.3   192.2
4   124     136.6   162.5   177.2   191.6
5   121.8   138.6   156.6   171     184
6   135.8   151.5   167.4   181.5   201.4
7   129.8   143.4   160     174.9   210
8   129.5   143.4   157.2   171.4   239.6
9   127.4   141.1   155.2   169     200.5
10  125.5   139.1   152.4   166.2   229.8
-----------------------------------------------
Avg 123.16  137.02  155.99  170.67  200.09


The tracks are :

1 : Ashram - All'Imbruinire (Soft piano and voice)
2 : Grieg - Peer Gynt - I Dovregubbens Hall, Neeme Järvi (Classical, symphonic, extremely dynamic)
3 : Graduel d'Aliénor de Bretagne - Kyrie, Ensemble Organum (Classical, choir)
4 : Faun - Isis (folk)
5 : Princess Mononoke soundtrack - The Journey to the West (orchestral soundtrack)
6 : Kraftwerk - Trans Europ Express / Metal on Metal / Franz Shubert (electro pop, 1977)
7 : Madonna - Hung Up (mainstream pop, 2005)
8 : Nightwish - Whishmaster (metal)
9 : Talk Talk - Such a Shame (new wave pop, 1984)
10 : Within Temptation - What Have You Done (clipressed pop)
Go to the top of the page
+Quote Post
MClemo
post Jun 1 2009, 10:42
Post #19





Group: Members
Posts: 12
Joined: 5-December 06
Member No.: 38391



On my collection of 12579 files I got an average bitrate of 228.23 kbps with -V 0 and 144.31 kbps with -V 4. That is with track length not weighed in and all lame 3.98.2. The flac source files are at 759.02 kbps. I guess the classical and spoken word brings it down that far (about 2000 files). Plus not that much loud metal, hard rock, etc.
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 21st October 2014 - 02:11