IPB

Welcome Guest ( Log In | Register )

35 Pages V  « < 28 29 30 31 32 > »   
Closed TopicStart new topic
Multiformat 128 kbps Listening Test, Pre-Test Discussion
Sebastian Mares
post Nov 30 2005, 21:59
Post #726





Group: Members
Posts: 3637
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



QUOTE (sehested @ Nov 30 2005, 09:55 PM)
Sebastian,

The samples are looking great!

Average bitrate between codec is excellent. smile.gif

I look so much forward to performing this test biggrin.gif

Keep up the good work!
*


Thanks. Well, the problem is that with the samples I have ATM, LAME and WMA Pro exceed the 10% tolerance margin (max is 140.8 kbps). Nero AAC is only 0.1 kbps away from it. Maybe the average bitrate can be lowered with a movie dialog sample for example.

This post has been edited by Sebastian Mares: Nov 30 2005, 21:59


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
Alex B
post Nov 30 2005, 22:19
Post #727





Group: Members
Posts: 1303
Joined: 14-September 05
From: Helsinki, Finland
Member No.: 24472



Some comments about the bitrate table:

Did you try this Nero encoder with full tracks? What is the used setting? Does it produce a comparable average bitrate with the other encoders when a big amount of varied complete tracks is encoded?

BTW, did you use -q 4.25 for Vorbis? (I suppose the other encoders have only one possible setting.)


--------------------
http://listening-tests.freetzi.com
Go to the top of the page
+Quote Post
Sebastian Mares
post Nov 30 2005, 22:21
Post #728





Group: Members
Posts: 3637
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



For Vorbis, I used -q 4.
For Nero AAC, I used the recommended streaming profile. On my music collection, it produced an average of 132 kbps.

This post has been edited by Sebastian Mares: Nov 30 2005, 22:21


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
Alex B
post Nov 30 2005, 22:38
Post #729





Group: Members
Posts: 1303
Joined: 14-September 05
From: Helsinki, Finland
Member No.: 24472



In my opinion Vorbis -q 4.25 would be closer with LAME V5n and WMA Pro 50 (I have no large scale experience of WMA Pro 50). The AAC encoders work differently.

guruboolez chose -q 4 for various tracks and 4.25 for classical in his test, but according to the results I have seen now I think 4.25 would have been a correct setting for all genres. Perhaps 4.2 could be used too. I could try if it makes any difference on my test tracks.

Edit: typo

This post has been edited by Alex B: Nov 30 2005, 22:48


--------------------
http://listening-tests.freetzi.com
Go to the top of the page
+Quote Post
yahknow1
post Nov 30 2005, 22:46
Post #730





Group: Members
Posts: 71
Joined: 23-February 05
Member No.: 20082



I'm new to this (still) and excuse this question if it silly or I'm jumping ahead of things, but, when I try saving the clips, they all come up as "index.php"? What am I doing wrong? sad.gif
Go to the top of the page
+Quote Post
Sebastian Mares
post Nov 30 2005, 22:47
Post #731





Group: Members
Posts: 3637
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



QUOTE (yahknow1 @ Nov 30 2005, 10:46 PM)
I'm new to this (still) and excuse this question if it silly or I'm jumping ahead of things, but, when I try saving the clips, they all come up as "index.php"? What am I doing wrong? sad.gif
*


Which browser are you using?


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
yahknow1
post Nov 30 2005, 22:52
Post #732





Group: Members
Posts: 71
Joined: 23-February 05
Member No.: 20082



QUOTE (Sebastian Mares @ Nov 30 2005, 02:47 PM)
QUOTE (yahknow1 @ Nov 30 2005, 10:46 PM)
I'm new to this (still) and excuse this question if it silly or I'm jumping ahead of things, but, when I try saving the clips, they all come up as "index.php"? What am I doing wrong? sad.gif
*


Which browser are you using?
*


IE 6.0...never ran into troubles like this with any other sample?
Go to the top of the page
+Quote Post
user
post Nov 30 2005, 23:14
Post #733





Group: Members
Posts: 873
Joined: 12-October 01
From: the great wide open
Member No.: 277



Some general thoughts about listening tests:

- the test samples of various format contenders themselves don't need to average to a certain bitrate. Any higher or lower bitrate on samples will work, because that is the goal to be tested: "Does the encoder a good job on distributing bitrate to frames ?"
That means, if we have a broad overview, of the average bitrate a format with specified quality setting averages at,
then it is fine. The samples can vary a lot from that targeted bitrate. That is vbr.

- Of course, it is another question, which samples you select. Only killer samples, or normal music, or some mix.

- I suggest selecting samples, which result to ca. the target bitrate, but also samples, which go way above the target bitrate with certain encoders,
but also samples, which result clearly below target bitrate with certain encoders.
So, you test "easy to encode" samples, "average to encode" samples, and "difficult to encode" samples.
It will then be interesting, if various encoders behave differently regarding bitrate sample distribution, and more important: "do they have success, eg. when they give certain samples only a low bitrate, or lower bitrate than other contenders ?"

- Maybe somebody here with a widefold music CD collection, covering several genres, from pop, jazz, classic, rock, techno, audio books, metal etc., should mix some albums, and keep them as reference in a Lossless format, for test encodes with current and upcoming formats/encoders, settings.
So, that we have some HA-reference, which setting yields to which average bitrate.
Eg. vorbis q4 , or q4.25, a q-step of 0.25 might have already some bigger influence in the 100-128k bitrate area, and it should be decided & tested with the vbr formats, that for each format a setting is chosen, which does yield on a broad average of music to same bitrate (with only few percent difference).
Guruboolez did something like this with his classic collection, don't know, if he has some other CDs/genres, which could make it complete.
I could help here, have various classic, jazz, pop, rock etc., but I don#t have installed every format yet. And don#t have the internet/program knowledge, to present the results such good, how guru does it. Or does foobar give everything automatically, or probably a connection/script necessary, to combine foobars values with excel ?
Eg., for 3.97 Lame -V5 --vbr-new, I can confirm, that 130 kbit/s is an average, but it depends a lot on albums/music. Eg., if you would ask somebody, who owns mainly modern clip-compressed pop music, he would say, that lame 3.97 -V5 -vbr-new will yield to 140, not 130k.


--------------------
www.High-Quality.ch.vu -- High Quality Audio Archiving Tutorials
Go to the top of the page
+Quote Post
sehested
post Nov 30 2005, 23:25
Post #734





Group: Members (Donating)
Posts: 325
Joined: 5-April 04
From: Copenhagen, Denmark
Member No.: 13246



QUOTE (Sebastian Mares @ Nov 30 2005, 12:59 PM)
Thanks. Well, the problem is that with the samples I have ATM, LAME and WMA Pro exceed the 10% tolerance margin (max is 140.8 kbps). Nero AAC is only 0.1 kbps away from it. Maybe the average bitrate can be lowered with a movie dialog sample for example.
All the codec is on the high side of the target bitrate of 128 kbps and indeed - as you point out - some are outside the 10% tolerance we have defined.

However the difference between lowest average bitrate (137.0 iTunes) and highest average bitrate (143.1 WMA Pro) is only 4%, which is excellent.

I see no need to adjust the average bitrate for any of the codec.

Although some could argue that the average bitrate is closer to that of a 160 kbps test, I would prefer to think of it as a 128+ kbps test, since 128 kbps VBR results in an average bitrate slightly above 128 kbps.
Go to the top of the page
+Quote Post
Sebastian Mares
post Nov 30 2005, 23:30
Post #735





Group: Members
Posts: 3637
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



I'd also keep the settings as-is and check the final result when I have all samples. If some codecs continue to fall outside 140.8 kbps, I might replace a sample or two.


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
sehested
post Nov 30 2005, 23:36
Post #736





Group: Members (Donating)
Posts: 325
Joined: 5-April 04
From: Copenhagen, Denmark
Member No.: 13246



What's with the sample Cockney Rebel ("Sebastian")?

The file sample I downloaded is WavePack 855 kbps but the sound has severe artifacts.
Go to the top of the page
+Quote Post
user
post Nov 30 2005, 23:41
Post #737





Group: Members
Posts: 873
Joined: 12-October 01
From: the great wide open
Member No.: 277



QUOTE
I'd also keep the settings as-is and check the final result when I have all samples. If some codecs continue to fall outside 140.8 kbps, I might replace a sample or two.


I don't understand this method. The vbr codecs are free in their behaviour.
Only importance of the to be tested quality settings is, that the target bitrate is nearly the same for all contenders, ie. 128k+- over various music genres.


--------------------
www.High-Quality.ch.vu -- High Quality Audio Archiving Tutorials
Go to the top of the page
+Quote Post
Sebastian Mares
post Nov 30 2005, 23:41
Post #738





Group: Members
Posts: 3637
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



QUOTE (sehested @ Nov 30 2005, 11:36 PM)
What's with the sample Cockney Rebel ("Sebastian")?

The file sample I downloaded is WavePack 855 kbps but the sound has severe artifacts.
*


Hmm... It's from "Rock Legends - All-Time Greatest Rock Ballads Volume 1" (track 9). I copied it with EAC, extracted the 35 seconds and encoded them with WavPack.
What kind of artifacts are you talking about? Some that are typical to lossy encoders or something else?


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
sehested
post Nov 30 2005, 23:45
Post #739





Group: Members (Donating)
Posts: 325
Joined: 5-April 04
From: Copenhagen, Denmark
Member No.: 13246



QUOTE (Sebastian Mares @ Nov 30 2005, 02:41 PM)
What kind of artifacts are you talking about? Some that are typical to lossy encoders or something else?
Distortion on singers voice most noticably from 0:09 to 0:20
Go to the top of the page
+Quote Post
Sebastian Mares
post Nov 30 2005, 23:57
Post #740





Group: Members
Posts: 3637
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



QUOTE (sehested @ Nov 30 2005, 11:45 PM)
QUOTE (Sebastian Mares @ Nov 30 2005, 02:41 PM)
What kind of artifacts are you talking about? Some that are typical to lossy encoders or something else?
Distortion on singers voice most noticably from 0:09 to 0:20
*



The tracks were copied from vinyl - maybe that's the reason. unsure.gif


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
Sebastian Mares
post Dec 1 2005, 00:00
Post #741





Group: Members
Posts: 3637
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



QUOTE (user @ Nov 30 2005, 11:41 PM)
QUOTE
I'd also keep the settings as-is and check the final result when I have all samples. If some codecs continue to fall outside 140.8 kbps, I might replace a sample or two.


I don't understand this method. The vbr codecs are free in their behaviour.
Only importance of the to be tested quality settings is, that the target bitrate is nearly the same for all contenders, ie. 128k+- over various music genres.
*



Yes, but if there are a lot of difficult samples and the average bitrate exceeds 128 kbps + 12.8 kbps, it would be better to replace one of the complex samples so the average bitrate remains near what's tested.
Nero for example produces an average of 131-132 kbps with my music collection and that's what Ivan said, too. With these particular samples however, the bitrate is boosted to more than 140 kbps. That's why I think that replacing one sample is appropriate.


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
rjamorim
post Dec 1 2005, 00:13
Post #742


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (kwanbis @ Nov 30 2005, 06:26 PM)
then the test won't be serving its purpose. as stated, an econder would adapt to a whole track diferently than to a 30 secs clip.
*


You could conduce a test yourself. I believe Argentinian law is much nicer than german law, in that aspect.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
Alex B
post Dec 1 2005, 00:16
Post #743





Group: Members
Posts: 1303
Joined: 14-September 05
From: Helsinki, Finland
Member No.: 24472



I have the Steve Harley & Cockney Rebel - The Cream Of compilation which includes Sebastian. I could quickly access my LAME 3.90.3 --alt-preset extreme encoding. Steve's voice is "distorded" on my version too. Perhaps the track is not a good choice for an encoder test.

I would like see a strong female voice included. A jazz or opera singer who can sing load and high would be good. Someone who can break glasses with her voice... this kind of sample might produce low bitrates and be difficult for the encoders at the same time.


--------------------
http://listening-tests.freetzi.com
Go to the top of the page
+Quote Post
ErikS
post Dec 1 2005, 00:30
Post #744





Group: Members
Posts: 757
Joined: 8-October 01
Member No.: 247



As ff123, user and I among others suggested earlier, it would be good to try to achieve a bitrate distribution of the sample set which resembles the distribution from a large random set. So perhaps add a few samples which lower the average a bit...?
Go to the top of the page
+Quote Post
Alex B
post Dec 1 2005, 01:10
Post #745





Group: Members
Posts: 1303
Joined: 14-September 05
From: Helsinki, Finland
Member No.: 24472



As said before, the used encoding options should be determined by checking the bitrates of a large amount of varied complete tracks and after that kept.

The samples should represent many musical genres and be difficult enough to produce audible artifacts. This is a listening test and no one should be interested what bitrates the individual encoded samples are. I don't think the bitrates should even be checked. The encoders should be left on their own to do the best they can with the sample material.

Naturally, the results of the preceding large-scale bitrate testing should be published.


--------------------
http://listening-tests.freetzi.com
Go to the top of the page
+Quote Post
Alex B
post Dec 1 2005, 01:17
Post #746





Group: Members
Posts: 1303
Joined: 14-September 05
From: Helsinki, Finland
Member No.: 24472



I updated my bitrate table with Vorbis -q 4.20 and gathered all previous results in the same table:

bitrates_public2.xls


--------------------
http://listening-tests.freetzi.com
Go to the top of the page
+Quote Post
ChiGung
post Dec 1 2005, 02:05
Post #747





Group: Members
Posts: 439
Joined: 9-February 05
From: county down
Member No.: 19713



QUOTE (kwanbis @ Nov 30 2005, 08:18 PM)
QUOTE (Triza @ Nov 30 2005, 06:27 PM)
Sadly we cannot do 2-pass because that would required to be executed on full tracks and Sebastian rightly do not want to get embroiled on copyright issues.
Sorry if already asked, but i don't think anybody would get into trouble for doing this, i mean, he won't keep the whole song, right? He would just encode the file, decode, cut, and delete...

I think it was resolved that the differences concerned are similar in a kind and in severity to Neros ABR encoding choice. iirc all devs who expressed an opinion see little problem with 2passing a joined sample corpus for Wma stds encode if its wished to include wma std in the test. ff123 agreed this would be fine if the samples where of average complexity, id suggest if they're mean achieved bitrate of the 'normal' vbrs doesnt match the target bitrate the 2pass could target the mean achieved bitrates of the other encoders to fairly compensate.
All in all, its not some to worry about at this stage, whatever the samples used a suitible Wma std targeting process is possible if required. smile.gif


--------------------
no conscience > no custom
Go to the top of the page
+Quote Post
ErikS
post Dec 1 2005, 02:26
Post #748





Group: Members
Posts: 757
Joined: 8-October 01
Member No.: 247



QUOTE (Alex B @ Dec 1 2005, 02:10 AM)
and be difficult enough to produce audible artifacts.
*

I'm repeating myself, but I still wonder how you decide that a track is difficult...?
Go to the top of the page
+Quote Post
Alex B
post Dec 1 2005, 02:41
Post #749





Group: Members
Posts: 1303
Joined: 14-September 05
From: Helsinki, Finland
Member No.: 24472



QUOTE (ErikS @ Dec 1 2005, 03:26 AM)
QUOTE (Alex B @ Dec 1 2005, 02:10 AM)
and be difficult enough to produce audible artifacts.
*

I'm repeating myself, but I still wonder how you decide that a track is difficult...?
*

I guess I'm the wrong person to answer to this question. Check my "failure" here:

A little VBR ~88 kbps ABX-test


--------------------
http://listening-tests.freetzi.com
Go to the top of the page
+Quote Post
NoXFeR
post Dec 1 2005, 05:54
Post #750





Group: Members
Posts: 84
Joined: 3-August 03
From: Trondheim, NO
Member No.: 8142



Vorbis should not be tested with -q 4.00, but in around 4.25. See the bitrate estimate thread for reasons why.
Go to the top of the page
+Quote Post

35 Pages V  « < 28 29 30 31 32 > » 
Closed TopicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 18th December 2014 - 09:00