IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
Evaluating the Sound Quality of Ipod Music Stations:Part 1, Listening Tests on Ipod Music Stations
solive
post Feb 6 2010, 03:01
Post #1





Group: Members
Posts: 162
Joined: 21-February 04
From: Los Angeles
Member No.: 12173



Here is a new article I posted about issues in designing listening tests on Ipod Music Stations with an illustrative video attached.

In parts 2 and 3, I intend to show some subjective and objective results of some recent tests we performed on this category of products.

Cheers
Sean Olive
Audio Musings

This post has been edited by solive: Feb 6 2010, 03:46


--------------------
Sean Olive
[url="http://seanolive.com"]Audio Musings[/url]
Go to the top of the page
+Quote Post
andy o
post Feb 6 2010, 05:24
Post #2





Group: Members
Posts: 1321
Joined: 14-April 09
Member No.: 68950



thanks should be interesting.
Go to the top of the page
+Quote Post
rpp3po
post Feb 7 2010, 12:27
Post #3





Group: Developer
Posts: 1126
Joined: 11-February 03
From: Germany
Member No.: 4961



Thank you! Parts 2 & 3 will be much more interesting for me. I think quite a few here know a little bit about the issues talked about in theory, but lack the resources to conduct such thorough evaluations themselves.
Go to the top of the page
+Quote Post
odigg
post Feb 7 2010, 16:25
Post #4





Group: Members
Posts: 629
Joined: 25-July 08
From: USA
Member No.: 56264



Sean, not to be a PITA but please don't feed the idea that the IPOD is a poor quality device as you *may* have done in your comments. There are people (me included) who have done blind tests with PMPs versus "better" systems and concluded that the PMPs are not the weak spot. They are basically audibly transparent, as they should be.

I am very curious about your results and am, as always, very happy that you are doing such experiments.

This post has been edited by odigg: Feb 7 2010, 16:26
Go to the top of the page
+Quote Post
solive
post Feb 7 2010, 20:21
Post #5





Group: Members
Posts: 162
Joined: 21-February 04
From: Los Angeles
Member No.: 12173



QUOTE (odigg @ Feb 7 2010, 07:25) *
Sean, not to be a PITA but please don't feed the idea that the IPOD is a poor quality device as you *may* have done in your comments. There are people (me included) who have done blind tests with PMPs versus "better" systems and concluded that the PMPs are not the weak spot. They are basically audibly transparent, as they should be.

I am very curious about your results and am, as always, very happy that you are doing such experiments.


I haven't done any listening tests on Ipod or other PMP devices themselves, but I would tend to agree with you that they are not the weak spot in these tests. I didn't mean to imply they are.

I often hear people (even in our own company) disparage Ipod or "MP3" without understanding that it's the quality of the CODEC/bit-rate that determines how good they sound, and at the higher bit-rates they can sound 100% transparent.


--------------------
Sean Olive
[url="http://seanolive.com"]Audio Musings[/url]
Go to the top of the page
+Quote Post
greynol
post Feb 7 2010, 20:27
Post #6





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



Unfortunately there are a lot of people who blindly believe that things like this are necessary:
http://blog.stereophile.com/ces2008/010708wadia/


--------------------
I should publish a list of forum idiots.
Go to the top of the page
+Quote Post
andy o
post Feb 7 2010, 21:07
Post #7





Group: Members
Posts: 1321
Joined: 14-April 09
Member No.: 68950



And talk about nonsensical:

QUOTE
Wadia's $349 iTransport can take the digital signal out of an iPod before the DAC, outputting 16-bit/44.1khz resolution for uncompressed files—it doesn't upconvert lower-rez files like MP3s, but it does reformat them to 16/44.1, according to Wadia's John Schaffer.


Usually when audiophiles talk nonsense they try to mask it with "quantum wording", but here the guy is not even making an effort!
Go to the top of the page
+Quote Post
rpp3po
post Feb 7 2010, 22:13
Post #8





Group: Developer
Posts: 1126
Joined: 11-February 03
From: Germany
Member No.: 4961



At least they are not lying, but just use some queer wording. The iTransport actually bypasses the iPod's DAC and accesses the iPod's files in USB host mode. It copies them to its internal memory where they are decoded ("reformatted" to 16/44.1) and fed into the iTransport's digital output, which has its own clock source.

This post has been edited by rpp3po: Feb 7 2010, 23:20
Go to the top of the page
+Quote Post
Arnold B. Kruege...
post Mar 5 2010, 15:34
Post #9





Group: Members
Posts: 3792
Joined: 29-October 08
From: USA, 48236
Member No.: 61311



QUOTE (rpp3po @ Feb 7 2010, 16:13) *
At least they are not lying, but just use some queer wording. The iTransport actually bypasses the iPod's DAC and accesses the iPod's files in USB host mode. It copies them to its internal memory where they are decoded ("reformatted" to 16/44.1) and fed into the iTransport's digital output, which has its own clock source.


Interesting. So the Wadia is essentially working like a PC that is playing music from an iPod that is being used as a storage device.

I took a peek at the Stereophile review and see that there are actually no technical tests other than for bit-perfect reproduction. They seem to be taking the view that bit-perfect copies are sufficient for sonic accuracy. There is an opporutity to involve the jitter myth, but they seem to have sloughed it. Should I cheer? ;-)
Go to the top of the page
+Quote Post
solive
post Apr 11 2010, 01:14
Post #10





Group: Members
Posts: 162
Joined: 21-February 04
From: Los Angeles
Member No.: 12173



QUOTE (solive @ Feb 5 2010, 19:01) *
Here is a new article I posted about issues in designing listening tests on Ipod Music Stations with an illustrative video attached.

In parts 2 and 3, I intend to show some subjective and objective results of some recent tests we performed on this category of products.

Cheers
Sean Olive
Audio Musings


I finally posted Part 2 of this article which summarizes some recent competitive benchmarking listening tests on three popular Ipod Music Stations.

Cheers
Sean Olive
Audio Musings



--------------------
Sean Olive
[url="http://seanolive.com"]Audio Musings[/url]
Go to the top of the page
+Quote Post
solive
post May 2 2010, 03:38
Post #11





Group: Members
Posts: 162
Joined: 21-February 04
From: Los Angeles
Member No.: 12173



QUOTE (solive @ Apr 10 2010, 17:14) *
QUOTE (solive @ Feb 5 2010, 19:01) *
Here is a new article I posted about issues in designing listening tests on Ipod Music Stations with an illustrative video attached.

In parts 2 and 3, I intend to show some subjective and objective results of some recent tests we performed on this category of products.

Cheers
Sean Olive
Audio Musings


I finally posted Part 2 of this article which summarizes some recent competitive benchmarking listening tests on three popular Ipod Music Stations.

Cheers
Sean Olive
Audio Musings


Part 3 - Objective Measurements - was posted today. In Part 3 I present the anechoic and in-room measurements to see which ones best explain listeners' sound quality ratings of the Music Stations.

Cheers
Sean Olive
Audio Musings


--------------------
Sean Olive
[url="http://seanolive.com"]Audio Musings[/url]
Go to the top of the page
+Quote Post
udauda
post May 5 2010, 05:27
Post #12





Group: Members
Posts: 101
Joined: 19-August 05
From: CA, USA
Member No.: 24009



Thanks for another great article!

It's quite interesting how the listeners found Device A a little too bright even with the high freq room attenuation. (I believe the system at Harman reference listening room has a high freq attenuation about -3dB @ 10kHz & -6dB @ 20kHz?)
Go to the top of the page
+Quote Post
2Bdecided
post May 5 2010, 09:43
Post #13


ReplayGain developer


Group: Developer
Posts: 5134
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



Thank you Sean. That last part is especially interesting - maybe our ears+brain "pick-apart" the sound field in a real listening room, so that (at least at higher frequencies) we are able to largely ignore the contributions from the room, and hear the quality of the direct sound.

I'm not sure everyone puts iPod docks up against the wall. One of the biggest problems with speakers for "normal" people is how much the sound changes depending on location. A simple "up a wall / not up a wall" switch on the speaker would be good.

Finally, I would debate how well you can "level match" such different sources. Maybe trained listeners can hear past this, but it's a strong source of bias in a direct switching situation IMO. Another possible version of the test would be to give the users + and - buttons to change the volume of each device separately (no scale, so no bias). This may be far more confusing for test subjects, and may make the task harder, but is far more representative of normal listening.

A separate test (which you did objectively, not subjectively) is how loud they go, and how soon they sound "nasty" when played loud. This is very important for some users.

(I'm sure you know all this - I know you've started with a test that hits the most fundamental issues first - I find it amazing that in 2010, such a test is so rare and almost grounding breaking - we should have been doing things at least this well for the last couple of decades - how did the industry go so wrong?)

Cheers,
David.
Go to the top of the page
+Quote Post
solive
post May 6 2010, 00:11
Post #14





Group: Members
Posts: 162
Joined: 21-February 04
From: Los Angeles
Member No.: 12173



QUOTE (udauda @ May 4 2010, 21:27) *
Thanks for another great article!

It's quite interesting how the listeners found Device A a little too bright even with the high freq room attenuation. (I believe the system at Harman reference listening room has a high freq attenuation about -3dB @ 10kHz & -6dB @ 20kHz?)

Thanks! I think the brightness of Music Station A has more to do with the slight broadband upward spectral tilt in the direct sound rather than the absorption characteristics of the listening room.


Cheers
Sean
Audio Musings

This post has been edited by solive: May 6 2010, 00:12


--------------------
Sean Olive
[url="http://seanolive.com"]Audio Musings[/url]
Go to the top of the page
+Quote Post
solive
post May 6 2010, 00:23
Post #15





Group: Members
Posts: 162
Joined: 21-February 04
From: Los Angeles
Member No.: 12173



QUOTE (2Bdecided @ May 5 2010, 01:43) *
Thank you Sean. That last part is especially interesting - maybe our ears+brain "pick-apart" the sound field in a real listening room, so that (at least at higher frequencies) we are able to largely ignore the contributions from the room, and hear the quality of the direct sound.

I'm not sure everyone puts iPod docks up against the wall. One of the biggest problems with speakers for "normal" people is how much the sound changes depending on location. A simple "up a wall / not up a wall" switch on the speaker would be good.

Finally, I would debate how well you can "level match" such different sources. Maybe trained listeners can hear past this, but it's a strong source of bias in a direct switching situation IMO. Another possible version of the test would be to give the users + and - buttons to change the volume of each device separately (no scale, so no bias). This may be far more confusing for test subjects, and may make the task harder, but is far more representative of normal listening.

A separate test (which you did objectively, not subjectively) is how loud they go, and how soon they sound "nasty" when played loud. This is very important for some users.

(I'm sure you know all this - I know you've started with a test that hits the most fundamental issues first - I find it amazing that in 2010, such a test is so rare and almost grounding breaking - we should have been doing things at least this well for the last couple of decades - how did the industry go so wrong?)

Cheers,
David.


Thanks David. I agree with you that these products should have a 2 or 3-position switch on them that compensates for placement next to a boundary or 4 pi setting. There should also be an "Apple Store" setting to help sell the product smile.gif

Loudness matching among different loudspeaker products is always a compromise. If the frequency responses are similar you can get very close with very little program-dependency. Otherwise, the loudness will slightly change depending on the differences in frequency responses of the products under test and the spectrum of the program.

We use uncorrelated pink noise and measure at the listening position and adjust until we get the same SPL (B-weighted, slow). More recently we've been using a CRC loudness meter-- which seems to give good results. One thing I have considered but not implemented is a separate loudness matching for each program... I don't think it would matter much for our standard programs because they all have reasonably broadband spectra.

We do allow listeners to adjust the volume for automotive audio system evaluation to judge differences in dynamics, distortion,etc. We are working on a similar test for products like this.

Cheers
Sean
Audio Musings


--------------------
Sean Olive
[url="http://seanolive.com"]Audio Musings[/url]
Go to the top of the page
+Quote Post
Arnold B. Kruege...
post Jun 2 2010, 13:51
Post #16





Group: Members
Posts: 3792
Joined: 29-October 08
From: USA, 48236
Member No.: 61311



QUOTE (solive @ May 5 2010, 19:23) *
We use uncorrelated pink noise and measure at the listening position and adjust until we get the same SPL (B-weighted, slow). More recently we've been using a CRC loudness meter-- which seems to give good results. One thing I have considered but not implemented is a separate loudness matching for each program... I don't think it would matter much for our standard programs because they all have reasonably broadband spectra.


I wonder where one obtains a CRC loudness meter download?

As close as I've been able to find is this one:

Orban Loudness meter with ITU 1770 metering feature download
Go to the top of the page
+Quote Post
solive
post Jun 6 2010, 18:59
Post #17





Group: Members
Posts: 162
Joined: 21-February 04
From: Los Angeles
Member No.: 12173



QUOTE (Arnold B. Krueger @ Jun 2 2010, 05:51) *
QUOTE (solive @ May 5 2010, 19:23) *
We use uncorrelated pink noise and measure at the listening position and adjust until we get the same SPL (B-weighted, slow). More recently we've been using a CRC loudness meter-- which seems to give good results. One thing I have considered but not implemented is a separate loudness matching for each program... I don't think it would matter much for our standard programs because they all have reasonably broadband spectra.


I wonder where one obtains a CRC loudness meter download?

As close as I've been able to find is this one:

Orban Loudness meter with ITU 1770 metering feature download


I contacted CRC directly, since I know the people who developed it, and they gave me a copy, as a favor. I think they might have plans selling it, along with a multichannel version.

Cheers
Sean
Audio Musings


--------------------
Sean Olive
[url="http://seanolive.com"]Audio Musings[/url]
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 15th September 2014 - 11:41