Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: quality checking tools (Read 6601 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

quality checking tools

can u guide me in using EAQUAL
i have only raw pcm files
i don;t have proper tool for converting raw pcm to .wav
and i think the error 3000007 is for wrong file names
error 3000001 is for somthing else

also any body have any idea about what should be
ODG values if one is implementing fixed point version of any audio coder
what degree of compramise taht one can give for encoder and for decoder and together(in respect to ODG values)?
thanks alot

quality checking tools

Reply #1 –
I just finished up working with EAQUAL for four hours, so I know that when I get error #3000001, it means "file not found" (I didn't input the filename of either the test or ref file correctly).

As for PCM audio data, I don't know.

But EAQUAL gave me some strange results for my tests (testing the quality of MPC files with various presets and modifications, as well as transcoding to MP3). I will post my findings in a different thread, possibly tomorrow.

quality checking tools

Reply #2 –
Quote
Originally posted by SometimesWarrior
But EAQUAL gave me some strange results for my tests (testing the quality of MPC files with various presets and modifications, as well as transcoding to MP3). I will post my findings in a different thread, possibly tomorrow.

EAQUAL is unreliable and seems more of a gee-whiz tool than anything serious. It does have some validity - it will rank an Lame API file over a Xing 128kbps CBR file - but you can't make too many solid conclusions with it.

quality checking tools

Reply #3 –
EAQual is reliable for what it is designed (ITU-R BS.1387 that is), and it is not designed to evaluate various codecs to say which is better.  PEAQ is designed to be integrated in QoS environment where only trends are to be shown, not absolute quality.

By the way, commercial ITU-R BS.1387 software price has 5 digits  EAQUAL is really a good thing, since it is GPL - otherwise one would have to pay sh*t load of $$$ for commercial versions.

quality checking tools

Reply #4 –
Quote
Originally posted by Ivan Dimkovic
EAQual is reliable for what it is designed (ITU-R BS.1387 that is), and it is not designed to evaluate various codecs to say which is better.  PEAQ is designed to be integrated in QoS environment where only trends are to be shown, not absolute quality.

By the way, commercial ITU-R BS.1387 software price has 5 digits  EAQUAL is really a good thing, since it is GPL - otherwise one would have to pay sh*t load of $$$ for commercial versions.



thanks for u'r responce

the EAQUAL.exe  is not working for me
i am trying with raw pcm audio files
can u give me detailed usage and limitations for the usage of EAQUAL.exe 

personally u can contact me at
[a href='mailto:ashok_i_m@rediffmail.com'][/a]
thanks alot

quality checking tools

Reply #5 –
Quote
Originally posted by Ivan Dimkovic
EAQual is reliable for what it is designed (ITU-R BS.1387 that is), and it is not designed to evaluate various codecs to say which is better.  PEAQ is designed to be integrated in QoS environment where only trends are to be shown, not absolute quality.

By the way, commercial ITU-R BS.1387 software price has 5 digits  EAQUAL is really a good thing, since it is GPL - otherwise one would have to pay sh*t load of $$$ for commercial versions.



hello
i had tried with 8 khz wav file on Windows 2000
it is giving the run time error


The instruction at 0x00404e1c referenced memory at 0x3a5fbcb6. The memory could not be read


with cmd dispaly



C:AUDIOAACaudio_quality_tester>eaqual -srate 8000 -offset 0 -ch 1 -fref C:AU
DIOAACaudio_quality_testerspeech5.wav    -ftest C:AUDIOAACaudio_quality_te
sterspeech5_1.wav

EAQUAL - Evaluation of Audio Quality
Version:        0.1.1alpha
Author:        Alexander Lerch, zplane.development
_______________________________________________________
Reference File:        C:AUDIOAACaudio_quality_testerspeech5.wav
Test File:              C:AUDIOAACaudio_quality_testerspeech5_1.wav
Sample Rate:            8000
Number of Channels:    1

Press Escape to cancel...

Frame:          0
C:AUDIOAACaudio_quality_tester>
thanks alot

quality checking tools

Reply #6 –
Quote
Originally posted by Ivan Dimkovic
EAQual is reliable for what it is designed (ITU-R BS.1387 that is), and it is not designed to evaluate various codecs to say which is better.  PEAQ is designed to be integrated in QoS environment where only trends are to be shown, not absolute quality.

By the way, commercial ITU-R BS.1387 software price has 5 digits  EAQUAL is really a good thing, since it is GPL - otherwise one would have to pay sh*t load of $$$ for commercial versions.



hello sir
where can i get the free source code for ITU-R BS.1387
the link that has mentioned

http://sourceforge.net/projects/eaqual/

is giving a message as

                  Invalid Project
if i want to purchase commercial ITU-R BS.1387 software (EAQUAL)
how can i proceed
thanks alot

quality checking tools

Reply #7 –
ashok: http://peaqb.sourceforge.net

But i don't sure that my software work fine with wave file 8000Hz.

quality checking tools

Reply #8 –
Quote
By the way, commercial ITU-R BS.1387 software price has 5 digits :) EAQUAL is really a good thing, since it is GPL - otherwise one would have to pay sh*t load of $$$ for commercial versions.


To be more specific about the price:

PEAQ/BS.1387 Basic and Advanced Model, for file based measurements only: total price EUR 7090.

So, its not 5 but oooonly 4 digits!

Daniel

quality checking tools

Reply #9 –
Quote
if i want to purchase commercial ITU-R BS.1387 software (EAQUAL)
how can i proceed

Two commercial products available from http://www.peaq.org/prod03.htm
I have been using OPERA. Yes, as Ivan mentioned, they are indeed very expensive (> US$10k per copy), but I found that it is quite useful in my codec development work. I always wished that I could have the source codes so that I can modify it to fit my need. Now, it seems that my dream come true.

Thank you, oveRet... 

Quote
ashok: http://peaqb.sourceforge.net

But i don't sure that my software work fine with wave file 8000Hz.


Do you mean that your software works fine with samling frequencies greater than 8kHz? OPERA only works for 44.1kHz and 48kHz. Yours seems to be even better... Could you pls provide some instruction of how to compile your software under Mac OSX and Win2k using MSVC++?

Thank you again for your wonderful work.

quality checking tools

Reply #10 –
Quote
Could you pls provide some instruction of how to compile your software under Mac OSX and Win2k using MSVC++?

It should be easily compilable under OSX using GCC, unless it uses x86-specific instructions.

For the time being, it seems it can only be compiled for win32 using MingW or Cygwin.

quality checking tools

Reply #11 –
Quote
EAQUAL is unreliable and seems more of a gee-whiz tool than anything serious. It does have some validity - it will rank an Lame API file over a Xing 128kbps CBR file - but you can't make too many solid conclusions with it.


Is these kind of software evaluation tools good enough?? I think in the field of professional audio-visual, people do not rely too much on software simulations tools to do design work... I know of some major Japanese Audio-Visual companies that would not even adopt Electronic Circuit Designs software to design the circuitry of their hi-fi or TVs systems...

It would be outrageous that one could "standardized" quality in this field... To me it seemed like walking into an art gallery and set a standard that paintings like the Mona Lisa is high quality and others are rubbish!!

quality checking tools

Reply #12 –
Quote
It would be outrageous that one could "standardized" quality in this field... To me it seemed like walking into an art gallery and set a standard that paintings like the Mona Lisa is high quality and others are rubbish!!


No, well - it is not working like that

The PEAQ model was designed with this philosophy in mind:

- Design as much perceptual analysis tools as possible (FFT based ear model, filterbank based ear model, etc..)

- Take all relevant distortion measures (total NMR, average NMR, RMS noise loudness, average linear distortion, modulation difference, harmonic distortion, perceived vs. original bandwidth, etc...)

- Adjust parametes of analysis tools to match worst-case studies known in psychoacoustics (i.e. - set the spreading function low slope to -27 dB/bark unit, etc..)

- Create a neural network based cognitive model which takes all distortion measures, weights them and outputs the final result on ITU scale.

- Train the neural network with available professional listening test data (done by ITU, MPEG, ...)



Ok,  so with that you have a model that acts like some average listener in MPEG and ITU listening test - with average threshold of hearing (approximated by Painter and Spanias formula),  etc...    If the model says that distortion is -0.5 (i.e. in "excellent" range)  there is a small possibility that the clip sounds awful 

I found that PEAQ has following drawbacks:

- Absolute threshold of hearing is averaged (well - I put similar to my AAC encoder and I got Guruboolez seriously complaining ;-)    but it is well known that ATH difference, especially in young listenters can be very big,  and in wrong (low) direction for the model

- Spreading function is SPL dependent, which is OK if the SPL is properly adjusted in listening environment which corresponds to ODG simulation - which is not the case in most home setups, headphone casual listenings and so on... also the volume playback level is not known a-priori, which makes this spreading function assumption very tricky,  and this can make differences between ODG findings and SDG reports for some users.

- FFT based ear model used in basic PEAQ (CRC SEAQ, EAQUAL, PEAQb) has a very big time resolution (21.3 ms for 48 kHz,  23.2 ms for 44.1 khz).  This time frame size is much bigger than human pre-masking ability, and some short time-domain artifacts (like pre-echo on very impulse tracks, or TNS bugs in AAC encoders) can't be properly detected.  I remember that Andree Buschmann told me his experience with gettng "excellent range" ODG with mp3Enc fatboy.mp3 encoded at 128 kbps    -- filterbank based model in advanced PEAQ should solve this problem.

- Non lineaar addition of frequency spreading is actually used to replace tonality estimation, by the findings of Lutfi.  Fixed "non linear" constant of 0.4 is used, but I am not really sure if that is constant for all critical bands, and does it really completely estimate complex masking (probably not) - or, most important,  does it over-estimate the masking threshold sometimes...  I've got couple other papers regarding estimation of masking power in complex signals (from Baumgarte,  Johnson, etc...) ... I never checked how they correlate with each other, but everyone claims to be better than plain tonality estimation (also possible in couple ways... at least 5-6 I know of)

- If I am not mistaken, BS.1387 does not take into account BMLD phenomenon in joint-coding configurations

- Bark->Hertz formula is using Schroeder Et. Al. estimation,  but I found some Ph.D. theses on Internet claiming that Traunmueller's formula is approximating this mapping better than others.

- Neural network weights various distortion measures (NMR, bandwidth, harmonic structure) - the problem here is that many of us might have different "weighting"  functions in our brains  But there is proably nothing that can be done here,  this is probably the best possible way.

quality checking tools

Reply #13 –
Quote
Ivan said:
I found that PEAQ has following drawbacks:


I think Ivan give a very good summary on the PEAQ tools. Now I propose another drawback on the ODG score:

The ODG is neural network weighted by 8~9 parameters. Bandwidth for reference and test signal are two parameters and you will find the bandwidth detection mechanism in PEAQ for not correct for songs especially for songs that contains high frequency compoents.

For me, I refer to survey all the 8~9 parameters (even SNR/ Segment SNR are also good index) among different codec or different version of the same codec. I think the whole parameters tell more story than just pick single ODG index. But, common people like single index rather then complex index even it is not precise enough to show everything.

quality checking tools

Reply #14 –
Quote
- If I am not mistaken, BS.1387 does not take into account BMLD phenomenon in joint-coding configurations


No, BMLD phenomenon must be taken into account in all cases when the signal is stereo.. not just when joint-coding is used... Stereofonics is a very complex field.. There are still a lot of on going research in this field...

quality checking tools

Reply #15 –
Quote
- If I am not mistaken, BS.1387 does not take into account BMLD phenomenon in joint-coding configurations

This is unfortunately very true. PEAQ processes each channel separately, it doesn't take into account of any stereo effect. In other words, PEAQ won't "hear" any stereo imaging artifacts as we can hear.

Thank Ivan for the nice overview. I fully agree with them. On the other hand, I found PEAQ is a useful analysis and screening tool for locating potential problems among a large selection of test samples, in which a real subjective test is just impossible to take place. For instance, after you made some changes to the codec, PEAQ can be used to see whether or not the change has introduced any potential regression among a broader range of music samples.

PEAQ is a tool, just like computer, it cannot completely replace human function, but can save us a lot of work if it's used properly.