IPB

Welcome Guest ( Log In | Register )

Overcoming the Perception Problem
item
post Oct 8 2012, 19:21
Post #1





Group: Members
Posts: 33
Joined: 7-August 12
Member No.: 102085



I've noticed an increase in forum debate about the validity of transferring the credibility of ABX from the physical domain to perception testing. I'm wondering if anyone has found a way past this issue?

The purpose of blind testing is to subtract subjectivity from the effect of - for instance - a drug trial: to assess a medication's impact on a subject's physiology with interference from their psychology. But what about when the purpose of a test is subjective perception? How do we then subtract the effect of the method to arrive at a meaningful outcome?

While we would like to remove expectation bias from the equation, if the conditions under which this is done also change the perceptive state of the listener, the test is invalidated as surely as they would be by tissue sample contamination.

Recent large scale public experiments by Lotto Labs (http://www.lottolab.org/) demonstrated that perception acuity is dramatically altered by test conditions: for instance, that time contraction/dilation effects are experienced when exposed to colour fields. In one experiment, two groups were asked to perform an identical fine-grained visual acuity test. One group was pre-emptively 'manipulated' by filling in a questionnaire designed to lower their self-esteem. This 'less confident' group consistently performed worse on the test that the unmanipulated one: their acuity was significantly impaired by a subtle psychological 'tweak' that wasn't even in effect during the test.

It seems undeniable that the much grosser differences between the mental states of sighted and 'blind' listening - considered generously - cast serious doubt on the results thus obtained.

The harder line is that blind perception tests are a fundamental misappropriation of methodology. In psychology it's axiomatic that for many experiments the subject must be unaware of the nature of the test (see Milgram). If a normalised state is not cunningly contrived, results are at best only indicative of what a subject thinks they should do; at worst, entirely invalid.

Probing hearing, the point is that a test must not change the mental state of the listener.

The contrast between outcomes of sighted and listening tests is as stark as those demonstrating suggestibility (see McGurk), but giving too much credence to such an intrinsically unsound experimental approach (not spotting this difficulty) does no favours to our credibility at all.

The only way past the dilemma seems to be direct mechanical examination of the mind during 'normal' listening to explore why the experiences of sighted and unsighted listening differ. This seems to be an interesting question.

In the meantime, the idea that - despite the method problem - results from blind ABX are valid is at least supported by the majority of data derived from home testing, Audio DiffMaker et al, so we needn't get hung up on it.
Go to the top of the page
+Quote Post
 
Start new topic
Replies
aethelberht
post Oct 9 2012, 13:17
Post #2





Group: Members
Posts: 13
Joined: 29-October 11
Member No.: 94780



What I can perhaps maybe possibly gather from your posts is that blind perception experiments are crude. What I don't gather is how the negative results have any impact on the positive results.

Ideally you would stop rambling and be more concise, but at the very least, you should explain explicitly what it is that is faulty. You begin by questioning "the credibility of ABX from the physical domain to perception testing." If it is this broad challenge, then consider the fact that Signal Detection Theory and discrimination experiments, ABX being one of them, have widespread use in the speech perception literature. Are you suggesting that subjects performing well in spite of the lack of cues leads to flawed conclusions?

But the issue closer at hand seems to be the much narrower one, of the retaliatory use of negative ABX results as evidence that things sound the same. And yet you keep responding to comments about the scientific (in)validity of this with comments such as "The problem is that negative results equally indict the efficacy of the method" without explaining how this is so.
Go to the top of the page
+Quote Post
item
post Oct 9 2012, 14:01
Post #3





Group: Members
Posts: 33
Joined: 7-August 12
Member No.: 102085



QUOTE (aethelberht @ Oct 9 2012, 13:17) *
What I can perhaps maybe possibly gather from your posts is that blind perception experiments are crude. What I don't gather is how the negative results have any impact on the positive results.

Ideally you would stop rambling and be more concise, but at the very least, you should explain explicitly what it is that is faulty. You begin by questioning "the credibility of ABX from the physical domain to perception testing." If it is this broad challenge, then consider the fact that Signal Detection Theory and discrimination experiments, ABX being one of them, have widespread use in the speech perception literature. Are you suggesting that subjects performing well in spite of the lack of cues leads to flawed conclusions?

But the issue closer at hand seems to be the much narrower one, of the retaliatory use of negative ABX results as evidence that things sound the same. And yet you keep responding to comments about the scientific (in)validity of this with comments such as "The problem is that negative results equally indict the efficacy of the method" without explaining how this is so.


Abstract:
A positive DBT result establishes reliably that two outcomes or entities differ.
A negative means - equally - either a) the two objects are identical, or b) that the method doesn't permit resolution of their differences.
Inherent in DBT is that 'blindness' de-normalises test conditions: the mechanism by which expectation bias is removed represses acuity by removing cues from which perceptive frameworks are built.
DBT is designed to separate subjective and objective responses and provide hard-to-falsify positive outcomes. Its suitability for physiological testing is not transferable to psychological testing but negative outcomes are aggressively touted as meaningful.
Go to the top of the page
+Quote Post
hlloyge
post Oct 10 2012, 21:04
Post #4





Group: Members
Posts: 701
Joined: 10-January 06
From: Zagreb
Member No.: 27018



QUOTE (item @ Oct 9 2012, 15:01) *
Abstract:
A positive DBT result establishes reliably that two outcomes or entities differ.
A negative means - equally - either a) the two objects are identical, or b) that the method doesn't permit resolution of their differences.


No, you are wrong at the second part. You don't set up DBT of lossy audio encoders on supertweeters, with FR form 35 kHz to 80 kHz. That method is not going to give anything useful. It's idiotic. Do you really think that peope can't make valid DBT audio tests? It has been done in the past numerous times.

If you set up test correctly, as expected, there will be only "yes, I can definitely hear the difference between A and B", and "no, I can't hear shit". It doesn't matter if the difference in reality is so subtle you can't hear it (lossless vs high bitrate lossy), because the only thing you are testing is if you can hear that difference, and the results are YES and NO. There is no MAYBE UNDER CERTAIN CIRCUMSTANCES, because you then recreate those circumstances and repeat the test. As many times you want, as long as you need.

The trouble is when you realize that you have all that 24/96 music, with fancy expensive amp and loudspeakers hand made from Siberian wood which grew in Tunguska at the crater (not to forget hand-made speaker drivers), and you can't hear the difference from 128 kbit aac file and that high bitrate lossless file. Then the imagination kicks in.

And you, sir, are the product of that mentality, which is contagious.
Go to the top of the page
+Quote Post
Porcus
post Oct 11 2012, 14:20
Post #5





Group: Members
Posts: 1995
Joined: 30-November 06
Member No.: 38207



QUOTE (hlloyge @ Oct 10 2012, 22:04) *
If you set up test correctly, as expected, there will be only "yes, I can definitely hear the difference between A and B", and "no, I can't hear shit". It doesn't matter if the difference in reality is so subtle you can't hear it (lossless vs high bitrate lossy), because the only thing you are testing is if you can hear that difference, and the results are YES and NO.


Ehem ... this is not how statistical tests work.

And even if it were, your description would only be valid if this “you” is what is supposed to be tested, and arguably not even then.


--------------------
One day in the Year of the Fox came a time remembered well
Go to the top of the page
+Quote Post
hlloyge
post Oct 12 2012, 14:26
Post #6





Group: Members
Posts: 701
Joined: 10-January 06
From: Zagreb
Member No.: 27018



QUOTE (Porcus @ Oct 11 2012, 15:20) *
Ehem ... this is not how statistical tests work.
And even if it were, your description would only be valid if this “you” is what is supposed to be tested, and arguably not even then.


When doing personal test of codec or parameters, I am testing them for my usage, for myself. And then it's either "i can hear the difference" or " i can't", I don't bother if someone else can hear it. And I don't care, really, if these music files sound different for anyone else - I am using them, not trading them, not uploading them.
And I understand that these results can't be solely used as the general truth - "130 kbit tvbr aac file is enough for everyone", for example, but for my usage, these tests are more than statistically enough.

I am sorry if I am missing the point - but isn't the point of ABX test to see if YOU can hear the difference between two files? It is not for viewing the difference between files - simple filesize or binary comparation can be used for that.

This post has been edited by hlloyge: Oct 12 2012, 14:26
Go to the top of the page
+Quote Post
Porcus
post Oct 12 2012, 15:04
Post #7





Group: Members
Posts: 1995
Joined: 30-November 06
Member No.: 38207



QUOTE (hlloyge @ Oct 12 2012, 15:26) *
QUOTE (Porcus @ Oct 11 2012, 15:20) *
Ehem ... this is not how statistical tests work.
And even if it were, your description would only be valid if this “you” is what is supposed to be tested, and arguably not even then.


When doing personal test of codec or parameters, I am testing them for my usage, for myself. And then it's either "i can hear the difference" or " i can't"


A (one-sided) statistical test would have two outcomes: Either that the alternative hypothesis has passed and should be accepted, or do not accept the alternative hypothesis (then, keep the null). Up to the threshold of the test (confidence could of course always be an artifact of chance), the latter should be the conclusion if there is not positive evidence for the alternative (in this case, “you can hear the difference”).

That should occur if
(I) You have no better chances than a random draw
(II) Your chances are better than the random draw, but you don't have sufficient data to prove it.

Notice that before testing, you are always in one of those two situations. Even if it will turn out when the samples are played that you get a 100/100, then you are in case II until the fact is established.



QUOTE (hlloyge @ Oct 12 2012, 15:26) *
I am sorry if I am missing the point - but isn't the point of ABX test to see if YOU can hear the difference between two files?


Not necessarily. Suppose I want to establish a “sufficiently good” (for whatever purpose) end-user format. Then I am not satisfied with your score on your music, unless I am only targetting you as a customer. Even if your music does not have nasty enough artifacts for you to detect (or find annoying), it might be different with other ears and other signals. (Of course, you then need to use the appropriate method (test / design of experiment) to check whether the accuracy is better than random, but that is a practical obstacle.)

If 5 percent of the listeners hear differences on 10 percent of their music collection, is then the format “transparent”? I think not. It may be good enough for the purpose, by all means, but it does not mean that there are no audible differences.


--------------------
One day in the Year of the Fox came a time remembered well
Go to the top of the page
+Quote Post
hlloyge
post Oct 12 2012, 22:14
Post #8





Group: Members
Posts: 701
Joined: 10-January 06
From: Zagreb
Member No.: 27018



QUOTE (Porcus @ Oct 12 2012, 16:04) *
Not necessarily. Suppose I want to establish a “sufficiently good” (for whatever purpose) end-user format. Then I am not satisfied with your score on your music, unless I am only targetting you as a customer. Even if your music does not have nasty enough artifacts for you to detect (or find annoying), it might be different with other ears and other signals. (Of course, you then need to use the appropriate method (test / design of experiment) to check whether the accuracy is better than random, but that is a practical obstacle.)
If 5 percent of the listeners hear differences on 10 percent of their music collection, is then the format “transparent”? I think not. It may be good enough for the purpose, by all means, but it does not mean that there are no audible differences.


You misunderstood me.
I am conducting ABX test for MYSELF. I am not conducting ABX test to gain statistical knowledge if people can hear difference.
I understand you have to have some sort of statistical chance for error when doing multiple users test, but I am talking about single person making test for his (or her's) own advantage and knowledge.
That margin, if testing codecs for personal knowledge, is irrelevant, IMO.
Go to the top of the page
+Quote Post
greynol
post Oct 13 2012, 14:25
Post #9





Group: Super Moderator
Posts: 10339
Joined: 1-April 04
From: San Francisco
Member No.: 13167



QUOTE (hlloyge @ Oct 12 2012, 14:14) *
I am conducting ABX test for MYSELF.

Even still, a failed test only fails to demonstrate that an individual can distinguish a difference during that instance. Training and/or rest may affect the outcome of a future test, as examples.

This post has been edited by greynol: Oct 13 2012, 15:10


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
hlloyge
post Oct 14 2012, 13:35
Post #10





Group: Members
Posts: 701
Joined: 10-January 06
From: Zagreb
Member No.: 27018



QUOTE (greynol @ Oct 13 2012, 15:25) *
QUOTE (hlloyge @ Oct 12 2012, 14:14) *
I am conducting ABX test for MYSELF.

Even still, a failed test only fails to demonstrate that an individual can distinguish a difference during that instance. Training and/or rest may affect the outcome of a future test, as examples.


Yes, but I am conducting the test at that one point of time. And the results are valid for that test.
Of course you should take what, 16 full turns? But they don't have to be the same day. Or week.
Go to the top of the page
+Quote Post
Porcus
post Oct 14 2012, 23:31
Post #11





Group: Members
Posts: 1995
Joined: 30-November 06
Member No.: 38207



QUOTE (hlloyge @ Oct 14 2012, 14:35) *
Yes, but I am conducting the test at that one point of time.


Strange to read your initial postings in this thread now after you have tried to downplay the applicability of the test to a one-time personal experience with a clear-cut full sensitivity/specificity.


--------------------
One day in the Year of the Fox came a time remembered well
Go to the top of the page
+Quote Post

Posts in this topic
- item   Overcoming the Perception Problem   Oct 8 2012, 19:21
- - Soap   You appear to be confused. Despite common shortha...   Oct 8 2012, 19:38
- - item   To rephrase: what, in any trial, is blind testing ...   Oct 8 2012, 19:52
|- - Soap   QUOTE (item @ Oct 8 2012, 14:52) To rephr...   Oct 8 2012, 19:59
|- - item   QUOTE (Soap @ Oct 8 2012, 19:59) QUOTE (i...   Oct 8 2012, 23:32
|- - item   Part of Beau Lotto's 'Public Perception pr...   Oct 9 2012, 00:30
||- - Porcus   QUOTE (item @ Oct 9 2012, 01:30) Part of ...   Oct 9 2012, 10:53
||- - item   QUOTE (Porcus @ Oct 9 2012, 10:53) You ca...   Oct 9 2012, 12:23
||- - Porcus   QUOTE (item @ Oct 9 2012, 13:23) This for...   Oct 9 2012, 16:04
|- - krabapple   QUOTE (item @ Oct 8 2012, 18:32) Partly, ...   Oct 9 2012, 03:51
|- - item   QUOTE (item @ Oct 8 2012, 18:32) Although...   Oct 9 2012, 11:14
- - DVDdoug   I just don't see how making a good scientific ...   Oct 9 2012, 00:59
- - greynol   @item: Perhaps you could share with us a little ab...   Oct 9 2012, 04:54
|- - Porcus   QUOTE (greynol @ Oct 9 2012, 05:54) a fai...   Oct 9 2012, 09:57
|- - item   QUOTE (greynol @ Oct 9 2012, 04:54) @item...   Oct 9 2012, 11:42
|- - 2Bdecided   QUOTE (item @ Oct 9 2012, 11:42) Positive...   Oct 9 2012, 14:15
||- - item   QUOTE (2Bdecided @ Oct 9 2012, 14:15) QUO...   Oct 9 2012, 14:37
||- - 2Bdecided   QUOTE (item @ Oct 9 2012, 14:37) Deprived...   Oct 9 2012, 17:43
|||- - item   Sorry - been away, but lots of noise (most - fasci...   Oct 19 2012, 17:34
||- - dhromed   QUOTE (item @ Oct 9 2012, 15:37) 'Kno...   Oct 9 2012, 19:11
|- - greynol   QUOTE (item @ Oct 9 2012, 03:42) QUOTE (g...   Oct 9 2012, 22:07
|- - item   QUOTE (greynol @ Oct 9 2012, 22:07) @item...   Oct 19 2012, 18:33
- - hlloyge   The only people I know of to shun DBT method of te...   Oct 9 2012, 12:35
|- - item   QUOTE (hlloyge @ Oct 9 2012, 12:35) The o...   Oct 9 2012, 13:25
|- - hlloyge   QUOTE (item @ Oct 9 2012, 14:25) The sole...   Oct 9 2012, 13:54
||- - item   QUOTE (hlloyge @ Oct 9 2012, 13:54) The o...   Oct 9 2012, 14:09
|- - Canar   QUOTE (item @ Oct 9 2012, 05:25) I think ...   Oct 9 2012, 17:09
|- - krabapple   QUOTE (item @ Oct 9 2012, 08:25) The sole...   Oct 10 2012, 01:25
|- - item   QUOTE (krabapple @ Oct 10 2012, 01:25) QU...   Oct 19 2012, 18:45
|- - Porcus   QUOTE (item @ Oct 19 2012, 19:45) QUOTE (...   Oct 19 2012, 19:22
|- - 2Bdecided   QUOTE (item @ Oct 19 2012, 18:45) Certain...   Oct 22 2012, 12:53
- - aethelberht   What I can perhaps maybe possibly gather from your...   Oct 9 2012, 13:17
|- - item   QUOTE (aethelberht @ Oct 9 2012, 13:17) W...   Oct 9 2012, 14:01
|- - aethelberht   "A negative means - equally - either a) the t...   Oct 9 2012, 14:14
|- - Arnold B. Krueger   QUOTE (item @ Oct 9 2012, 09:01) DBT is d...   Oct 9 2012, 14:53
|- - hlloyge   QUOTE (item @ Oct 9 2012, 15:01) Abstract...   Oct 10 2012, 21:04
|- - Porcus   QUOTE (hlloyge @ Oct 10 2012, 22:04) If y...   Oct 11 2012, 14:20
|- - hlloyge   QUOTE (Porcus @ Oct 11 2012, 15:20) Ehem ...   Oct 12 2012, 14:26
|- - Porcus   QUOTE (hlloyge @ Oct 12 2012, 15:26) QUOT...   Oct 12 2012, 15:04
|- - hlloyge   QUOTE (Porcus @ Oct 12 2012, 16:04) Not n...   Oct 12 2012, 22:14
|- - greynol   QUOTE (hlloyge @ Oct 12 2012, 14:14) I am...   Oct 13 2012, 14:25
|- - hlloyge   QUOTE (greynol @ Oct 13 2012, 15:25) QUOT...   Oct 14 2012, 13:35
|- - Porcus   QUOTE (hlloyge @ Oct 14 2012, 14:35) Yes,...   Oct 14 2012, 23:31
|- - hlloyge   QUOTE (Porcus @ Oct 15 2012, 00:31) QUOTE...   Oct 16 2012, 09:54
- - Porcus   While I certainly agree that putting humans in a l...   Oct 10 2012, 13:09
- - 2Bdecided   It's like The Princess and the Pea. Audiophile...   Oct 11 2012, 11:58
- - skamp   If ABXing negatively alters one's ability to h...   Oct 11 2012, 14:28
|- - krabapple   QUOTE (skamp @ Oct 11 2012, 09:28) If ABX...   Oct 11 2012, 18:24
|- - mzil   QUOTE (krabapple @ Oct 11 2012, 13:24) QU...   Oct 15 2012, 17:14
|- - krabapple   You're right that different terms apply when w...   Oct 15 2012, 20:51
- - greynol   I believe our skeptic has flown the coop.   Oct 11 2012, 16:17
- - dhromed   Not everyone is as much a netizen as most of us. P...   Oct 11 2012, 17:05
- - googlebot   While the OP's reasoning and claimed inference...   Oct 11 2012, 21:44
|- - [JAZ]   QUOTE (googlebot @ Oct 11 2012, 22:44) Do...   Oct 11 2012, 22:37
||- - bandpass   QUOTE ([JAZ] @ Oct 11 2012, 22:37...   Oct 12 2012, 06:44
|- - sld   QUOTE (googlebot @ Oct 12 2012, 04:44) Lo...   Oct 12 2012, 03:38
|- - krabapple   QUOTE (googlebot @ Oct 11 2012, 16:44) Wh...   Oct 12 2012, 06:59
||- - googlebot   QUOTE (krabapple @ Oct 12 2012, 07:59) Ex...   Oct 12 2012, 21:44
||- - Nick.C   QUOTE (googlebot @ Oct 12 2012, 21:44) Th...   Oct 12 2012, 21:51
|||- - googlebot   QUOTE (Nick.C @ Oct 12 2012, 22:51) Is th...   Oct 12 2012, 21:58
|||- - Nick.C   QUOTE (googlebot @ Oct 12 2012, 21:58) Ye...   Oct 12 2012, 22:21
|||- - googlebot   I do not see how calling the phenomenon "prec...   Oct 12 2012, 23:43
||- - krabapple   QUOTE (googlebot @ Oct 12 2012, 16:44) QU...   Oct 13 2012, 22:05
|- - 2Bdecided   @Porcus: QUOTE (googlebot @ Oct 11 2012, 21...   Oct 12 2012, 10:08
|- - Arnold B. Krueger   QUOTE (googlebot @ Oct 11 2012, 16:44) Co...   Oct 16 2012, 15:51
- - greynol   I would be careful not to limit the word perceive....   Oct 11 2012, 22:58
- - greynol   I extract all the joy I could ever need from simpl...   Oct 12 2012, 06:11
- - Porcus   Among my friends, we have been blind testing ... h...   Oct 12 2012, 09:30
- - dhromed   But is there a problem?   Oct 12 2012, 22:12
- - AndyH-ha   The sighted test difference is not coming from the...   Oct 13 2012, 03:32
|- - greynol   QUOTE (AndyH-ha @ Oct 12 2012, 19:32...   Oct 13 2012, 14:16
- - Nick.C   @googlebot: You are now allowing the results to be...   Oct 13 2012, 09:43
- - 2Bdecided   I think Googlebot is making a valid philosophical ...   Oct 15 2012, 12:20
|- - pisymbol   QUOTE (2Bdecided @ Oct 15 2012, 07:20) I ...   Oct 15 2012, 13:48
- - skamp   QUOTE (mzil @ Oct 15 2012, 18:14) B. The ...   Oct 15 2012, 22:49
|- - krabapple   QUOTE (skamp @ Oct 15 2012, 17:49) What g...   Oct 16 2012, 03:58
- - greynol   There are ways of cheating to get positive ABX res...   Oct 15 2012, 22:57
- - mzil   [Trying to bring this back on topic] There is not...   Oct 16 2012, 04:32
|- - Woodinville   QUOTE (mzil @ Oct 15 2012, 20:32) [Trying...   Oct 16 2012, 04:33
|- - Porcus   QUOTE (mzil @ Oct 16 2012, 05:32) Here...   Oct 16 2012, 06:19
- - mzil   Please enlighten me. I am not a scientist nor have...   Oct 16 2012, 04:39
|- - krabapple   QUOTE (mzil @ Oct 15 2012, 23:39) Please ...   Oct 16 2012, 22:28
|- - mzil   QUOTE (krabapple @ Oct 16 2012, 17:28) QU...   Oct 17 2012, 00:30
|- - saratoga   QUOTE (mzil @ Oct 16 2012, 19:30) So even...   Oct 17 2012, 03:03
||- - mzil   QUOTE (saratoga @ Oct 16 2012, 22:03) QUO...   Oct 17 2012, 05:35
||- - saratoga   QUOTE (mzil @ Oct 17 2012, 00:35) Sorry, ...   Oct 17 2012, 05:43
|- - krabapple   QUOTE (mzil @ Oct 16 2012, 19:30) QUOTE (...   Oct 17 2012, 04:53
||- - mzil   QUOTE (krabapple @ Oct 16 2012, 23:53) QU...   Oct 17 2012, 06:36
||- - mzil   QUOTE (krabapple @ Oct 16 2012, 23:53) Do...   Oct 17 2012, 07:22
|||- - mzil   QUOTE (mzil @ Oct 17 2012, 02:22) QUOTE (...   Oct 18 2012, 19:16
||- - Arnold B. Krueger   QUOTE (krabapple @ Oct 16 2012, 23:53) Do...   Oct 18 2012, 14:04
|- - Porcus   QUOTE (mzil @ Oct 17 2012, 01:30) QUOTE R...   Oct 17 2012, 18:51
- - knutinh   Self-reporting about ones mental state surely carr...   Oct 16 2012, 21:02
- - greynol   Let me get this straight, the subconscious mind is...   Oct 17 2012, 05:28
- - mzil   Thanks, Porcus. I'll check it out.   Oct 17 2012, 20:14
- - krabapple   It's best to be careful drawing conclusions fr...   Oct 19 2012, 17:53
|- - 2Bdecided   QUOTE (krabapple @ Oct 19 2012, 17:53) It...   Oct 22 2012, 12:49
- - AndyH-ha   I could be missing the point, or dozens of them, b...   Oct 19 2012, 20:56
|- - Arnold B. Krueger   QUOTE (AndyH-ha @ Oct 19 2012, 15:56...   Oct 19 2012, 21:30
- - AndyH-ha   My point was not that many reported tests involve ...   Oct 19 2012, 22:31
2 Pages V   1 2 >


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 25th December 2014 - 15:19