Re: [Speechsc] [RAI] RAI review of draft-ietf-speechsc-mrcpv2-19

["Roni Even" <[e-mail]>]

Looks OK

Roni

 

From: [e-mail] [mailto:[e-mail]] On Behalf Of Dan
Burnett
Sent: Tuesday, December 29, 2009 1:01 PM
To: Roni Even
Cc: [e-mail]; [e-mail]; [e-mail]; [e-mail]
Subject: Re: [RAI] RAI review of draft-ietf-speechsc-mrcpv2-19

 

Hi Roni,

 

Just to finish up on your last comments . . .

 

-- dan

 

On Aug 12, 2009, at 3:15 AM, Roni Even wrote:





Hi Dan,

I understand your explanation about all these "vendor specific" parameter. I
think that since this a standard track document there should be some text
explaining the usage of these parameters as well as making a note that since
these are vendor specific information you cannot compare the values coming
from different vendors

 

Thank you.  I will note this in the next draft and suggest how these
parameters may be used in light of their vendor dependence.





 

 

As for my comment number 5 on payload type 96. My comment was that if the
m-line has a payload type number of 96 you must have a a=rtpmap line mapping
96 to a specific subtype name while for pcmu it is not mandatory to have
a=rtpmap like you have in your examples since payload type number 0 is a
static payload type number assigned to pcmu

 

 

I'm sorry, I did not explain this very well.  I understood your comment.  My
reply was that of the three examples, example 2 did actually provide the
a=rtpmap line for 96.  Since the payload type of 96 should not even have
been included in the first and third examples, once I removed it from those
two examples all three contained the proper a=rtpmap lines.

Although not necessary to have an a=rtpmap line for payload type 0, others
in the past had requested it so I left it in.





 

Roni Even

 

From: Dan Burnett [mailto:[e-mail]] 
Sent: Tuesday, August 11, 2009 9:22 PM
To: Roni Even
Cc: [e-mail]; [e-mail]; 'Eric Burger'; [e-mail];
[e-mail]
Subject: Re: RAI review of draft-ietf-speechsc-mrcpv2-19

 

 

On Jul 7, 2009, at 3:40 PM, Roni Even wrote:






Hi,

I was assigned to do a RAI review of the draft.  The draft looks ready for
publication to me. I have some comments mostly editorial.

The only issue I see that is not pure editorial is the issue of the
different parameters like confidence threshold, sensitivity level (see
comments 11, 13, 15, 16 and 17). I think that some clarification on the
semantics and the scale (for example are the values linearly spaced) as well
as when they are useful will be helpful to implementers.

1.       In figure 1 Expand the abbreviations TTS, ASR, SV , SI and how they
are related to the media resource types in 3.1

 

Done.  Added some text explaining Figure 1 and enhanced Figure 1 slightly
for clarification.




2.       In figure 1 there is a SIP dialog between the MRCPv2 client and the
media source/sink, what is this dialog, I only saw in section 4 a dialog
between the client and server.

Clarified in the first example of section 4.2 that the SIP dialog with the
media source/sink is not shown.

3.       In section 3.2 you have "For example:  <sip:[e-mail]>
sip:[e-mail]" twice one after the other.

 

Fixed.






4.       In the example in section 4.2 you "a=cmid:1", cmid is specified
later in the document so maybe you can add some reference to where it is
specified

 

Done.






 

5.       In the example is section 4.2 and in following examples you have
"m=audio 49170 RTP/AVP 0 96" but do not have an rtpmap parameter for mapping
96 (dynamic payload type number) to a media encoding name.

 

It is not in the first or third examples (Synthesizer only), but it is in
the second example (Recognizer).  I have removed 96 as an option for the
Synthesizer-only examples but let it remain as an addition for the
Recognizer example.






 

6.       In section 4.3 "Also note that more that one media session can be
associated with a single resource if need be, but this scenario is not
useful for the current set of resources". There is a typo the second "that"
should be "than". I am also not sure if the current syntax in this document
can support the mode.

 

Fixed the typo.






 

7.       In section 4.3 "The formatting of the"cmid" attribute in SDP
RFC3388 [RFC4566]". I think you meant SDP grouping and need the reference to
RFC 3388.

 

I removed the reference altogether because it already exists (correctly)
earlier in the paragraph.






 

8.       In section 5.1 "The message-length field specifies the length of
the message, including the start-line" is the length in Bytes, there is no
unit specified.

 

Changed "length of the message" to "length of the message in bytes".






 

9.       In section 6.3.1, typo you have "Verfication " instead of
verification. It appears twice in the section.

 

Fixed.






 

10.   In the example in section 7 you have "m=audio 0 RTP/AVP 0 1 3" payload
type 1 was deleted from the IANA registry, maybe have another payload type
number.

 

I just removed that payload type.  It is not germane to the example.






 

11.   In section 9.4.1, 9.4.2 and 9.4.3 you specify confidence threshold,
sensitivity level and speed vs accuracy. What is the scale here; is it
linear between 0 and 1. What is the absolute value of the number, if you
receive the same confidence level from two recognizers are they the same
(e.g. when using context block to switch servers).  For the speed vs
accuracy, how does the client know what is the relation between the value
and the number of available sessions, since this seems to be the reason for
using this parameter.

 

The interpretation of all of these parameters is implementation-specific
because the underlying technologies used to implement them vary and can even
be proprietary.  In practice the speech recognition and synthesis and
speaker authentication communities have lived with this state of affairs for
many years, and users of other APIs for this technology are well aware of
and have built applications that accommodate this variability in
interpretation.  It is outside the scope of this specification to attempt to
standardize interpretations of these values.






12.   In 9.4.9 and in 10.4.8, 11.4.11 what are the values for
media-type-value, you also mention audio and video but it looks to me that
this document only discusses voice.

 

Yes.  Although the original intent was to record speech, application authors
today are beginning to look at ways to incorporate other audio or video.
The intent of the sentences in these sections is to clarify that the
specification itself imposes no restriction on the types of media that are
allowed.






 

13.   In 9.4.35 and 9.4.36 what is the scale for the consistency here. How
does one know what close means. What is the consistency between different
recognizers.

 

The answer to question 11, above, applies here as well.






 

14.   In section 9.6.3.3 in the example (figure 2) confidence should be 0.75
and not 75

 

Fixed.






 

15.   In section 10.4.1 it is not clear how you measure the sensitivity in
order to specify, is it based on some SNR translated to 0 to 1 scale?

 

The answer to question 11, above, applies here as well.






 

16.   In 11.4.6 the same issue with the scale, how does the client know how
to set a value when working with different speaker verification servers.

 

Ditto.  I should point out that in all of these cases the parameters are
typically passed directly to the engine, and their interpretations are
defined (and described) in the vendors' documentation.  The most common
MRCPv2 server implementations are by the technology vendors themselves (the
providers of the synthesis, recognition, and verification engines).  This is
commonly understood in this technology industry (meaning those who use this
technology regularly).






 

17.   In 11.5.2.9 you state that the verification-score is not a
probability, so what is it. How can the client decide if, for example, 0 is
a good score for specifying the threshold.  I also noticed that the values
in the example in section 11.5.2.10 are very precise like 0.98514 is this
the expected precision. The examples here and in section 11.11 do not show
the threshold, if the threshold is required for this flow why not show it in
the example?

 

This parameter, as others mentioned above, has only a vendor-specific
interpretation.  In practice authors interpret these values based both on
guidance from the technology vendors and via experimentation on large sets
of recorded data.

 

The Min-Verification-Score threshold is not required to be set.  In many
cases the technology vendor has a fairly good understanding of what the
default threshold should be.  The verification-score is returned, however,
in case the application author determines (through experimentation, as
described above) that the default threshold is not producing optimal results
for the application.  In that case the author can set the threshold to a
different value or can set it to -1 and make the determination within the
application itself based on the verification-score values.






 

18.   In section 12.3 the suggestion is to use SRTP as the mandatory
interoperability mode. If the reason for mandating SRTP is for a common mode
you should also decide on a key exchange mechanism. I suggest you look at
http://tools.ietf.org/html/draft-ietf-avt-srtp-not-mandatory-02 for
discussion on media security.

 

Based on the discussion between you and Dan York on the list, I will change
this:

 

12.3. Media session protection 
Sensitive data is also carried on media sessions terminating on MRCPv2
servers (the other end of a media channel may or may not be on the MRCPv2
client). This data includes the user's spoken utterances and the output of
text-to-speech operations. MRCPv2 servers MUST support SRTP for protection
of audio media sessions. MRCPv2 clients that originate or consume audio
similarly MUST support SRTP. Alternative media channel protection MAY be
used if desired (e.g. IPSEC).

 

to this:

 

12.3. Media session protection 
Sensitive data is also carried on media sessions terminating on MRCPv2
servers (the other end of a media channel may or may not be on the MRCPv2
client). This data includes the user's spoken utterances and the output of
text-to-speech operations. MRCPv2 servers MUST support a security mechanism
for protection of audio media sessions. MRCPv2 clients that originate or
consume audio similarly MUST support a security mechanism for protection of
the audio. If appropriate, usage of the Secure Real-time Transport Protocol
(SRTP) [RFC3711] is recommended.

 

19.   In section13.7.2 you specify the attribute resource as session level
yet in the example in section 4.2 it is a media level attribute. The same
goes for the channel attribute

 

I have corrected both in section 13.7.2 to be media-level.






 

Thanks

 

Roni Even