Re: [codec] #5: Mention DTMF in requirements

["Raymond (Juin-Hwey) Chen" <[e-mail]>]

Roman,

Answers to you questions in-line.

Raymond

From: Roman Shpount [mailto:[e-mail]]
Sent: Friday, April 02, 2010 6:26 PM
To: Raymond (Juin-Hwey) Chen
Cc: [e-mail]
Subject: Re: [codec] #5: Mention DTMF in requirements

I understand your points. There are few questions that I have:

1. Do we need a real world DTMF and telephone signals recording database to implement this? Correct me if I am wrong, you've done your tests for BV16 using your own DTMF generator and unspecified "commercial" DTMF detector. How would define this test in a way it is possible to replicate it by third parties and reference it in a standard?
[Raymond]: We can use real-world recorded DTMF signals if we want to, but we don't have to. Yes, in our DTMF testing of BV16 we used a DTMF generator software to generate the waveform of 20,000 random DTMF symbols for each of 24 DTMF test conditions, for a total of nearly half a million DTMF symbols. After passing through the codec, the DTMF signals were passed through a PC executable program which simulated a DTMF decoder that was widely deployed in commercial VoIP applications. The decoded DTMF symbols were then compared with the original DTMF keys used to generate the symbols to calculate the error rates. As you can see, such a test method is purely based on software and can easily be duplicated by third parties.  If the codec WG decides not to do anything with DTMF, then obviously nothing further needs to be done, but if the WG decides that the codec candidate needs to be tested for DTMF pass-through, I will try to dig up our DTMF test software package and get internal approval to donate it to IETF for this purpose.
2. Did you ever test your CODEC for other telephony signals, such as /ANSam, ANS, CED, and CI tones? These tones are needed in order to detect modems and fax machines and switch to an appropriate payload. What about progress tones?
[Raymond]: No, we didn't.  However, if these are single-frequency tones, I would expect that it should be easier for a typical codec to pass these tones than to pass the dual-frequency DTMF signals without causing significant degradation in the subsequent processing of these tones.
3. Is this criteria feasible for large frame CELP CODECs? Are we going to exclude them based on this criteria?
[Raymond]: If a CELP codec with a large frame size is designed such that it can provide high encoding fidelity for DTMF signals, then it is still feasible for such a CELP codec to pass DTMF signals without causing subsequent DTMF decoding errors, so being a large frame CELP codec does not automatically mean it will be excluded.
_____________________________
Roman Shpount - www.telurix.com<http://www.telurix.com>

On Fri, Apr 2, 2010 at 9:05 PM, Raymond (Juin-Hwey) Chen <[e-mail]<mailto:[e-mail]>> wrote:
Comments in-line.

Raymond

From: [e-mail]<mailto:[e-mail]> [mailto:[e-mail]<mailto:[e-mail]>] On Behalf Of Roman Shpount
Sent: Friday, April 02, 2010 12:03 PM
To: [e-mail]<mailto:[e-mail]>
Subject: Re: [codec] #5: Mention DTMF in requirements

This is all wonderful, but what seems to be missing here is any type of standard test for DTMF tone detection. We have a number of tests for things that should not be detected as DTMF (various talk-off tapes), but there is no test with samples of valid DTMF signals. In real world you get quite a variety of tones that different handsets or telephone devices generate when a digit is pressed. Some of those tones are almost at border line conditions for a valid DTMF tone.
[Raymond]: I commented on this earlier. ITU-T G.720 can be used for general guidelines. There were some published papers on codec DTMF pass-through testing based on G.720. Our BroadVoice16 paper is one of them (even though we relaxed some corner conditions of G.720 to get a better idea of real-world performance). If people are interested, I can share more details about the test method.
Second issue, which was already mentioned here, would be performance of DTMF tone detector in the presence of packet loss or time stretching caused by jitter buffer. Those things, even if they do not prevent DTMF detection, are almost guaranteed to split a single long tone in two.
[Raymond]: Packet loss certainly will cause problems, but if the packet loss rate is relatively low, there is still a good chance that DTMF digits can get through.
Finally, we seemed to concentrate on DTMF tones only, but in reality all the signaling/special tones are quite important. Fax Send/Receive tones come to mind almost immediately as tones that are typically detected inband (very few gateways encode those tones using RFC 4733/2833)

I think we should formulate this requirement not in terms of DTMF tone but in terms of accuracy of reproduction of  periodic signals in specified frequency ranges. To be honest, the most robust solution would be to incorporate RFC4733/4744 as a part of the codec in a manner similar to comfort noise frames.
[Raymond]: I don't think we can do it in terms of "accuracy of reproduction of periodic signals in specified frequency ranges".  That may work for some single-frequency signaling tones, but it won't work for DTMF, since DTMF signals are not periodic in any sense as the two frequencies of the sine waves are not harmonically related.
_____________________________
Roman Shpount - www.telurix.com<http://www.telurix.com>
On Fri, Apr 2, 2010 at 1:34 PM, Christian Hoene <[e-mail]<mailto:[e-mail]>> wrote:
Hi Stephan,

Comments inline:


---------------------------------------------------------------
Dr.-Ing. Christian Hoene
Interactive Communication Systems (ICS), University of Tübingen
Sand 13, 72076 Tübingen, Germany, Phone [phone]
http://www.net.uni-tuebingen.de/
From: stephen botzko [mailto:[e-mail]<mailto:[e-mail]>]
Sent: Friday, April 02, 2010 4:50 PM
To: Christian Hoene
Cc: Michael Knappe; [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>
Subject: Re: [codec] #5: Mention DTMF in requirements
(a) "MUST NOT" means that it must fail.  Perhaps you meant "need not" or "might not"?
CH: Yes, I meant need not (or the German "muss nicht")

(b) SHOULD is commonly used in establishing requirements, and it certainly was used in MARTINI and other working groups at IETF77 in
requirements presentations with no confusion. It has a clear meaning in the requirements context (desirable but not essential) and I
see no reason to avoid its use at this phase.  There will certainly be other things that are "nice to have", and it is appropriate
to track them and consider them in the selection/standardization process.
CH: Better? "The codec SHOULD support the transmission DTMF at most transmission conditions."

I agree that language about DTMF might not need to be in the final RFC.  Though if DTMF quality is known to be inadequate, it
probably makes sense to tell people that.
CH: Agreed. The limits of the codec shall be clearly stated.

I also agree to the obvious comment that DTMF detection methods are out of scope.
CH: Agreed.

CH: Isn't time to include the MUST requirement for DTMF testing and SHOULD requirement for transmission support into the
requirements document.
Then, we could close issue #5 and continue with other things.

 Christian



_______________________________________________
codec mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/codec