[AVT] SF Issue Summary: MWPP

[John Lazzaro <[e-mail]>]

Hi everyone,

	Here is the combined status report for the MWPP normative
(draft-ietf-avt-mwpp-midi-rtp-06.txt) and informative
(draft-lazzaro-avt-mwpp-coding-guidelines-02.txt) I-D's, also at:

http://www.cs.berkeley.edu/~lazzaro/sa/pubs/txt/current-mwpp.txt
http://www.cs.berkeley.edu/~lazzaro/sa/pubs/txt/current-guide.txt

	The normative document went through one revision (-06.txt),
fixing up many small details (see change log for the list). The
informative document was completed (-01.txt) and then touched up to
conform to the normative -06.txt (-02.txt).

	I'm assuming there is no reason for the informative document
to be a WG item, let me know if there is a wish for the document to be
relabeled as a WG item and I will retitle and resubmit.

The Road to Last Call ...
-------------------------

	Since Atlanta, many hours has been spent in e-dialogue with
MMA (MIDI Manufacturers Association) folks. The MMA has taken a more
active interest in MWPP recently, Jim Wright is the transport-expert
MMA contact. We have an informal list of open issues of potential
interest to the MMA (mixed in with a few from other sources sent in
recently), and I'm doing design work point-by-point off this list.

	Once the design is complete, and consensus occurs on the
result, we may be only an ID-nit-check away from Last Call. Most of
these items are better discussed after I release an I-D with a
proposed solution, since they are much more about MIDI than RTP
issues.

	However, below I bring up two points that would benefit from
pre-design discussion (on list, and at the meeting if there is
interest). Both are timing-related, and concern MWPP on the LAN (or
well-behaved WAN) in a stage or studio context. Both describe cases
where the natural MWPP interpretation of the default AVP audio timing
semantics do not completely address the needs of the application
domain.


[1] Sending-time proxies
------------------------

	One issue concerns "operating at the underlying latency of the
network, on a network known to have low jitter". This is of particular
interest with musicians who play a (soundless) keyboard controller
that generates MWPP, which is received by a separate box that makes
noise. As network latency is in the human feedback loop, low latency
is essential.

	In these environments, it may be desirable to set up a timing
mode where a sender takes on responsibility to transmit, at a precise
time interval relative to the MIDI source timing, RTP MWPP packets
that (usually) encode a single MIDI command. The receiver, aware that
the timing mode is in use (via an SDP parameter), uses this "time of
flight" timing together with the timestamp of the packet, as inputs
into whatever algorithm it chooses to use for playback. One potential
playback algorithm assumes zero network jitter, ignores the timestamp,
and executes the command on receipt. Other algorithms incorporate RTP
timestamps into the algorithm.
 	
	Aside from the new SDP fmtp parameter to code that the sender
is communicating timing information in its sending dynamics, a few
other MWPP additions are helpful for this mode:

 -- Defining the timestamp of the last MIDI command in the
    MIDI command section as the proxy for the sending time. This
    definition is particularly useful when combined with MIDI
    command section syntax (in the current I-D) that permits
    the last MIDI command in this section to be the "null" command.

 -- Defining the ptime and maxptime SDP values as originally intended
    in SDP -- the length of time encoded by the payload. A second
    set of MWPP fmtp parameters should be introduced to take over
    current MWPP use of ptime/maxptime (the delta of the RTP
    timestamps of two successive packets).


[2] Multi-format sync
---------------------

	A second LAN timing issue concerns multi-format sync.
Consider this block diagram (fixed-font recommended):



   -------------                                           --------
  | Digital     |  Digital Audio (ADAT, AES/EBU, S/PDIF)  | Digital| 
  | Audio       |----------->----------------------->-----| Mixer  |
  | Workstation |                                         | With   |-->out
   -------------                                          | Word   |
          |                                               | Clock  |
          |                                                --------
          | RTP/MWPP                                          |
  	  |                                                   |
          |                                                   ^
   -------------                                              |
   |           |  Digital Audio (ADAT, AES/EBU, S/PDIF)       |
   | SoftSynth |----------->--------------------------->-------
   |           |
   -------------


	The Digital Audio Workstation (DAW) generates two pre-recorded
streams: digital audio, sent out over one of the standard digital I/O
wire/fiber formats (ADAT, AES/EBU, S/PDIF, TDM, etc), and RTP/MWPP
over a LAN. The RTP/MWPP stream goes to a separate box running a
synth, that also has standard digital I/O format out. A digital mixer,
with word-clock to get frame-accurate audio, does the final mix (sent
to studio monitors, a mastering rig, etc).

	The DAW is playing back pre-recorded data here (overdubs are a
separate issue, but I believe solving the playback-only issue also
solves overdubs). DAWs normally let you time-shift all of the tracks
(MIDI and audio), to compensate for delays on the way to the final out.

	But, how does one compensate for a softsynth with an RTP
receiver, whose receiver-buffer latency may be variable? I can
imagine two approaches:

  [A] Assuming the softsynth and the DAW have NTP clocks that are kept
  in near-frame-accurate sync, you want SDP semantics that request a
  time differential between the RTCP NTP of the sender and the synced
  NTP clock of the receiver.

  [B] You want the ability to specify a fixed receiver latency, perhaps
  with options that break apart packet buffering latency from post-RTP
  synth latency (or maybe not).

[A] is potentially "calibration free", while [B] merely gives a studio
engineer a shot at manually calibrating the rig.

So, the question for the WG is, how do you feel about [A] and [B] as
design paths to pursue? [A], [B], both, neither? As always, an option
is to declare the problem domain "out of scope of RTP." But it would
be good to include this "anti-recommendation" in the memo.

-------------------------------------------------------------------------
John Lazzaro -- Research Specialist -- CS Division -- EECS -- UC Berkeley
lazzaro [at] cs [dot] berkeley [dot] edu     www.cs.berkeley.edu/~lazzaro
-------------------------------------------------------------------------

_______________________________________________
Audio/Video Transport Working Group
[e-mail]
https://www1.ietf.org/mailman/listinfo/avt