Re: [rtcweb] New Version Notification for draft-jesup-rtcweb-data-01.txt

[Randell Jesup <[e-mail]>]

On 11/2/2011 9:12 PM, Michael Thornburgh wrote:
> comments on this draft:
>
> section 3.2: reliable streams would be fairly simple to layer on top of any reliable and in-order datagram delivery service, not just SCTP's.


Yes.

> section 4.1: DCCP is not a good choice when any kind of reliability is desired.


I  agree; it was added in there by one of the authors to cover the 
options available.

> general comments on SCTP:
>
> Matthew and i examined SCTP when we were designing MFP (the predecessor of RTMFP). in fact, if you look at the MFP protocol spec in the Internet Archive, you might notice a suspicious similarity to SCTP's chunk scheme.  :)
>
> we rejected SCTP as inappropriate for real-time communication and our other use cases for several reasons. some of these have been at least partially addressed since our initial dismissal of SCTP in 2004 (in particular, DTLS in SCTP addresses the limitations of using TLS in SCTP [RFC3436], which at the time was the only way to "secure" SCTP messages; that mechanism meant that all secure messages/streams had to be fully reliable and in-order, which was unacceptable for real-time communication).


Ok, that sounds like SCTP/DTLS (or at least DTLS/SCTP) is usable given 
your comments above.

> issues with SCTP that remain, and which i believe make it undesirable for WebRTC, include:
>
>    o as you point out in section 5.3, running SCTP over DTLS, rather than DTLS over SCTP, is desired so that all SCTP fields, not just the user data, are secured. however, this configuration is currently not defined by IETF and would have to be specified.


Ok - though conceptually SCTP should be able to run over anything 
presenting a UDP-like interface.   There are some fallouts, like no 
multi-homing.  Can some others here (Magnus?) with better understanding 
of the exact RFC requirements state if it would be acceptable to 
reference a SCTP-over-DTLS draft, which should be fairly straightforward 
to create and move forward?

>    o DTLS and SCTP handshakes must be performed serially (no matter which order they happen in), which increases the number of round-trips necessary to establish communication.


True, though since we're not sending media data over the SCTP link I 
don't think this is a significant problem; almost any 
reliable-datagram-over-DTLS would have a similar issue I think.

>    o (most important for real-time communication): each user data fragment/chunk is assigned an SCTP Transmission Sequence Number (TSN) at the time of first transmission. that means even if your SCTP implementation supported stream prioritization somehow, the priority decision is only made at first transmission time. since there's just one TSN space and the Gap Ack structure only talks about TSNs, it's undesirable for gaps to persist (else the Gap Ack structure will continue to grow as more losses naturally happen). therefore it's desirable to repair gaps as quickly as possible. this may inappropriately increase the priority of a low priority fragment/chunk/stream during periods of congestion, which is exactly when priority matters (this is a "priority inversion").


Priority inversion is a bad thing; though since it would only apply to 
the data streams the impact is limited.  I'd be interested in hearing 
Michael Tuexen's/etc discussion of this, and how big an impact it would be.

>    o (second most important for real-time communication): there's only one receive window advertisement for all of the streams, rather than one receive window per stream. this means there's no per-stream flow control. so if you're receiving (for example) a bulk file transfer and real-time player position updates and text chat messages, and you need to suspend the file transfer stream for a time, that means you must also suspend the player position updates and text chat messages. unless you spin up an entirely separate SCTP for each flow control domain, which is lame and defeats the purpose of stream multiplexing.


That's more concerning.  Correct me if I've misunderstood: if I stop 
reading from the file-transfer stream, that will block the other 
streams?  One could layer something on top of SCTP to provide per-stream 
queues I imagine, and ensure in the rtcweb code that the main receive 
window doesn't stall, but the problem with that would be providing 
reliability if the file-transfer stream is reliable - if it's reliable, 
we'd have to in theory buffer a large amount of data *or* block other 
channels *or* put a flow-control layer on top of the reliable channels, 
which hurts the overall complexity and advantage of using existing 
code.  Perhaps I do misunderstand this issue (I hope so actually).

Or we state that the JS app must service all channels expeditiously or 
real-time channels may be blocked, which is probably the way we'd 
resolve it.  Definitely sub-optimal and an annoyance for the JS app 
writer in some cases, but probably not a major problem.

>    o SCTP specifies the maximum number of streams in each direction at association startup. web applications may not know the number of streams needed up front; in fact, the number of streams needed in any real-world non-SS7 data application is very likely to evolve over the lifetime of that application, naturally increasing and decreasing.


I would assume we would define a maximum number of data streams, and 
specify that number when opening the association.  Not optimal, but 
workable.

>    o the semantics of unordered messages are confusing and not a good map to WebRTC. they are semantically equivalent to an entirely separate stream loosely associated to the ordered stream of the same number. there's no way to tell (without application layer sequencing) if an unordered message has been lost/abandoned by the sender. at the protocol level, TSNs can be used to recover the queuing order of received unordered messages, but the TSNs are semantically disconnected from the SCTP user. recovering the original queuing order over a short reorder/reassembly window period is desirable in some real-time applications.


Thus far we haven't required that unordered delivery be exposed to the 
JS app, though that capability would be handy for some uses.  I have no 
problem with the application being required to include its own sequence 
number in the data if it cares and if that helps with this objection.

>    o an SCTP receiver should be able to choose to receive stream messages in originally-queued order or as-received-on-the-network order on a per-stream basis, and be able to recover the original queuing order to whatever extent desired (potentially limited by real-time constraints) when receiving in network order. SCTP's unordered message semantics are designed for "out-of-band" messages, and are not a good fit for general "real-time" data. transmission order should be determined by the sender, reception order should be determined by the receiver.


Agreed; but the spec (and code) for SCTP exists, even if sub-optimal, 
and timeframe is a major constraint.

>    o the stream sequence number being only 16 bits limits the maximum message rate through high delay networks where message gaps can be reliably detected at the receiver, when the sender uses limited-reliability, to 32767 messages/RTT. that sounds large, but could be easily reached even today in moderately high bandwidth*delay paths if messages are small.


Worst-case RTT is probably on the order of 2s (~2 satellite links each 
direction).  Note that for most use-cases this would be an intolerably 
long delay, but not for all.  16K messages/second may be reachable in 
theory, but given the likely bandwidth of said long-latency links I 
don't think this is a restriction that would bother me at all (even with 
high bandwidth I don't think this would bother me).

> specifying and implementing SCTP over DTLS over ICE over UDP is probably nearly as much work as specifying and implementing a new network protocol that actually solves the problems cleanly and holistically. having done that before a few times, i'm in favor of the clean holistic approach.  :)

Before I answer that: Thanks greatly for the detailed critique!

In response I have to say: put a protocol proposal on the table or 
gather a group to do so, and do so ASAP, because if we don't get an 
alternative, it *will* be either SCTP/DTLS/(ICE/UDP) or 
pseudo-TCP/DTLS/(ICE/UDP), and I don't know what we'd do for the 
unreliable congestion-controlled channels (maybe DCCP).

If you do put forward a proposal, do you really believe it can be done, 
standardized, agreed to and implemented in the timeframe required?  
Specifying SCTP/DTLS seems relatively straightforward compared to 
standardizing a new protocol...  The timeframe for this is quite 
constrained already; one of the strongest arguments against SCTP would 
also be timeframe, except that SCTP gives us congestion-controlled 
unreliable data, which pseudo-TCP-over-DTLS doesn't - we'd have to use 
yet another unspecified mechanism for congestion control of the 
unreliable data channels.




I think the hottest item you've flagged from my perspective is the 
receive-window blocking, followed by wanting to understand the possible 
priority-inversions.  Michael?  Randy?

-- 
Randell Jesup
[e-mail]