Re: [rtcweb] #13: Transport of DATA_CHANNEL_OPEN

I've also been silent during this discussion, but I have a different view.

We've seen arguments about that OPEN "only" has a value for the 
application. But isn't the main target for the rtcweb work web 
applications? Randell has shown that OPEN has a big value for the web 
application, and, furthermore, the OPEN conveys information that is 
needed for the API that has been designed over at the W3C side.

And I have not really seen any arguments on why we should omit OPEN 
(only the view that it is not needed).

I think we should move on. Let's detail out how we treat data before 
OPEN has been received. And that design may not have to be perfect day 
one, we could tweak it based on experience from real world usage (there 
have been discussion on how (un-)likely it is that you could get a lot 
of data over but not OPEN). If we don't change the API the applications 
won't break.

Stefan

On 4/29/13 4:11 PM, Ejzak, Richard P (Richard) wrote:
> After remaining neutral during most of this discussion about the need for OPEN, I've finally come to the conclusion that Matthew is right.  The only thing the OPEN provides is the label, which is just an opaque string with no meaning to the browser or any intermediaries.  It only has application significance.  If this is the case, why not just use stream id?  What can you possibly do with label that you can't do with pre-assigned stream ids (this is a serious question to Randell since it is fundamental to whether we need OPEN)?  Since label only has meaning when both browsers run the same application, what does it matter if you agree on a set of opaque strings to correspond to various functions or just agree on a set of stream ids for these functions?  It doesn't change the complexity of the code and it doesn't even change the readability of the code if you just enumerate the reserved stream ids and "label" them in the code.
>
> You can even reserve stream ids in odd/even pairs so that either end can initiate them.  You can reserve blocks of stream ids if you want multiple instances of the same type of DC.  I don't see any significant limitation here.
>
> Getting rid of OPEN eliminates any issues around their handling in the unordered case.
>
> Getting rid of OPEN does not preclude use of new subprotocols as needed and does not preclude use of any other signaling to negotiate use of DataChannels between heterogeneous applications.  OPEN does not help in this case anyway.
>
>
>> On 4/25/2013 12:36 PM, Randell Jesup wrote:
>> On 4/26/2013 9:53 AM, Matthew Kaufman wrote:
>>> On 4/25/2013 2:42 PM, Randell Jesup wrote:
>>>> On 4/25/2013 5:10 PM, Cullen Jennings (fluffy) wrote:
>>>>> So with my co-chair hat on here...
>>>>>
>>>>> It seems we have been around the need for OPEN several times and
>>>>> have come to consensus on it in the past. Can someone please:
>>>>>
>>>>> 1) summarize the arguments that in the past that lead us to think
>> we
>>>>> need OPEN
>>>> The Open message has some important and useful properties:
>>>>
>>>> 1) It's easier to work with.  JS isn't an ideal language for writing
>>>> network protocols, especially for non-network-programmers (i.e. a
>>>> large portion of the expected developer community).  Open makes it
>>>> really easy for a developer to get the type of bidirectional stream
>>>> them want with little fuss, and in a manner that maps easily to APIs
>>>> they're already used to (e.g. WebSockets).  In many cases the label
>>>> field will avoid the JS programmer having to build their own
>>>> mini-protocol to identify their channels (and this is especially
>>>> painful for them if it's an unreliable channel).
>>> Easier how? The initiating side needs to create their object without
>>> the benefit of the contents of an open message. If a JS developer
>>> isn't smart enough to set up the channel parameters they want at the
>>> initiating end, then there's nothing we can do for them. There's then
>>> several ways forward after that, including "the other side does the
>>> same thing to create their end" (my preference) and "the parameters
>>> are transported using the existing SDP O/A mechanism to the far end"
>>> (generally how the WG solves this kind of problem).
>>>
>>> As far as I can tell, the one useful thing in the OPEN message is the
>>> label (as I pointed out in my very first inquiry about why anyone
>>> thought it was needed), and yet the label is allowed to be null, so
>> in
>>> that case how can it be helpful?
>> It's helpful if you need it (if you're opening a number of channels,
>> such as one per participant in a conference).  Protocol is also helpful
>> (note it's in the dictionary and is also optional); both for cross-
>> application channels and for within an app to indicate what handling
>> logic the channel should feed.
>>
>> // pseudo-codey:
>> function called_from_ondatachanne(event) {
>>     channel = event.channel;
>>     if (channel.protocol == "file transfer") { channel.onmessage =
>> handle_incoming_file; }
>>     else if (channel.protocol == "chat") {chat = new
>> chat_instance(channel.label); channel.onmessage = chat.incoming; } }
>>
>>>> 2) It makes it possible to have different applications exchange
>> data,
>>>> by having an IANA-registered protocol name, like WebSockets (and
>>>> there was strong agreement on it's being needed for that at
>>>> Atlanta.)  With external-negotiation-only, it would be very hard for
>>>> different apps to interoperate, since they'd need to agree on
>>>> negotiation protocols as well, which are likely to be highly
>>>> divergent between apps.
>>> If we went with "negotiated via the SDP O/A mechanism" then we could
>>> go to our favorite SDP-defining WG and have a negotiation protocol
>>> that is standardized, so not divergent at all between apps.
>> We also don't have one that's divergent by using Open (and we leave the
>> option of externally negotiating or pre-defining channels). And SDP is
>> at least one signaling-RTT to complete, plus a bunch of logic to handle
>> matching everything up (what if channels disappear in the offer or
>> answer, or in an offer or answer they change properties? What if the
>> signaling channel is slow or unavailable any more (server rebooted,
>> server has network issues, etc)?
>>
>> I'm not saying there aren't answers/solutions to all these.  But those
>> answers and solutions would need to be speced out or at least
>> understood, and the code to handle all of them is likely to be similar
>> in scope (though quite different in detail).  And the less done in SDP,
>> the better IMHO; I have no wish to increase our reliance on SDP.
>>
>>>> 3) The Open message, being in-channel, reliable and in-order, makes
>>>> the issue that was the genesis of this thread (what to do with data
>>>> that arrives unexpectedly) simpler.
>>> I disagree. It increases the number of possible states... there's
>>> cases where the OPEN message arrives but the application doesn't want
>>> to be receiving that data, and the cases where the OPEN message
>>> doesn't arrive but the application does want the data that is already
>>> arriving. Both of those states don't exist if there's no OPEN
>> message.
>>
>> My point was you need to decide what to do with that data regardless of
>> if the channel gets created with a delayed OPEN or with a delayed
>> external negotiation.  And with OPEN, those cases don't even arise for
>> ordered channels.
>>
>>>> With Open, only degenerate cases can cause more than a relatively
>>>> small amount of data to be buffered.
>>> Sure, but those degenerate cases occur many times per day on the real
>>> Internet.
>> I doubt that.  The degenerate cases for Open require that all the open
>> packets get lost, while a sizable amount of non-open packets get
>> through
>> - and after a time, SCTP will fail the entire association if reliable
>> data isn't getting through.  Plus, as I indicated, we can also have
>> limits on time and/or amount of data buffered.
>>
>> Also: nothing *requires* that an application send data immediately on
>> onopen.  An application can institute it's own handshake trivially:
>> function my_ondatachannel(event) {
>> event.channel.send(ok_to_start_sending); ... }
>>
>>>> With external notification, the external negotiation channel can
>> fail
>>>> (or be very slow), or the app can have a bug and fail to install the
>>>> negotiated values, leading to larger or unbounded buffering
>>>> requirements - or you punt the problem to the application by
>>>> delivering the data, but the application is facing the same
>> conundrum
>>>> of buffer it or throw it away.
>>>>
>>>>> 2) sketch out the range of possible solutions to deal with
>>>>> unexpected data before the OPEN
>>>>
>>>> The issue exists regardless of whether Open is used or external
>>>> negotiation (and in fact is much simpler for Open).
>>>>
>>>> *tl;dr: *I'm ok with any setting of maximum sizes and/or times that
>>>> would not adversely impact temporary buffering for normal cases with
>>>> Open of unordered channels.  This is option C below. I also would be
>>>> ok with B, but I realize others may not be.
>>>>
>>>> Regardless of supporting Open or not, any external negotiation of
>>>> dynamic channels must use one of these:
>>>>
>>>> A) a 2-or-3-way handshake so the sender knows the receiver is ready
>>>> to receive the data on the specified channel before sending it, or
>>>>
>>>> B) unbounded buffering of data if the external negotiation messages
>>>> are delayed (again, what we're discussing in this thread), or
>>>>
>>>> C) bounded buffering of data (bounded by time, size or both), with
>>>> data being dropped and the channel closed if the limits are
>> exceeded, or
>>>> D) deliver unexpected data to the application, which will do .... I
>>>> don't know what with it.
>>>>
>>>> For (D), the application will likely drop it on the floor (leading
>> to
>>>> hard-to-test-for problems if the channel is later configured by Open
>>>> or external negotiation), or buffer it waiting for the channel to
>> open.
>>>> Supporting Open or not has little bearing on these scenarios
>>> So if that's true, then why do we need it?
>> This was a discussion of "what to do with data that arrives
>> unexpectedly
>> (i.e. before Open, or before the application installed the result of an
>> external negotiation); this was the original point of the thread before
>> you decided to question the selection of Open as a message.
>>
>> The caveat at the top was "external negotiation of dynamic channels
>> must
>> do one of these" regardless of whether Open is supported.  The options
>> for use with Open are the same, but the amount of data that can be
>> buffered is smaller (since with external negotiation there are no
>> limits
>> on the amount that could attempt to be queued -- the same applies SDP
>> if
>> you allow the sender to send data before the negotiation is complete;
>> SDP is effectively a form of external negotiation).
>>
>>>> -- and in fact, since Open is in-channel, reliable, and ordered, it
>>>> reduces the problem set (when Open is used) to only unordered
>>>> channels (in ordered channels Open will always be first).
>>>>
>>>> Buffering unexpected data on channels (options B or C) is useful. It
>>>> means that in the external negotiation case, one side asking for a
>>>> new channel to open by some private means doesn't need to wait to
>>>> start sending data on that channel.
>>> Of course this is different from the TCP model, where the buffering
>>> happens at the *sender* until the handshake is complete.
>> Ok, but this isn't TCP.  And handshakes (and the resultant delay before
>> onopen fired) were eliminated in order to respond to the requests of
>> multiple people, giving us a 0-RTT declarative protocol where you can
>> send immediately, which will help apps that need to exchange data
>> quickly at the start of a connection.
>>
>>> The problem here is that we have an existing mechanism inside of SCTP
>>> and then we're adding another layer on top of that which is neither
>>> inside SCTP (where it probably belongs) or under the application
>>> developer's control (where I'd like to see it, if the data transport
>>> protocol can't have it), and this layer conflicts with what both of
>>> those might be doing.
>> If you're referring to Open, you can totally ignore Open and not use it
>> in your datachannels, and use all external negotiation or pre-defined
>> channels.  If you want, you can just pre-define N open channels that
>> the
>> other side can then use to send data to you, and nothing will ever be
>> buffered, and you can handle it all yourself.
>>
>>>> Note that with external negotiation (possibly on a non p2p path,
>> like
>>>> via signaling), the receiver might not know what to install for a
>>>> short while, especially if there's a routing issue or server issue
>>>> (not a problem that happens with Open).
>>> Which is fine if the sender doesn't start sending (like TCP). Or if
>>> the transport protocol itself handles the delivery of the channel
>>> metadata. (Which is what SCTP, or whatever transport we choose for
>>> RTCWEB, should do) then you just handle the label and whatever other
>>> metadata the receiver would like to know at that layer.
>> I.e. data streams from RTMFP, I take it? ;-)
>>
>>>> Since normal Open cases have very little chance of triggering this
>>>> problem (triggering buffering), some arbitrary size limit seems
>>>> reasonable (option C).  Often in network protocols there are small
>>>> buffers (4, 16, 64KB).  I prefer a larger value of say 256KB so that
>>>> apps using external negotiation can just send largish data
>>>> immediately - and note: actually buffering data is still an unlikely
>>>> occurrence even in most external negotiation cases.  If people want
>>>> to bikeshed on the buffersize or timeout, that's fine. ;-)
>>>>
>>>> Also, external negotiation is the only case where more than a
>> trivial
>>>> amount of data can "pile up" in the buffer waiting for the receiving
>>>> side to finish it's side of the negotiation (i.e. if your external
>>>> negotiation channel fails or the app has a brain fart).
>>>>
>>>>
>>> Another great argument for having this negotiation happen within the
>>> transport itself.
>> If you want to argue for replacing SCTP as the base layer - we made
>> that
>> decision a good long while ago.  And there's a good, maintained,
>> BSD-licensed implementation - and the authors of that are working with
>> us here in designing a DataChannel protocol to ride on top of it.  If
>> you want to argue that the layer on top of SCTP should be handshaked
>> once again (as it was until people objected) that's up to you to make
>> the case for.
>>
>> IMO:
>> I'd like to avoid continuing to circle through the range of all
>> possible
>> protocols one at a time, repeating every few months.  I'll assert we
>> can
>> bikeshed on this until we all retire, and we'll never have something
>> that makes everyone entirely happy (unless most people get tired and
>> stop caring).  All solutions have tradeoffs; none is perfect for all
>> use-cases and users.  We arrived at a proposal that had general
>> agreement in the room to move forward with per the consensus call, and
>> I'm preparing a WG draft in response to that.  Some may like it more
>> and
>> some less, but there was agreement that people could work with it.
>>
>> --
>> Randell Jesup
>> randell-ietf@jesup.org
>>
>> _______________________________________________
>> rtcweb mailing list
>> rtcweb@ietf.org
>> https://www.ietf.org/mailman/listinfo/rtcweb
> _______________________________________________
> rtcweb mailing list
> rtcweb@ietf.org
> https://www.ietf.org/mailman/listinfo/rtcweb