Re: [Netconf] SSE and HTTP/2 in restcon-notif [Was: activate-configured-subecription - WGLC subscribed-notifications-16]

["Reshad Rahman (rrahman)" <[e-mail]>]

Hi,


On 2018-09-20, 4:06 PM, "Eric Voit (evoit)" <[e-mail]> wrote:

    Hi,
    
    > From: Martin Bjorklund, September 20, 2018 3:20 AM
    > 
    > Hi,
    > 
    > [subject change; this thread is now about SSE and HTTP/2]
    > 
    > "Eric Voit (evoit)" <[e-mail]> wrote:
    > > > From: Martin Bjorklund, September 19, 2018 2:38 AM
    > > >
    > > > Hi,
    > > >
    > > > "Eric Voit (evoit)" <[e-mail]> wrote:
    > > > > Hi Douglas,
    > > > >
    > > > > From: Douglas Hubler, September 18, 2018 5:35 PM
    > > > >
    > > > >
    > > > > On Tue, Sep 18, 2018 at 3:40 PM Martin Bjorklund
    > > > > <[e-mail]<mailto:[e-mail]>> wrote:
    > > > > So it seems you support my point to make this draft use SSE.  Then
    > > > > if you use HTTP/2 you (the client) can simply create one HTTP/2
    > > > > stream per notification stream.  This just falls out for free with HTTP/2.
    > > > >
    > > > > yes, with caveat that I haven't attempted actual implementation yet.
    > > > >
    > > > > <Eric> Agree that SSE over HTTP2 should work.  However to me using
    > > > > SSE here seems redundant as HTTP2 provides the necessary mechanisms.
    > > >
    > > > Eric, can you explain how this would work w/o SSE?  Which mechanism
    > > > in
    > > > HTTP/2 are you referring to?
    > >
    > > After an HTTP2 OK, the HTTP2 server has the ability to forward an
    > > unbounded number of HTTP2 Data Frames on the HTTP2 stream.
    > 
    > But it must still adhere to the normal HTTP semantics.  If there was a request, it
    > needs to send the response.  It can split the response into several DATA frames
    > of course.  But it cannot start to send "other stuff" instead of the response,
    > which figure 1 in
    > restconf-notif-07 possibly implies.
    
    Agree.   Not sure where you see that in Figure 1.   The HTTP2 OK doesn't mean the end of the response. (RFC-7950, page 57.)      Any suggestions on how to make this clearer?
     
    > > The stream
    > > is not considered complete until an END_STREAM is passed.  (Note this
    > > is independent of anything regarding HTTP2's PUSH_PROMISE.)
    > >
    > > You can see an example flow in Figure 1 of
    > > draft-ietf-netconf-restconf-notif.  At IETF 97 we asked for some
    > > Feedback on this proposal.
    > > http://www.voit.org/IETF/Subscriptions-NETCONF-IETF97.ppt (see slide
    > > 14).
    > > But I am not sure how much effort the WG should place here.  The work
    > > here is several years old, and the market has moved substantially
    > > since then.
    > 
    > So do we agree that the restconf-notif draft will be updated to use SSE
    > regardless of HTTP version?  If there is any text at all about HTTP/2, it will be an
    > "implementation note" where it is recommended to use a separate stream per
    > subscription?
    
    I will leave that up to Reshad to frame.  I won't fight against it, although there is interplay here with subscription bundling (per below).

<RR> I'm ok with that.  

Regards,
Reshad.
     
    > > > > But
    > > > > I won’t get in the way if people really do want to go in this
    > > > > direction.
    > > > >
    > > > > > I don't understand the details surrounding using an RPC to make
    > > > > > a dynamic subscription.  Unless it's optional, then I can start
    > > > > > to understand.
    > > > >
    > > > > The client would POST to the "rpc" establish-subscripion, and in
    > > > > the reply to the POST is a "location" leaf (*) that contains a
    > > > > URI.  The client then GET this URI; this is the SSE event stream.
    > > > >
    > > > > I see.  But if a client reads the YANG as
    > > > >
    > > > >  module my-module {
    > > > >     ...
    > > > >     notification my-event {
    > > > >     }
    > > > > }
    > > > >
    > > > > and client knew base stream URI was:
    > > > >
    > > > >    /restconf/my-streams
    > > > >
    > > > > Couldn't the client then deduce SSE URL as:
    > > > >
    > > > >  GET  /restconf/my-streams/my-module:my-event?...
    > > >
    > > > No.
    > > >
    > > > With RFC 8040, the SSE URL is "stable", and if the client has read
    > > > it onve, it doesn't have to read it again (modulo some definition of
    > > > "stable").
    > > > The client
    > > > can pass a filter in the GET request for the SSE stream.
    > > >
    > > > With the mechanism in this draft, the SSE URL will vary for each
    > > > request that has a different filter (at a minimum), since the filter
    > > > is passed a parameter to "establish-subscription" and the SSE URL is
    > > > sent back in the reply to
    > > > "establish-
    > > > subscription".  With the SSE URL, the server may hide some state
    > > > that corresponds to the filter params etc.  (NOTE the word *may* in
    > > > the previous sentence.  A server could as well send back a SSE URL
    > > > with the filter parameters embedded in the
    > > > URL...)
    > > >
    > > > So in general, a client cannot assume that it can re-use a SSE URL
    > > > that was returned from a call to "establish-subscription".
    > > >
    > > > > Thereby saving a trip to server?
    > > > >
    > > > > <Eric> Yes, it would be possible to set something up to deduce the
    > > > > URL.  draft-ietf-netconf-restconf-notif doesn’t currently attempt
    > > > > this as being able to deduce the URL doesn’t stop the extra trip
    > > > > to the server.
    > > > >
    > > > > The reason it doesn’t get rid of the extra round trip has to do
    > > > > with
    > > > > HTTP2 streams.  If we are going to use HTTP2, there is still a
    > > > > need to initiate the GET (or POST) in a separate stream other than
    > > > > the one being used for RESTCONF signaling.
    > > >
    > > > I disagree.  If the client wants to have a separate HTTP/2 stream
    > > > for the notifs, then it should start such a stream.  If the client
    > > > doesn't want a separate HTTP/2 stream then that should be ok too.
    > >
    > > Per your below, once SSE is applied a stream is blocked.  For that
    > > subscription, you still need to support 'modify-subscription' and
    > > 'delete-subscription' rpcs.
    > 
    > But the solution cannot depend on the subscriber being able to send "delete-
    > subscription" on the same transport session (BTW, this requirement should be
    > clarified in the YANG definition for "delete-subscription" in SN), since this
    > would rule out HTTP/1.1.
    
    Tweaked it to "using a transport session connecting to the subscriber."
    
    > "modify-subscription" can be sent on another session, e.g., another
    > HTTP/1.1 session that is not doing the SSEs.
    > 
    > This makes me wonder... is "delete-subscription" really necessary, or is "kill-
    > subscription" sufficient?
    >
    > > Beyond that, one thing to keep an eye on is the potential for bundling
    > > of event records from different subscriptions into a single
    > > transmittable notification message.
    > 
    > How would an HTTP subscriber even request this?
    
    It wouldn't be determined by the subscriber at all.  It would be up to the publisher to determine what gets bundled or not.   If someone wants to create signaling to manipulate what could be bundled, it would require augmentations to SN's establish-subscription RPC parameters.
    
    > The subscriber sends an establish-subscription and get an URL for that
    > subscription back.
    > 
    > Then it sends another establish-subscription and gets another URL back.
    > 
    > How can a bundled message be sent when there are two different subscription
    > resources?
    
    If bundling is supported, you initially send a URL to a shared bundling resource.  Streams can be added and removed from that resource.   These possibilities are the kinds of reasons why a URI is sent back as part of the establish-subscription response (rather than it being well-known).  Further clarifications in this space will be needed as we get deeper into bundling.
    
    > > (Something like in
    > > draft-ietf-netconf-notification-messages.)  Anywhere such bundling
    > > occurs, the publisher will have to use a consistent HTTP2 stream
    > > across a set of subscriptions in order to preserve event record
    > > ordering.  In addition, all event records bundled will have to
    > > consistently use the same DSCP.
    > >
    > > Note: this is another reason why it is a good idea to use separate
    > > HTTP2 streams for RESTCONF subscription RPC interactions, and for the
    > > streaming of event records.
    > >
    > > > Compare to NETCONF.  HTTP/2 streams are similar to SSH channels.  A
    > > > NETCONF client can open a separate SSH channel for each
    > > > subscription, or it can use a single channel.  Up to the client.
    > >
    > > HTTP2 streams have far less overhead, lower setup latency, and more
    > > scalability than separate SSH channels.
    > 
    > Really?  An SSH channel is super cheap.  Why do you think that HTTP/2 streams
    > have "far less" overhead?  There can be 2^32 channels in an SSH session, and
    > 2^31 streams in a HTTP/2 session.
    > 
    > (For reference, in case it wasn't clear: RFC 4254, section 5)
    >
    > > > > Someone could write a specification which gets around this by
    > > > > forcing every RESTCONF transaction on its own HTTP2 stream.  But
    > > > > nobody has proposed such a specification.  Plus there are QoS
    > > > > scenarios where you might want multiple subscriptions on the same
    > HTTP2 stream.
    > > >
    > > > How would this work?  Once you start SSE on a stream, that stream is
    > > > "blocked", right?
    > >
    > > And per above, this is a good reason to separate RESTCONF subscription
    > > RPCs from the HTTP2 stream used for events.  (The way it could be done
    > > is have modify-subscription and delete-subscriptions come in on a
    > > different stream, impacting the HTTP2 stream where
    > > establish-subscription started things off.  Again I am not
    > > recommending this for multiple reasons.)
    > >
    > > > > So if
    > > > > someone wanted this, we would still need to refine
    > > > > draft-ietf-netconf-restconf-notif requirements with the ability to
    > > > > modify or delete an existing subscription on a different HTTP2 stream.
    > > >
    > > > Isn't this how it is supposed to work?
    > >
    > > Yes.  My point was that nobody has proposed refining the spec to
    > > forcing of each RESTCONF transaction on different HTTP2 streams.
    > >
    > > > The draft is silent on how to delete a subscription, and also on how
    > > > modify-subscription is supposed to work.
    > >
    > > Certainly these can come in on a different HTTP2 stream.
    > >
    > > > The draft is also silent on the lifetime of the result of
    > > > "establish-subscription".
    > > > With NETCONF, the subscription ends if the session ends, but in
    > > > RESTCONF there are no sessions, so how long does the subscription
    > > > live after it has been created?
    > >
    > > There are mechanisms defined.  See
    > > draft-ietf-netconf-restconf-notif, Section 3.1.
    > 
    > This section says:
    > 
    >    Where quick
    >    recognition of the loss of a publisher is required, a subscriber
    >    SHOULD connect over TLS [RFC5246], and use a TLS heartbeat [RFC6520]
    >    to track HTTP session continuity.  In the case where a TLS heartbeat
    >    is included, it should be sent just from receiver to publisher.  Loss
    >    of the heartbeat MUST result in any subscription related TCP sessions
    >    between those endpoints being torn down.  A subscriber can then
    >    attempt to re-establish.
    > 
    > This is not clear to me.
    > 
    > First it says that TLS SHOULD be used.  But with RESTCONF, TLS is already a
    > MUST.
    
    The section previously was supporting HTTP2 only configured subscriptions.   Updates are needed for points like this.
    
    > Second, it says that in the case of missed TLS heartbeats, "related TCP sessions"
    > are torn down.  What are these sessions (plural), and what exactly does
    > "related" mean?
    
    This should be singular.  Related means transiting.
     
    > Third, it says that a subscriber can re-establish.  What can it re-establish?  Re-
    > send the establish-subscription?  Just resend the POST to the subscription URL?
    
    Text needs to be updated to say "establish-subscription". 
    
    > The flow is:
    > 
    > 
    >     Subscriber                                   Publisher
    >     ----------                                   ---------
    > 
    >        | RESTCONF POST (RPC:establish-subscription)   |
    >        |--------------------------------------------->|
    >        |                          HTTP 200 OK (ID,URI)|
    >        |<---------------------------------------------|
    >        |
    >      (***)
    >        |   HTTP POST (URI)
    >        |    |--------------------------------------------->|
    >        |    |                                   HTTP 200 OK|
    > 
    > 
    > After the publisher has sent a URI for the subscription to the subscriber, there
    > is a delay (***) until the subscriber has sent the new POST.  The resource that
    > the URI points to is subscriber-specific, which means that the publisher may
    > have created some associated state.  This state will eventually have to be
    > destroyed.  When can the publisher safely destroy this state?
    
    Cases include:
    - delete-subscription rpc
    - kill-subscription rpc
    - loss of TLS heartbeat
    - inability to deliver sequential sets of notification messages
         "For connectionless or stateless transports like
          HTTP, a lack of receipt acknowledgment of a sequential set of
          notification messages and/or keep-alives can be used to trigger a
          termination of a dynamic subscription. "   (per SN 1.3)
    - " A publisher MAY terminate a dynamic subscription at any time."   (per SN 1.3)
    
    > For example, what happens if the new POST never arrives?
    
    Then it will fail to deliver a sequential set of notification messages
    
    > If the subscriber
    > loses its transport session, can it send a new POST to the same URL?  Can it
    > send multiple POSTs to the same URL?
    
    Agree the text can be enhanced to cover your points above.  I had been assuming this would not be the case.  But I haven't thought deeply about it.
     
    Eric
    
    > > and
    > > draft-ietf-netconf-subscribed-notifications, sections 1.3 & 5.3.
    > 
    > I didn't find the answers to the questions above in these sections.
    > 
    > 
    > /martin
    > 
    > 
    > >
    > > Eric
    > >
    > > > Note that the approach in RFC 8040 is doesn't have this issue.
    > > >
    > > >
    > > > /martin
    > > >
    > > >
    > > > > This could be done of course, but someone would need to drive the
    > > > > definition.
    > > > >
    > > > > Eric
    > > > >
    > > > >
    > > > > Would it be wrong for a client to make any assumptions?  I realize
    > > > > this ASSUMEs a transport protocol, but could/would there be a
    > > > > one-time, capability check the client could make to test for
    > > > > available transports?
    > > > >
    > > > > I really appreciate your time explaining, I have been following
    > > > > the emails best I could but I think some basic things are still
    > > > > eluding me.
    > > > >