[Netconf] SSE and HTTP/2 in restcon-notif [Was: activate-configured-subecription - WGLC subscribed-notifications-16]

[Martin Bjorklund <[e-mail]>]

Hi,

[subject change; this thread is now about SSE and HTTP/2]

"Eric Voit (evoit)" <[e-mail]> wrote:
> > From: Martin Bjorklund, September 19, 2018 2:38 AM
> > 
> > Hi,
> > 
> > "Eric Voit (evoit)" <[e-mail]> wrote:
> > > Hi Douglas,
> > >
> > > From: Douglas Hubler, September 18, 2018 5:35 PM
> > >
> > >
> > > On Tue, Sep 18, 2018 at 3:40 PM Martin Bjorklund
> > > <[e-mail]<mailto:[e-mail]>> wrote:
> > > So it seems you support my point to make this draft use SSE.  Then if
> > > you use HTTP/2 you (the client) can simply create one HTTP/2 stream
> > > per notification stream.  This just falls out for free with HTTP/2.
> > >
> > > yes, with caveat that I haven't attempted actual implementation yet.
> > >
> > > <Eric> Agree that SSE over HTTP2 should work.  However to me using SSE
> > > here seems redundant as HTTP2 provides the necessary mechanisms.
> > 
> > Eric, can you explain how this would work w/o SSE?  Which mechanism in
> > HTTP/2 are you referring to?
> 
> After an HTTP2 OK, the HTTP2 server has the ability to forward an
> unbounded number of HTTP2 Data Frames on the HTTP2 stream.

But it must still adhere to the normal HTTP semantics.  If there was a
request, it needs to send the response.  It can split the response
into several DATA frames of course.  But it cannot start to send
"other stuff" instead of the response, which figure 1 in
restconf-notif-07 possibly implies.

> The stream
> is not considered complete until an END_STREAM is passed.  (Note this
> is independent of anything regarding HTTP2's PUSH_PROMISE.)
> 
> You can see an example flow in Figure 1 of
> draft-ietf-netconf-restconf-notif.  At IETF 97 we asked for some
> Feedback on this proposal.
> http://www.voit.org/IETF/Subscriptions-NETCONF-IETF97.ppt (see slide
> 14).
> But I am not sure how much effort the WG should place here.  The work
> here is several years old, and the market has moved substantially
> since then.

So do we agree that the restconf-notif draft will be updated to use
SSE regardless of HTTP version?  If there is any text at all about
HTTP/2, it will be an "implementation note" where it is recommended to
use a separate stream per subscription?

> > > But
> > > I won’t get in the way if people really do want to go in this
> > > direction.
> > >
> > > > I don't understand the details surrounding using an RPC to make a
> > > > dynamic subscription.  Unless it's optional, then I can start to
> > > > understand.
> > >
> > > The client would POST to the "rpc" establish-subscripion, and in the
> > > reply to the POST is a "location" leaf (*) that contains a URI.  The
> > > client then GET this URI; this is the SSE event stream.
> > >
> > > I see.  But if a client reads the YANG as
> > >
> > >  module my-module {
> > >     ...
> > >     notification my-event {
> > >     }
> > > }
> > >
> > > and client knew base stream URI was:
> > >
> > >    /restconf/my-streams
> > >
> > > Couldn't the client then deduce SSE URL as:
> > >
> > >  GET  /restconf/my-streams/my-module:my-event?...
> > 
> > No.
> > 
> > With RFC 8040, the SSE URL is "stable", and if the client has read it
> > onve, it
> > doesn't have to read it again (modulo some definition of "stable").
> > The client
> > can pass a filter in the GET request for the SSE stream.
> > 
> > With the mechanism in this draft, the SSE URL will vary for each
> > request that
> > has a different filter (at a minimum), since the filter is passed a
> > parameter to
> > "establish-subscription" and the SSE URL is sent back in the reply to
> > "establish-
> > subscription".  With the SSE URL, the server may hide some state that
> > corresponds to the filter params etc.  (NOTE the word *may* in the
> > previous
> > sentence.  A server could as well send back a SSE URL with the filter
> > parameters embedded in the
> > URL...)
> > 
> > So in general, a client cannot assume that it can re-use a SSE URL
> > that was
> > returned from a call to "establish-subscription".
> > 
> > > Thereby saving a trip to server?
> > >
> > > <Eric> Yes, it would be possible to set something up to deduce the
> > > URL.  draft-ietf-netconf-restconf-notif doesn’t currently attempt this
> > > as being able to deduce the URL doesn’t stop the extra trip to the
> > > server.
> > >
> > > The reason it doesn’t get rid of the extra round trip has to do with
> > > HTTP2 streams.  If we are going to use HTTP2, there is still a need to
> > > initiate the GET (or POST) in a separate stream other than the one
> > > being used for RESTCONF signaling.
> > 
> > I disagree.  If the client wants to have a separate HTTP/2 stream for
> > the notifs,
> > then it should start such a stream.  If the client doesn't want a
> > separate HTTP/2
> > stream then that should be ok too.
> 
> Per your below, once SSE is applied a stream is blocked.  For that
> subscription, you still need to support 'modify-subscription' and
> 'delete-subscription' rpcs.

But the solution cannot depend on the subscriber being able to send
"delete-subscription" on the same transport session (BTW, this
requirement should be clarified in the YANG definition for
"delete-subscription" in SN), since this would rule out HTTP/1.1.

"modify-subscription" can be sent on another session, e.g., another
HTTP/1.1 session that is not doing the SSEs.

This makes me wonder... is "delete-subscription" really necessary, or
is "kill-subscription" sufficient?

> Beyond that, one thing to keep an eye on is the potential for bundling
> of event records from different subscriptions into a single
> transmittable notification message.

How would an HTTP subscriber even request this?

The subscriber sends an establish-subscription and get an URL for that
subscription back.

Then it sends another establish-subscription and gets another URL
back.

How can a bundled message be sent when there are two different
subscription resources?


> (Something like in
> draft-ietf-netconf-notification-messages.)  Anywhere such bundling
> occurs, the publisher will have to use a consistent HTTP2 stream
> across a set of subscriptions in order to preserve event record
> ordering.  In addition, all event records bundled will have to
> consistently use the same DSCP.
> 
> Note: this is another reason why it is a good idea to use separate
> HTTP2 streams for RESTCONF subscription RPC interactions, and for the
> streaming of event records.
> 
> > Compare to NETCONF.  HTTP/2 streams are similar to SSH channels.  A
> > NETCONF client can open a separate SSH channel for each subscription,
> > or it
> > can use a single channel.  Up to the client.
> 
> HTTP2 streams have far less overhead, lower setup latency, and more
> scalability than separate SSH channels.

Really?  An SSH channel is super cheap.  Why do you think that HTTP/2
streams have "far less" overhead?  There can be 2^32 channels in an
SSH session, and 2^31 streams in a HTTP/2 session.

(For reference, in case it wasn't clear: RFC 4254, section 5)

> > > Someone could write a specification which gets around this by forcing
> > > every RESTCONF transaction on its own HTTP2 stream.  But nobody has
> > > proposed such a specification.  Plus there are QoS scenarios where you
> > > might want multiple subscriptions on the same HTTP2 stream.
> > 
> > How would this work?  Once you start SSE on a stream, that stream is
> > "blocked", right?
> 
> And per above, this is a good reason to separate RESTCONF subscription
> RPCs from the HTTP2 stream used for events.  (The way it could be done
> is have modify-subscription and delete-subscriptions come in on a
> different stream, impacting the HTTP2 stream where
> establish-subscription started things off.  Again I am not
> recommending this for multiple reasons.)
>  
> > > So if
> > > someone wanted this, we would still need to refine
> > > draft-ietf-netconf-restconf-notif requirements with the ability to
> > > modify or delete an existing subscription on a different HTTP2 stream.
> > 
> > Isn't this how it is supposed to work? 
> 
> Yes.  My point was that nobody has proposed refining the spec to
> forcing of each RESTCONF transaction on different HTTP2 streams.
> 
> > The draft is silent on how to delete a
> > subscription, and also on how modify-subscription is supposed to work.
> 
> Certainly these can come in on a different HTTP2 stream.
>  
> > The draft is also silent on the lifetime of the result of
> > "establish-subscription".
> > With NETCONF, the subscription ends if the session ends, but in
> > RESTCONF
> > there are no sessions, so how long does the subscription live after it
> > has been
> > created?
> 
> There are mechanisms defined.  See 
> draft-ietf-netconf-restconf-notif, Section 3.1. 

This section says:

   Where quick
   recognition of the loss of a publisher is required, a subscriber
   SHOULD connect over TLS [RFC5246], and use a TLS heartbeat [RFC6520]
   to track HTTP session continuity.  In the case where a TLS heartbeat
   is included, it should be sent just from receiver to publisher.  Loss
   of the heartbeat MUST result in any subscription related TCP sessions
   between those endpoints being torn down.  A subscriber can then
   attempt to re-establish.

This is not clear to me.

First it says that TLS SHOULD be used.  But with RESTCONF, TLS is
already a MUST.

Second, it says that in the case of missed TLS heartbeats, "related
TCP sessions" are torn down.  What are these sessions (plural), and
what exactly does "related" mean?

Third, it says that a subscriber can re-establish.  What can it
re-establish?  Re-send the establish-subscription?  Just resend the
POST to the subscription URL?

The flow is:


    Subscriber                                   Publisher
    ----------                                   ---------

       | RESTCONF POST (RPC:establish-subscription)   |
       |--------------------------------------------->|
       |                          HTTP 200 OK (ID,URI)|
       |<---------------------------------------------|
       |                                              
     (***)
       |   HTTP POST (URI)
       |    |--------------------------------------------->|
       |    |                                   HTTP 200 OK|


After the publisher has sent a URI for the subscription to the
subscriber, there is a delay (***) until the subscriber has sent the
new POST.  The resource that the URI points to is subscriber-specific,
which means that the publisher may have created some associated
state.  This state will eventually have to be destroyed.  When can the
publisher safely destroy this state?

For example, what happens if the new POST never arrives?  If the
subscriber loses its transport session, can it send a new POST to the
same URL?  Can it send multiple POSTs to the same URL?

> and
> draft-ietf-netconf-subscribed-notifications, sections 1.3 & 5.3.

I didn't find the answers to the questions above in these sections.


/martin


> 
> Eric
> 
> > Note that the approach in RFC 8040 is doesn't have this issue.
> > 
> > 
> > /martin
> > 
> > 
> > > This could be done of course, but someone would need to drive the
> > > definition.
> > >
> > > Eric
> > >
> > >
> > > Would it be wrong for a client to make any assumptions?  I realize
> > > this ASSUMEs a transport protocol, but could/would there be a
> > > one-time, capability check the client could make to test for available
> > > transports?
> > >
> > > I really appreciate your time explaining, I have been following the
> > > emails best I could but I think some basic things are still eluding
> > > me.
> > >