RE: Idea for packet numbers

["Swindells, Thomas (Nokia - GB/Cambridge, UK)" <[e-mail]>]

> -----Original Message-----
> From: Dmitri Tikhonov [mailto:[e-mail]]
> Sent: 25 July 2017 17:27
> To: Swindells, Thomas (Nokia - GB/Cambridge, UK)
> <[e-mail]>
> Cc: Mikkel Fahnøe Jørgensen <[e-mail]>; Magnus Westerlund
> <[e-mail]>; IETF QUIC WG <[e-mail]>
> Subject: Re: Idea for packet numbers
> 
> On Tue, Jul 25, 2017 at 02:28:52PM +0000, Swindells, Thomas (Nokia -
> GB/Cambridge, UK) wrote:
> > 4 wasn't meant to imply how you queue the packet on the NIC - I was
> > assuming via a kernel API call into normal udp send buffers, but both
> > methods are valid.  You are right though there is the potential for
> > the queuing operation to be blocked/rejected.
> 
> Since we were talking about implementation, I wanted to point out that packet
> number allocation is more complicated in practice.  I realize this is not what
> you meant. :)
> 
> > As I understand it the packet number is used in the encryption
> > process, so changing a packets number is a non-trivial operation.  A
> > fast path route for queuing acks makes sense, again however the only
> > real way
>                                                         ^^^^^^^^^^^^^
> 
> By this, do you mean "without encrypting data twice?"
Yes - re-encrypting can't really be considered a 'trivial' operation
 
> > to do that would be to give the ack packet a higher packet number but
> > then send it before lower marked packets - even potentially if they do
> > also contain ack frames.
> 
> That is a good approach if it does not break packet number derivation.
> I am not sure whether it can...  The draft says:
> 
>   " A packet number is decoded by finding the packet number
>   " value that is closest to the next expected packet.
> 
> Thus, after receiving ACK packet with large gap, which is the next expected
> packet number from the point of view of the peer?
The spec says that you should give plenty of room in the number space, so as long as you haven't reached half your number space across that period this should be extremely rare or perhaps impossible. 
The worst case is that if you do violate it then either you may be requested to retransmit some packets, or you just send the original packets first and then the ack.
 
> > For RST_STREAM I think it is permitted that currently queued packets
> > are still sent, but if they only contain that streams packets then it
> > is a beneficial optimization if they weren't sent (leaving packet
> > number gaps).
> 
> Current draft says:
> 
>   " An endpoint that receives a RST_STREAM frame (and which
>   " has not sent a FIN or a RST_STREAM) MUST immediately
>   " respond with a RST_STREAM frame, and MUST NOT send any
>   " more data on the stream.
> 
> I suppose one could interpret "send" to mean "enqueue," but I would err on
> the side of caution.
With a basic integration between an application and quic stack the api would likely just be the application writing a load of data onto a stream which then gets multiplexed into a quic connection.
In this case the application view would be once it has written it to the stream it is being 'sent' - in the same way the assumption is held for TCP transmission. 
Packetization and enqueuing for transmission would therefore be considered part of the sending process and with that API it would seem to be fair to send queued data but not allow new writes from the application.
Assuming no re-encryption/packetization, that is pretty much all you can do if packets contain data from multiple stream frames anyway. 
The "immediately respond with a RST_STREAM" perhaps then needs to be read as: "immediately, after any pending packets have been transmitted, such that no new packets with data from that stream are generated with a higher packet number than the RST_STREAM response frame."?

Thomas