RE: I-D Action: draft-han-6man-in-band-signaling-for-transport-qos-00.txt

[Lin Han <[e-mail]>]

Hi, Brain

Thanks for the comments, see my reply in line with [LH]

Lin

-----Original Message-----
From: ipv6 [mailto:[e-mail]] On Behalf Of Brian E Carpenter
Sent: Friday, October 13, 2017 7:16 PM
To: 6man <[e-mail]>
Subject: Re: I-D Action: draft-han-6man-in-band-signaling-for-transport-qos-00.txt

Hi,

RSVP in IPv6 [RFC2205] uses packets with a Router Alert hop-by-hop option. So although it is not embedded in application packets, it is otherwise very similar: in band, following the same path as application packets, and triggering hop-by-hop processing in the same routers.
[LH] The difference of this idea with RSVP is that RSVP needs a separate protocol to be running, but this in-band signaling is always carried in a packet of upper layer protocols (TCP, UDP, etc) and does not need to run a new protocol.
RSVP for Integrated Services was a complete failure. So my first questions are:

1: What is the authors' full analysis of why RSVP failed?
[LH] The complexity of the RSVP makes its scalability an issue. Specifically, RSVP needs every router run the protocol and maintain the state for every RSVP session on the router; To maintain the RSVP soft-state, each source of RSVP session has to periodically (~30sec and configurable) multicast the PATH message with Tspec to whole network and the interested receiver has to return the RESV with Rspec to root of the mcast tree; All these control plane work make the RSVP session supported on a router limited by the controller CPU processing power and memory size. 
2: How does the proposed solution avoid the same failure?
[LH] The in-band signaling proposal is based on the following ideas and assumptions
1) Network processor widely used in latest network device are capable of not only doing table lookup and switching but also doing general packet processing and other works which was only available by controller CPU (in old generation network device).
2) If we simplify the QoS protocol work in request, reserve and  state keeping to be fitting into the capability of a NPU, we can greatly reduce the complexity of what RSVP or IntServ tries to do to provision the flow-level QoS and maintain the soft-sate for each flow with QoS.
3) Following are some details, 
3.1) We simplify the Tsepc and Rspec to explicit bandwidth and latency request, and rely on the NPU to program the hardware to satisfy the QoS requirement, All QoS related traffic shaping, scheduling and queuing are hardware dependent and should be irrelevant to the control plane; 
3.2) Instead of using two directional message (in RSVP) to finish the reservation for a session, this propose only uses one directional in-band signal messaging to program the hardware for the QoS for the flow on same direction. The reason we can do such simplification is because we don't have to have the complicated feature like the merging of request in RSVP. The proposed in-band signaling only supports unicast, but RSVP was designed to support both unicast and multicast, so, RSVP needs merging feature, and needs two directional messages to finish the resource reservation. The new idea will also be fitting to the IP nature that IP path is not symmetric in many scenarios. As comparison, to use RSVP for asymmetric path is a problem. 
3.3) The soft-state of each flow's QoS channel is maintained by the data packet with the same flow identification. The QoS state programmed in the hardware will be erased and the resource will be released if there is no data packet for a programmed QoS state for a period of time. A host can simply send zero-size data packet to refresh the QoS state for the flow if there is no application data. This could completely remove the state refreshing mechanism by a separate control protocol like RSVP which was a big overhead for CPU to be scalable.
3.4) The NPU will use the pre-defined flow identification to forward a packet with QoS guarantee, and the forwarding state will be updated at each hop-by-hop aware node and feed back to the source host. Source host can easily decide if it needs to re-program the QoS for a flow if there is some failure (link, node, topology change, etc)
3.5) In summary, this approach will use a simplified mechanism (compared with RSVP) to provision QoS, and it also moves the QoS control complexity and processing burden from centralized controller CPU to distributed NPU of line card. Normally there are more NPU than CPU in a router, so, the redistribution of the complexity will greatly improve the scalability. 
3.6) Here are some hypothetical and experimental data in our POC: A typical Huawei router can only support about 5k RSVP sessions (No data from other vendors, but should be in same scale hopefully -:)). In POC, we apply the new approach to a low/middle class access router (ATN9xxx, the NPU was designed more than 5 year ago), each NPU can handle about 1k QoS session (The limit is majorly caused by the memory for queue and scheduler allocation), then whole system can support (num. of NPU) x 1k QoS session. Depending on port configuration, a router can easily have more than 10 NPU. 
4) Since we only did the experiment with our own proprietary NPU, and not sure if other company's NPUs have the similar capability as above, we put the document as "informational" (not sure if "experiment" is better, but can be changed). 

In particular, I do not understand your argument about scalability. While that was an obvious issue for RSVP, it could be fixed in the way you describe: by limiting the number of using applications. The fundamental scaling issue for RSVP was the number of session state items per router, and your solution needs exactly the same number of session state items.
So why will you succeed where RSVP failed?

> End user application cannot directly use DiffServ.

Not true. The advanced socket API allows the transport user to set the DSCP via IPV6_TCLASS [RFC3542]. It's been done for many years, for example in IP telephones.
[LH] agreed, will correct the document.

What is lacking is a host software package to make this API feature useful, but any QoS solution needs that. In your terminology, that is the application interface to the closed-loop control system.

Note that RTCWEB has chosen to support Diffserv [https://datatracker.ietf.org/doc/draft-ietf-tsvwg-rtcweb-qos/]

> 3.2.  IP In-band signaling
> 
>    There is no definition for IP in-band signaling.

Not true. That's exactly what DiffServ is. You could argue that it's too limited, but after 19 years we have not exhausted the DSCP code space defined in [RFC2474]. (As far as I know, nobody has yet combined use of the DSCP with the Flow Label. There is scope for creativity there.)

[LH] The difference of this approach with DiffServ is the granularity of QoS. DiffServ was very successful. As I stated in the document, some application needs much more bandwidth (than others) and we need a way to specify  the QoS requirement explicitly for those flows. 

>    Flow level In-band Signaling
>       The control message and data packet share the same flow
>       identification.  The flow identification could be 5 tuples ...

I agree with the comment that the IPv6 flow label is a much better choice. It is already well defined [RFC6437] and widely supported by operating systems, it works for any transport protcol, works for encrypted packets, and works for packets with multiple extension headers. If you want per-flow granularity, the flow label is perfect.

[LH] Agreed, 

>  The HbH-EH may be examined and processed by the nodes that are
>    explicitly configured to do so [RFC8200]

which also says in section 4.8:

"  New hop-by-hop options are not recommended because nodes may be
   configured to ignore the Hop-by-Hop Options header, drop packets
   containing a Hop-by-Hop Options header, or assign packets containing
   a Hop-by-Hop Options header to a slow processing path.  Designers
   considering defining new hop-by-hop options need to be aware of this
   likely behavior.  There has to be a very clear justification why any
   new hop-by-hop option is needed before it is standardized."

I believe you need much more explanation of why we should ignore this recommendation.

[LH] The propose does not introduce new hop-by-hop option header, we only introduce new option carried in the existing hop-by-hop option header, it is the same as other defined options carried by hop-by-hop option header, such as "Jumbo Payload", "RPL Option", "Quick-Start", etc. I think the IPv6 document sometime uses the general "hop-by-hop option" word as the " hop-by-hop option extension header" really confuse people. I believe the above document does not recommend to introduce new hop-by-hop option extension header, but not stop introducing new options carried in the current hop-by-hop option header.  
I will check with IPv6 author for clarification.
Below are existing IPv6 hop-by-hop and destination options defined (HbH: hop-by-hop option header; Dst: Destination extension header)

0x00	00	0	00000	Pad1	[[IPV6]]	- HbH
0x01	00	0	00001	PadN	[[IPV6]]	- HbH
0xC2	11	0	00010	Jumbo Payload	[RFC2675]	- HbH
0x63	01	1	00011	RPL Option	[RFC6553]	- HbH
0x04	00	0	00100	Tunnel Encapsulation Limit	[RFC2473]	- Dst
0x05	00	0	00101	Router Alert	[RFC2711]	- HbH
0x26	00	1	00110	Quick-Start	[RFC4782][RFC Errata 2034]	- HbH
0x07	00	0	00111	CALIPSO	[RFC5570]	- HbH
0x08	00	0	01000	SMF_DPD	[RFC6621]	- HbH
0xC9	11	0	01001	Home Address	[RFC6275]	- Home Addr Dst
0x8A	10	0	01010	Endpoint Identification (DEPRECATED)	[[CHARLES LYNN]]
0x8B	10	0	01011	ILNP Nonce	[RFC6744]	- Dst
0x8C	10	0	01100	Line-Identification Option	[RFC6788]	- Dst
0x4D	01	0	01101	Deprecated	[RFC7731]
0x6D	01	1	01101	MPL Option	[RFC7731]	- HbH
0xEE	11	1	01110	IP_DFF	[RFC6971]	-HbH
0x0F	00	0	01111	Performance and Diagnostic Metrics (PDM)	[RFC8250]	- Dst

> 8.  Security Considerations
> 
>    There is no security issue introduced by this document

I think this section needs more work.
[LH] Yes, will add more

Regards
   Brian Carpenter

On 12/10/2017 07:05, [e-mail] wrote:
> 
> A New Internet-Draft is available from the on-line Internet-Drafts directories.
> 
> 
>         Title           : IPv6 in-band signaling for the support of transport with QoS
>         Authors         : Lin Han
>                           Guoping Li
>                           Boyan Tu
>                           Xuefei Tan
>                           Frank Li
>                           Richard Li
>                           Jeff Tantsura
>                           Kevin Smith
> 	Filename        : draft-han-6man-in-band-signaling-for-transport-qos-00.txt
> 	Pages           : 40
> 	Date            : 2017-10-11
> 
> Abstract:
>    This document proposes a method to support the IP transport service
>    that could guarantee a certain level of service quality in bandwidth
>    and latency.  The new transport service is fine-grained and could
>    apply to individual or aggregated TCP/UDP flow(s).
> 
> 
> The IETF datatracker status page for this draft is:
> https://datatracker.ietf.org/doc/draft-han-6man-in-band-signaling-for-
> transport-qos/
> 
> There are also htmlized versions available at:
> https://tools.ietf.org/html/draft-han-6man-in-band-signaling-for-trans
> port-qos-00
> https://datatracker.ietf.org/doc/html/draft-han-6man-in-band-signaling
> -for-transport-qos-00
> 
> 
> Please note that it may take a couple of minutes from the time of 
> submission until the htmlized version and diff are available at tools.ietf.org.
> 
> Internet-Drafts are also available by anonymous FTP at:
> ftp://ftp.ietf.org/internet-drafts/
> 
> _______________________________________________
> I-D-Announce mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/i-d-announce
> Internet-Draft directories: http://www.ietf.org/shadow.html or 
> ftp://ftp.ietf.org/ietf/1shadow-sites.txt
> 

--------------------------------------------------------------------
IETF IPv6 working group mailing list
[e-mail]
Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
--------------------------------------------------------------------