Re: [hybi] Extensibility mechanisms?

[Mike Belshe <[e-mail]>]

I have some possibly noobie questions.  I apologize if these have been
answered before:

a) In 3.1.2, what is the structure for the length?  I assume its a 4byte
integer?
b) It would be nice if every frame had a length in a fixed position within
the frame. Having the length in different places for different frame types
is just mechanically annoying.  With exception of the UTF-8 frame, it looks
like bytes 2-5 of any frame are the length?
c) Is the fragment really needed?
d) although not stated, I assume that if the first bit of a frame is 0 (e.g.
UTF-8), that the remaining 7 bits of the frame block header must be 0?  Do
the batch/fragment/type fields have meaning for UTF-8 frames?
e) 3.2 I like the channels. More details are needed on the channel lifetime
& state.  I assume you're looking at a half-closed type of thing.

My biggest complaint:
The logical unit of operation for a receiver of data over BWTP appears to be
a "frame batch", not a "frame".   Because frame types are as granular as
"header name", "channel id", etc, they are really more like field types
within a batch than a frame-id.  By having such fine-grained identifiers,
doesn't it allow for creation of very strange frames? It also creates the
opportunity for each implementation to encode equivalent frames in very
different ways.  In general, it is nice if equivalent frames end up being
the same sequence of bytes.

About layering of HTTP over BWTP:
* The example in 3.3 doesn't close the channel.
* Should HTTP over BWTP use a single channel?  Or should many channels be
used?
* The example in 3.3 - how does the receiver determine the end of a request
block?  Is it required to be a single batch?  If so, how does a large post
work?  Is this where a fragment would work.

One last question, and I hope this won't sound like pride, because it is not
- I just want to understand. Why not use SPDY for BWTP?  BWTP is definitely
aimed at the same goals as SPDY, but SPDY is much further along.  Is there
something where SPDY fails which makes BWTP necessary?  Aside from syntax,
there are no features in BWTP not already present in SPDY?  By contrast,
there are several features in SPDY not available in BWTP.  I propose we
understand such differences and unify these projects.

It seems odd to me that first, BWTP defines itself as a "sub-protocol" for
WebSockets, but then section 3.4 defines how to run WebSockets over BWTP :-)
 It sounds like indecision.

Mike



On Fri, Apr 16, 2010 at 8:29 AM, Greg Wilkins <[e-mail]> wrote:

>
> All,
>
> Julian corrected my xml2rfc goofup.   Here is a more
> readable version of the BWTP draft (still not submitted yet).
>
> cheers
>
>
> Greg Wilkins wrote:
> > Martin,
> >
> > here is my strawman BWTP extension.
> >
> > The key need is for it to be able to use the spare bits in
> > the frame type byte.
> >
> > Note, that xml2rfc is giving me grief, so I've attached
> > the XML and a horrid PDF dump.   As soon as I get a ms
> > free I'm sure I'll find out what stupid thing I've done
> > to the xml.
> >
> > Note that this is has not yet been submitted as a draft,
> > but I'm happy to do so when I've got the format right.
> > Remember that BWTP is little more than a thought
> > experiment - but I think a worthwhile one to test out
> > these concepts.
> >
> > cheers
> >
>
> Network Working GroupG. WilkinsInternet-DraftWebtideIntended status:
> InformationalP. HintjensExpires: September 13, 2010March 12, 2010
>
> Bidirectional Web Transfer Protocol -- BWTP/1.0
> draft-wilkins-hybi-bwtp-01
> Abstract <#1280740bc6667cea_rfc.abstract>
>
> The Bidirectional Web Transfer Protocol (BWTP) is an proposed websocket
> subprotocol that will provide richer transport semantics.
> Status of this Memo <#1280740bc6667cea_rfc.status>
>
> This Internet-Draft is submitted to IETF in full conformance with the
> provisions of BCP 78 and BCP 79.
>
> Internet-Drafts are working documents of the Internet Engineering Task
> Force (IETF), its areas, and its working groups. Note that other groups may
> also distribute working documents as Internet-Drafts.
>
> Internet-Drafts are draft documents valid for a maximum of six months and
> may be updated, replaced, or obsoleted by other documents at any time. It is
> inappropriate to use Internet-Drafts as reference material or to cite them
> other than as “work in progress”.
>
> The list of current Internet-Drafts can be accessed at
> http://www.ietf.org/ietf/1id-abstracts.txt.
>
> The list of Internet-Draft Shadow Directories can be accessed at
> http://www.ietf.org/shadow.html.
>
> This Internet-Draft will expire in September 13, 2010.
> Copyright Notice <#1280740bc6667cea_rfc.copyrightnotice>
>
> Copyright © 2010 IETF Trust and the persons identified as the document
> authors. All rights reserved.
>
> This document is subject to BCP 78 and the IETF Trust's Legal Provisions
> Relating to IETF Documents (http://trustee.ietf.org/license-info) in
> effect on the date of publication of this document. Please review these
> documents carefully, as they describe your rights and restrictions with
> respect to this document. Code Components extracted from this document must
> include Simplified BSD License text as described in Section 4.e of the Trust
> Legal Provisions and are provided without warranty as described in the BSD
> License.
> ------------------------------
> Table of Contents <#1280740bc6667cea_rfc.toc>
>
>    - 1.   Introduction <#1280740bc6667cea_Intro>
>       - 1.1   Purpose <#1280740bc6667cea_Purpose>
>    - 2.   Terminology <#1280740bc6667cea_rfc.section.2>
>    - 3.   Overall Operation <#1280740bc6667cea_overall>
>       - 3.1   Frame Types <#1280740bc6667cea_rfc.section.3.1>
>          - 3.1.1   Data <#1280740bc6667cea_rfc.section.3.1.1>
>          - 3.1.2   Fragmentation <#1280740bc6667cea_rfc.section.3.1.2>
>          - 3.1.3   Message Meta Data <#1280740bc6667cea_rfc.section.3.1.3>
>       - 3.2   Channels <#1280740bc6667cea_rfc.section.3.2>
>       - 3.3   HTTP over BWTP <#1280740bc6667cea_rfc.section.3.3>
>       - 3.4   WebSocket over BWTP <#1280740bc6667cea_rfc.section.3.4>
>    - 4.   new section <#1280740bc6667cea_rfc.section.4>
>    - 5.   Normative References <#1280740bc6667cea_rfc.references>
>    - Authors' Addresses <#1280740bc6667cea_rfc.authors>
>
> 1. <#1280740bc6667cea_rfc.section.1> Introduction<#1280740bc6667cea_Intro>
> 1.1 <#1280740bc6667cea_rfc.section.1.1> Purpose<#1280740bc6667cea_Purpose>
>
> This IETF draft is intended for further discussion in the [e-mail] list for "Bidirectional communication for hypertext".
>
> The Hypertext Transfer Protocol (HTTP) is an application-level protocol for
> distributed, collaborative, hypermedia information systems. HTTP has been in
> use by the World-Wide Web global information initiative since 1990, but it
> is intrinsically a request/response protocol and cannot well service the
> growing use-case for data to be pushed from a server to a client.
>
> The websocket protocol is a message framing protocol that can be used to
> upgrade a HTTP/1.1 connection to be able to exhange bidirectional messages.
> The protocol is currently being standardized by the IETF HyBi working group.
>
> The Bidirectional Web Transfer Protocol (BWTP) is an proposed websocket
> subprotocol that will provide richer transport semantics. Some specific
> use-cases for BWTP are:
>
>    - A browser providing the websocket API that wishes to transparently
>    provide features such as compression, multiplexing. The javascript using the
>    the weebsocket API would be unaware that any additional transport features
>    are being used.
>    - A browser could use BWTP to provide an enhanced websocket API that
>    includes support features such as binary content and meta data.
>    - A websocket intermediary in cooperation with a server, could use BWTP
>    to transparently enhance websocket connections received from clients with
>    features such as connection aggregation, SSL offload, load balancing.
>    - A non browser client that wishes to use websocket framing protocol to
>    transit the internet, could use BWTP to achieve a richer messaging semantic
>    than that provided by the javascript API.
>    - A browser and server could use BWTP to transport HTTP requests and
>    responses over a websocket connection. Not only would this allow a single
>    connection to be used, but also features (many inspired by SPDY) such as
>    header compression, multiple responses and responses sent in anticipation.
>
> This draft of BWTP differs significantly from the first draft of BWTP, in
> that it uses websocket and length encoded framing. However the semantics and
> capabilities expressed in the first draft are retained as well as new
> features inspired SPDY have been incorporated.
> 2. <#1280740bc6667cea_rfc.section.2> Terminology
>
> In this document, the key words "MUST", "MUST NOT", "REQUIRED", "SHALL",
> "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED",
> "MAY", and "OPTIONAL" are to be interpreted as described in BCP 14, RFC
> 2119 <#1280740bc6667cea_RFC2119> [RFC2119] and indicate requirement levels
> for compliant implementations.
>
> An implementation is not compliant if it fails to satisfy one or more of
> the MUST or REQUIRED level requirements for the protocols it implements. An
> implementation that satisfies all the MUST or REQUIRED level and all the
> SHOULD level requirements for its protocols is said to be "unconditionally
> compliant"; one that satisfies all the MUST level requirements but not all
> the SHOULD level requirements for its protocols is said to be "conditionally
> compliant".
>
> This specification uses a number of terms to refer to the roles played by
> participants in, and objects of, the HTTP or BWTP communication.
>
> In this document, as short hand notation is used to represent websocket
> frames. Each websocket frame is delimited by the characters '{' and '}' and
> the character '|' is used to separate elements within the frame. The
> contents of each element is informally described and the context and
> surrounding text may need to be consulted for interpretations. For example
> the string "Hello World" sent as a WebSocket UTF-8 frame could be expressed
> as {0x00|Hello World|0xFF}. The same string sent as a length encoded binary
> frame can be represented as {0x80|length|Hello World}. The characters '['
> and ']' are used to enclose frames that are sent as part of the same batch.
> connection:3. <#1280740bc6667cea_rfc.section.3> Overall Operation<#1280740bc6667cea_overall>
>
> The BWTP protocol is designed to be a subprotocol of websocket and makes
> use of the unused bits of the frame type byte to send meta data and channel
> control frames. This section provides a quick overview of the operation of
> the protocol and it's main features.
>
> The following diagram shows how the User Agent (UA) initiates the BWTP
> connection by sending an upgrade request (an HTTP request) to the Origin
> Server (OS), over a connection (v):
>
>   UA -------------------v------------------- OS
>      WS upgrade request ------------------->
>      <-------------- 101 WS upgrade response
>      <------------ BWTP Frames ------------>
>
> Following the upgrade request-response, both server and client can
> immediately start to send each other frames. The inbound and out- bound
> frames streams are independent and asynchronous (frames do not have
> responses).
> 3.1 <#1280740bc6667cea_rfc.section.3.1> Frame Types
>
> BWTP messages are batches of one or more websocket frames. The websocket
> frame type byte is used to indicate frames that represent semantic
> boundaries and/or batch boundaries:
>
> +------------+-------------+----------+-----+-----+-----+-----+-----+
> | 7          | 6           | 5        | 4   | 3   | 2   | 1   | 0   |
> +------------+-------------+----------+-----+-----+-----+-----+-----+
> | Framing:   | Fragment:   | Batch:   |     | BWTP Frame Type       |
> |  1=binary  |  1=Fragment |  1=Batch |     | xxxx=Meta Data        |
> |  0=UTF-8   |  0=Complete |  0=End   |     | 00000=Data            |
> +------------+-------------+----------+-----+-----------------------+
>
> The highorder bit of the WebSocket frame type is defined by the websocket
> protocol to determine if UTF-8 framing or length encoded binary framing is
> used. A BWTP implementation MUST accept both framing types, but it SHOULD
> only send frames that use the length encoded binary framing.
>
> The 2nd highest order bit of the WebSocket frame type is used by BWTP to
> indicate a frame that has been fragmented into multiple frames. The receiver
> should aggregate fragment frames until a non fragmented frame is received.
> It is hoped that the WebSocket specification will adopt the definition of
> this bit.
>
> The 3rd highest order bit of the WebSocket frame type is use by BWTP to
> indicate batches of frames. A frame with the batch bit set does not need to
> be transmitted towards the destination until a subsequent frame with the
> batch bit clear is available. A BWTP message is a batch of one or more
> websocket frames. It is hoped that the WebSocket specification will adopt
> the definition of this bit.
>
> BWTP defines several non data frame types that are used to coordinate the
> protocol or to send meta. The 4 low order bits of the WebSocket frametype
> are used the carry the BWTP frame type. The following BWTP frame types are
> defined:
> xxxx0000Dataxxxx0001*undefined*xxxx0010Message Header Namexxxx0011Message
> Header Valuexxxx0100Sequence numberxxxx0101Acknowlege numberxxxx0110*
> undefined*xxxx0111*undefined*xxxx1000Channel Identifierxxxx1001Channel
> Statexxxx1010Channel Header Namexxxx1011Channel Header Valuexxxx1100*
> undefined*xxxx1101*undefined*xxxx1110*undefined*xxxx1111*undefined*3.1.1<#1280740bc6667cea_rfc.section.3.1.1>
>  Data
>
> The simplest BWTP message is a single WebSocket data frame, either UTF-8
> encoded frame:
>
> {00000000|Hello Beautiful World|0xFF}
>
> or length encoded binary frame:
>
> {10000000|0x15|Hello Beautiful World}
>
> 3.1.2 <#1280740bc6667cea_rfc.section.3.1.2> Fragmentation
>
> A BWTP frame may be fragmented into multiple frames, with each incomplete
> frame having bit 6 of the frame type byte set to 1. For example, the
> following three frames represent a single data message "Hello Beautiful
> World":
>
> {11000000|0x06|Hello }
> {11000000|0x0A|Beautiful }
> {10000000|0x05|World}
>
> 3.1.3 <#1280740bc6667cea_rfc.section.3.1.3> Message Meta Data
>
> A BWTP message may include meta data by including meta data frames in a
> batch with a data frame. The following example show a BWTP message with a
> content type set as meta data header and value:
>
> [{10100010|0x0C|Content-Type}
>  {10100011|0x09|text/html}
>  {10000000|0x14|<h1>Hello World</h1>}]
>
> A BWTP message batch may also be fragmented:
>
> [{10100010|0x0C|Content-Type}
>  {10100011|0x09|text/html}
>  {11000000|0x0A|<h1>Hello }
>  {10000000|0x0A|World</h1>}]
>
> Note that in order to avoid the overheads associated with verbose redundant
> headers in HTTP, BWTP allows headers to be be sent associated with a
> channel, so that all messages in channel will inherit the meta data, which
> will not need to be sent with every message.
> 3.2 <#1280740bc6667cea_rfc.section.3.2> Channels
>
> BWTP connections may contain multiple Channels. Each Channel is an ordered
> sequence of frames and all BWTP endpoints and intermediaries must preserve
> the order of frames within a Channel. However, the frames of one channel may
> be interleaved with the frames of another and intermediaries are free to
> change the interleaving.
>
> A Channel is identified by a 32 bit signed integer and is created by the
> first usage of the channel. Channels that are created by a client MUST be
> assigned an even ID and channels created by the server MUST be assigned an
> odd ID, so as to avoid creation collissions [Note that this is opposite to
> the similar convention in SPDY, because channel 0 is implicitly opened by
> the client]. Once created, a channel is bidirectional.
>
> A Channel ID is sent in a Channel Identifier frame and once sent, all
> subsequent frames transmitted are associated with the channel, until a
> subsequent Channel Identifier frame is sent. This avoid the overhead of
> sending a channel identifier with every message.
>
> The following example shows messages for channel 5 interleaved with
> messages for channel 6:
>
> [{10001000|0x04|0x00 0x00 0x00 0x05}]
> [{00000000|Hello Channel 5|0xff}]
> [{00000000|Hello Channel 5 again|0xff}]
> [{10001000|0x04|0x00 0x00 0x00 0x06}]
> [{10100010|0x0C|Content-Type}
>  {10100011|0x09|text/html}
>  {10000000|0x14|<h1>Hello Chan 6</h1>}]
> [{10001000|0x04|0x00 0x00 0x00 0x05}]
> [{00000000|Hello Channel 5 yet again|0xff}]
>
> [ Option? A channel ID that is part of a batch could only effect the frames
> in that batch ??? but what if the channel ID was not the first in the batch
> ??? ]
>
> Channels have a simple state machine which applies to both the inbound and
> outbound parts of the Channel. The state machine allows for channels to be
> Open, Blocked by flow control, Closing, Closed and Reset. The state of a
> channel is sent in a Channel State frame. The following example shows a BWTP
> message to initiate an orderly close of a channel:
>
> [{11001000|0x04|0x00 0x00 0x00 0x05}
>  {10001001|0x01|CLOSED}]
>
> 3.3 <#1280740bc6667cea_rfc.section.3.3> HTTP over BWTP
>
> A BWTP Channel can be used to carry HTTP requests and responses as follows:
>
>    - The request line is sent as 2 value frames at the start of a message
>    batch (HTTP/1.1 is assumed).
>    - Headers are sent as header frame and value frame pairs
>    - Headers that are not expected to change (eg User-Agent) are sent once
>    as channel header frames and value frame pairs
>    - The response status lines is sent as 3 value frames at the start of a
>    message batch.
>    - The order of responses is the same as the request order, only within
>    the same channel.
>
> The following example shows a channel being opened and two pipelined GET
> requests:
>
> [{10001000|0x04|0x00 0x00 0x00 0x02}]
> [{10101010|0x0A|User-Agent}
>  {10101011|0x14|Chromezilla Exploder}
>  {10101010|0x04|Host}
>  {10101011|0x0B|example.org}
>  {10101010|0x06|Cookie}
>  {10001011|0x15|JSESSIONID=1234567890}]
> [{10100011|0x03|GET}
>  {10100011|0x01|/}
>  {10100010|0x??|If-Modified-Since}
>  {10000011|0x??|12:34 5 January 1964}]
> [{10100011|0x03|GET}
>  {10000011|0x??|/favicon.ico}]
>
>
> 3.4 <#1280740bc6667cea_rfc.section.3.4> WebSocket over BWTP
>
> A BWTP Channel can be used to carry a WebSocket connection, so that
> multiple WebSocket connections can be multiplexed over the a shared BWTP
> Websocket connections. The WebSocket handshake is sent as HTTP over BWTP, so
> that URL and sub protocol can be established.
> 4. <#1280740bc6667cea_rfc.section.4> new section5.<#1280740bc6667cea_rfc.section.5>Normative References
> *[RFC2119]*Bradner, S. <[e-mail]>, “Key words for use in RFCs to
> Indicate Requirement Levels <http://tools.ietf.org/html/rfc2119>”, BCP 14,
> RFC 2119, March 1997.*[RFC2616]*Fielding, R. <[e-mail]>, Gettys,
> J. <[e-mail]>, Mogul, J. <[e-mail]>, Frystyk, H.<[e-mail]>,
> Masinter, L. <[e-mail]>, Leach, P. <[e-mail]>,
> and T. Berners-Lee <[e-mail]>, “Hypertext Transfer Protocol --
> HTTP/1.1 <http://tools.ietf.org/html/rfc2616>”, RFC 2616, June 1999.
> Authors' Addresses <#1280740bc6667cea_rfc.authors>Greg WilkinsWilkinsGreg
> WebtideEMail: [e-mail] HintjensHintjensPieterEMail:
> [e-mail]
>
> _______________________________________________
> hybi mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/hybi
>
>