Re: [hybi] CONNECT handshake text

[Ian Fette (イアンフェッティ) <[e-mail]>]

2010/12/8 Simon Pieters <[e-mail]>

> On Wed, 08 Dec 2010 16:42:04 +0100, Ian Fette (イアンフェッティ) <
> [e-mail]> wrote:
>
>  2010/12/8 Simon Pieters <[e-mail]>
>>
>>  On Tue, 07 Dec 2010 19:01:34 +0100, Ian Fette (イアンフェッティ) <
>>> [e-mail]> wrote:
>>>
>>>  There's a lot of back and forth lately about a possible CONNECT
>>> handshake
>>>
>>>> and how it should look. I thought it might be helpful to send out some
>>>> draft
>>>> text that is a somewhat minimal CONNECT handshake, without things like
>>>> JSON-encoded data and payload masking, as I think we may be getting
>>>> distracted by some of these things and I'm not sure they're actually
>>>> necessary or material.
>>>>
>>>> Below is some text that I would like the group to consider for a CONNECT
>>>> handshake, as it would fit into the WebSockets protocol.
>>>>
>>>>
>>> At the risk of distracting more, I'll provide some quick comments.
>>>
>>>
>>>  1.2.  Protocol overview
>>>
>>>>
>>>>
>>> I didn't look at the non-normative sections.
>>>
>>>
>>>
>>>  5.  Opening Handshake
>>>
>>>>
>>>> 5.1.  Client Requirements
>>>>
>>>>  User agents running in controlled environments, e.g. browsers on
>>>>  mobile handsets tied to specific carriers, may offload the management
>>>>  of the connection to another agent on the network.  In such a
>>>>  situation, the user agent for the purposes of conformance is
>>>>  considered to include both the handset software and any such agents.
>>>>
>>>>  When the user agent is to *establish a WebSocket connection* to a
>>>>  WebSocket URL /url/, it must meet the following requirements.  In the
>>>>  following text, we will use terms from Section 3 such as "/host/" and
>>>>  "/secure/ flag" as defined in that section.
>>>>
>>>>
>>> The WebSocket API passes the following "parameters" to the "establish a
>>> WebSocket connection" algorithm:
>>>
>>> [[
>>> Establish a WebSocket connection to a host host, on port port (if one was
>>> specified), from origin, with the flag secure, with resource name as the
>>> resource name, with protocols as the (possibly empty) list of protocols,
>>> and
>>> with the defer cookies flag set. [WSP]
>>> ]]
>>>
>>> i.e. host, port, origin, secure, resource name, protocols, defer cookies.
>>> The URL is *not* passed.
>>>
>>
>>
>> This needs to be parsed from the Sec-WebSocket-URL header. I will clarify.
>> (or, if we decide not to mask host, then it may change slightly.)
>>
>
> I don't follow... The WebSocket API spec says to parse the URL passed to
> the WebSocket() constructor and invoke "establish a WebSocket connection"
> with the above parameters -- it does not pass the URL to the algorithm.
>
>
>
>>
>>>
>>>
>>>   1.  The WebSocket URL and its components MUST be valid according to
>>>
>>>>      Section 3.3.  If any of the requirements are not met, the client
>>>>      MUST fail the WebSocket connection and abort these steps.
>>>>
>>>>
>>> The WebSocket API validates the URL.
>>>
>>>
>>
>> I'm not sure I understand your comment. Are you saying it's validated
>> client-side?
>>
>
> Yes. See http://dev.w3.org/html5/websockets/#dom-websocket
>
>
>
>  Even if so, the HTML5 clients are not necessarily the only
>> users of the protocol. I think it's reasonable to leave validation text in
>> for servers, if the browser also validates which it should, then there
>> shouldn't be any problem. Or am I mis-interpreting?
>>
>
> We're discussing the section which covers client-side requirements, not the
> server-side requirements.


But we shouldn't make the assumption that the HTML5 spec documents the only
client, IMO.


>
>
>
>
>>>
>>>   2.  If the user agent already has a WebSocket connection to the
>>>
>>>>      remote host (IP address) identified by /host/, even if known by
>>>>      another name, the user agent MUST wait until that connection has
>>>>      been established or for that connection to have failed.  If
>>>>      multiple connections to the same IP address are attempted
>>>>      simultaneously, the user agent MUST serialize them so that there
>>>>      is no more than one connection at a time running through the
>>>>      following steps.
>>>>
>>>>      If the user agent cannot determine the IP address of the remote
>>>>      host (for example because all communication is being done through
>>>>      a proxy server that performs DNS queries itself), then the user
>>>>      agent MUST assume for the purposes of this step that each host
>>>>      name refers to a distinct remote host, but should instead limit
>>>>      the total number of simultaneous connections that are not
>>>>      established to a reasonably low number (e.g., in a Web browser,
>>>>      to the number of tabs the user has open).
>>>>
>>>>      NOTE: This makes it harder for a script to perform a denial of
>>>>      service attack by just opening a large number of WebSocket
>>>>      connections to a remote host.  A server can further reduce the
>>>>      load on itself when attacked by making use of this by pausing
>>>>      before closing the connection, as that will reduce the rate at
>>>>      which the client reconnects.
>>>>
>>>>      NOTE: There is no limit to the number of established WebSocket
>>>>      connections a user agent can have with a single remote host.
>>>>      Servers can refuse to connect users with an excessive number of
>>>>      connections, or disconnect resource-hogging users when suffering
>>>>
>>>>
>>>>
>>>> Fette                     Expires June 8, 2011                 [Page 24]
>>>> %0CInternet-Draft           The WebSocket protocol            December
>>>> 2010
>>>>
>>>>
>>>>
>>>>      high load.
>>>>
>>>>  3.  _Proxy Usage_: If the user agent is configured to use a proxy
>>>>      when using the WebSocket protocol to connect to host /host/
>>>>      and/or port /port/, then the user agent SHOULD connect to that
>>>>      proxy and ask it to open a TCP connection to the host given by
>>>>      /host/ and the port given by /port/.
>>>>
>>>>         EXAMPLE: For example, if the user agent uses an HTTP proxy for
>>>>         all traffic, then if it was to try to connect to port 80 on
>>>>         server example.com, it might send the following lines to the
>>>>         proxy server:
>>>>
>>>>
>>>>             CONNECT example.com:80 HTTP/1.1
>>>>             Host: example.com
>>>>
>>>>         If there was a password, the connection might look like:
>>>>
>>>>
>>>>             CONNECT example.com:80 HTTP/1.1
>>>>             Host: example.com
>>>>             Proxy-authorization: Basic ZWRuYW1vZGU6bm9jYXBlcyE=
>>>>
>>>>      If the user agent is not configured to use a proxy, then a direct
>>>>      TCP connection SHOULD be opened to the host given by /host/ and
>>>>      the port given by /port/.
>>>>
>>>>      NOTE: Implementations that do not expose explicit UI for
>>>>      selecting a proxy for WebSocket connections separate from other
>>>>      proxies are encouraged to use a SOCKS proxy for WebSocket
>>>>      connections, if available, or failing that, to prefer the proxy
>>>>      configured for HTTPS connections over the proxy configured for
>>>>      HTTP connections.
>>>>
>>>>      For the purpose of proxy autoconfiguration scripts, the URL to
>>>>      pass the function must be constructed from /host/, /port/,
>>>>      /resource name/, and the /secure/ flag using the steps to
>>>>      construct a WebSocket URL.
>>>>
>>>>      NOTE: The WebSocket protocol can be identified in proxy
>>>>      autoconfiguration scripts from the scheme ("ws:" for unencrypted
>>>>      connections and "wss:" for encrypted connections).
>>>>
>>>>  4.  If the connection could not be opened, either because a direct
>>>>      connection failed or because any proxy used returned an error,
>>>>      then the user agent MUST fail the WebSocket connection and abort
>>>>      the connection attempt.
>>>>
>>>>
>>>>
>>>> Fette                     Expires June 8, 2011                 [Page 25]
>>>> %0CInternet-Draft           The WebSocket protocol            December
>>>> 2010
>>>>
>>>>
>>>>
>>>>  5.  If /secure/ is true, the user agent MUST perform a TLS handshake
>>>>      over the connection.  If this fails (e.g. the server's
>>>>      certificate could not be verified), then the user agent MUST fail
>>>>      the WebSocket connection and abort the connection.  Otherwise,
>>>>      all further communication on this channel MUST run through the
>>>>      encrypted tunnel.  [RFC2246]
>>>>
>>>>      User agents MUST use the Server Name Indication extension in the
>>>>      TLS handshake.  [RFC4366]
>>>>
>>>>  Once a connection to the server has been established (including a
>>>>  connection via a proxy or over a TLS-encrypted tunnel), the client
>>>>  MUST send a handshake to the server.  The handshake consists of a
>>>>  CONNECT request, along with a list of required and optional headers.
>>>>  The requirements for this handshake are as follows.
>>>>
>>>>  1.   The handshake must be a valid HTTP request as specified by
>>>>       [RFC2616].
>>>>
>>>>
>>> The current draft says which bytes should be put on the wire exactly. I
>>> prefer the current draft over referencing RFC2616 since doing the latter
>>> makes many variations conforming which can cause interop problems.
>>>
>>>
>>>  And this was a source of contention for many people.
>>
>>
>>
>>>
>>>   2.   The Method of the request MUST be CONNECT, and the authority
>>>
>>>>       MUST be websocket.invalid:443.  The HTTP version MUST be at
>>>>       least 1.1.
>>>>
>>>>
>>> Why at least 1.1 and not exactly 1.1?
>>>
>>
>>
>> Is there any particular reason to exclude future versions? I don't know if
>> it will really matter, happy to change if people want it to be exactly
>> 1.1.
>>
>>
>>
>>>
>>>
>>>        The first line sent SHOULD be "CONNECT websocket.invalid:443
>>>
>>>>       HTTP/1.1"
>>>>
>>>>  3.   The request MUST contain a "HOST" header whose value is equal to
>>>>       "websocket.invalid:443"
>>>>
>>>>
>>> "HOST" or "Host"?
>>>
>>
>>
>> Typo, Host. Thanks.
>>
>>
>>
>>>
>>>
>>>   4.   The request MUST include a header with the name "Sec-WebSocket-
>>>
>>>>       Key".  The value of this header MUST be a nonce consisting of a
>>>>       randomly selected 16-byte value that has been base64-encoded
>>>>       [RFC3548].  The nonce MUST be randomly selected for each
>>>>       connection.
>>>>
>>>>       NOTE: As an example, if the randomly selected value was the set
>>>>       of bytes 0x01 0x02 0x03 0x04 0x05 0x06 0x07 0x08 0x09 0x0a 0x0b
>>>>       0x0c 0x0d 0x0e 0x0f 0x10, the value of the header should be
>>>>       "AQIDBAUGBwgJCgsMDQ4PEC=="
>>>>
>>>>  5.   The request MUST include a header with the name "Sec-WebSocket-
>>>>       Origin".  The value of this header MUST be the origin of the
>>>>       context in which the code establishing the connection is
>>>>       running.
>>>>
>>>>
>>> The value must be /origin/ that is passed to the algorithm. The WebSocket
>>> API has requirements on what /origin/ is.
>>>
>>>
>> Ok, here I wasn't sure as again, the HTML5 WebSocket API is not
>> necessarily
>> the only thing implementing the protocol. Perhaps I can change it though
>> to
>> simply reference the definition in the HTML5 spec for HTML clients, and
>> leave the somewhat more generic description for other non-HTML clients?
>> Thoughts? E.g. it's not clear referencing the HTML5 API exactly would be
>> appropriate if a plugin wished to implement the API.
>>
>
> I'm not saying you should reference the WebSocket API spec. I'm saying you
> should do what the current spec does and expect the input parameter /origin/
> to be correct. See
> http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-03#section-11


Apologies. I now understand a lot of the confusion. I neglected to update
Section 11. I agree that as it stands, this is in conflict, and will fix.



>
>
>
>>
>>>
>>>        As an example, if code is running on www.example.com attempting
>>>
>>>>       to establish a connection to ww2.example.com, the value of the
>>>>       header should be "http://www.example.com".
>>>>
>>>>
>>>>
>>>> Fette                     Expires June 8, 2011                 [Page 26]
>>>> %0CInternet-Draft           The WebSocket protocol            December
>>>> 2010
>>>>
>>>>
>>>>
>>>>  6.   The request MUST include a header with the name "Sec-WebSocket-
>>>>       URL".  The value of this header MUST be the WebSocket URL to
>>>>       which the connection is to be made.
>>>>
>>>>
>>> Why URL? Current handshake uses /resource name/.
>>>
>>
>>
>> Because the current handshake also has the Host and resource as part of
>> the
>> GET, and the new handshake using CONNECT doesn't have that, so it has to
>> go
>> somewhere. At that point, it becomes a question of adding a host and
>> resource name header, or adding a single URL header. Adding a single URL
>> header seems cleaner.
>>
>
> OK. Then you either need to change section 11 to say to pass /URL/ to
> "establish a WebSocket connection" and get the WebSocket API spec changed,
> or let the input parameters be the same as in -03 and invoke "*construct a
> WebSocket URL* from a /host/, a /port/, a /resource name/, and a /secure/
> flag" to get the URL.
>
>
Agreed, sorry, I neglected to update Section 11. I will ensure these two are
in sync. Whether we keep a URL or a seprate host and resource name depends a
bit on whether we go the UPGRADE+CONNECT or CONNECT route, but either way
the two sections do need to agree. Sorry I didn't understand your point the
first time around.


>
>
>
>>
>>>
>>>
>>>   7.   The request MUST include a header with the name "Sec-WebSocket-
>>>
>>>>       Draft".  The value of this header must be 4.
>>>>
>>>>  8.   The request MAY include a header with the name "Sec-WebSocket-
>>>>       Protocol".  If present, this value indicates the subprotocol(s)
>>>>       the client wishes to speak.  The ABNF for the value of this
>>>>       header is 1#(token | quoted-string), where the definitions of
>>>>       /token/ and /quoted-string/ are as given in [RFC2616].
>>>>
>>>>
>>> MAY? This should be MUST if /protocols/ is not empty, with the value of
>>> /protocols/ joined with U+0020, or MUST NOT if /protocols/ is empty.
>>>
>>
>>
>> I was trying to describe the requirements for a handshake. The header is
>> optional in that the protocols field is not a required component. I also
>> thought we had moved towards agreement on getting away from re-defining
>> header parsing in this spec and instead  referencing 2616, e.g.
>> Sec-WebSocket-Protocol: chat, superchat, "my, cool, thing"
>>
>
> So space should be allowed in a subprotocol then? I assume backslash
> escaping is also necessary? My knee-jerk reaction is that using the above
> syntax instead of the simpler separate-with-U+0020 will lead to more bugs in
> server impls.


It's already allowed in certain HTTP headers. The intent was to re-use
existing HTTP header parsing code rather than defining yet another format
for HTTP headers. E.g. see Pragma in HTTP. It's defined as
Pragma            = "Pragma" ":" 1#pragma-directive
pragma-directive  = "no-cache" | extension-pragma
extension-pragma  = token [ "=" ( token | quoted-string ) ]

So this basically gives a comma separated list of words or quoted strings
and browsers already have code to parse this. It seems easier to re-use this
than implement yet another set of parsing semantics.


>
>
>
>>
>>
>>>
>>>
>>>   9.   The request MAY include a header with the name "Sec-WebSocket-
>>>
>>>>       Extensions".  If present, this value indicates the protocol-
>>>>       level extension(s) the client wishes to speak.  The ABNF for the
>>>>       value of this header is 1#(token | quoted-string), where the
>>>>       definitions of /token/ and /quoted-string/ are as given in
>>>>       [RFC2616].
>>>>
>>>>
>>> There are no extensions defined yet, right? Are vendor-proprietary
>>> extensions allowed or not?
>>>
>>
>>
>> There's one at the end of the spec (compression) in -03. And vendor
>> proprietary extensions would certainly be allowed.
>>
>
> Ok.
>
>
>
>>>
>>>
>>>   10.  The request MAY include headers associated with sending cookies,
>>>
>>>>       as defined by the appropriate specifications.  [RFC2109]
>>>>       [RFC2965]
>>>>
>>>>
>>> MAY? Which cookies? Current draft has more precise rules here.
>>>
>>
>>
>> Again, MAY as this is not required for a successful handshake. I'm happy
>> to
>> throw in a MUST for browser clients specifically, but it seems like that
>> ought to be in the HTML spec and not the protocol spec. The protocol spec
>> should really be specifying the protocol for all clients, and what is
>> allowed to be sent over that protocol. Instructions specific to a
>> particular
>> client seem like they belong with the specification for that client, e.g.
>> the HTML spec should specify what cookies must be sent with a WebSocket
>> request for browsers, not the protocol spec.
>>
>
> Ok. Then I guess a list of cookies is also needed as an input parameter to
> "establish a WebSocket connection" (and section 11 changed as appropriate)
> so that the WebSocket API spec can say which cookies to pass to the
> algorithm. I'd still like the protocol spec to say that if there are cookies
> passed to the algorithm, then they must be sent.
>
>
That seems reasonable, I will make changes to that effect.


>
>
>>
>>>
>>>
>>>   Once the client's opening handshake has been sent, the client MUST
>>>
>>>>  wait for a response from the server before sending any further data.
>>>>  The server's response is detailed in the next section.
>>>>
>>>>
>>> *Sending* the server's response is detailed in the next section. Reading
>>> it
>>> is not.
>>>
>>
>>
>> It seemed somewhat duplicative to describe the exact same thing twice.
>> It's
>> something I find quite frustrating in -03. I want to know what the format
>> of
>> the handshake is, I don't want a precise set of steps each side must take
>> --
>> that is much harder to interpret, and frankly I doubt people will code it
>> exactly as written.
>>
>
> Well at Opera we implemented it exactly as written. Browser vendors are
> often anal about interoperability these days. Having loose rules leads to
> interop problems down the road with some sites just working in the most
> popular browser and the other browsers need to reverse engineer it to figure
> out what to do to be compatible instead of just implementing the spec.
>
>
>
>>
>>>
>>>   Specifically,
>>>
>>>>  the client MUST receive from the server a 200 status code in the HTTP
>>>>  response, and a "Sec-WebSocket-Accept" header.  The value of this
>>>>  header MUST be the value specified in the following section, namely a
>>>>  SHA-1 hash of the concatenation of the Sec-WebSocket-Key with the
>>>>  string "258EAFA5-E914-47DA-95CA-C5AB0DC85B11", base64-encoded.  Any
>>>>  other status code or value of "Sec-WebSocket-Accept" MUST NOT be
>>>>  interpreted as having completed a WebSocket handshake, and the client
>>>>  MUST NOT send WebSocket framed data over this connection until a
>>>>  successful handshake is completed.
>>>>
>>>>
>>> Why not put this as part of the algorithm? See step 31 to 51 in the
>>> current
>>> draft. It misses a lot of critical details present in the current draft,
>>> like actually invoking _fail the WebSocket connection_, defining *the
>>> WebSocket connection is established*, applying cookies, and other things.
>>>
>>>
>> It is. As for applying cookies, text welcome.
>>
>
> See -03, step 44 and step 51.
>
>
>
>
>>
>>>
>>>
>>>   Where the algorithm above requires that a user agent fail the
>>>
>>>>  WebSocket connection, the user agent may first read an arbitrary
>>>>  number of further bytes from the connection (and then discard them)
>>>>  before actually *failing the WebSocket connection*.  Similarly, if a
>>>>  user agent can show that the bytes read from the connection so far
>>>>  are such that there is no subsequent sequence of bytes that the
>>>>  server can send that would not result in the user agent being
>>>>  required to *fail the WebSocket connection*, the user agent may
>>>>  immediately *fail the WebSocket connection* without waiting for those
>>>>  bytes.
>>>>
>>>>
>>>>
>>>>
>>>> Fette                     Expires June 8, 2011                 [Page 27]
>>>> %0CInternet-Draft           The WebSocket protocol            December
>>>> 2010
>>>>
>>>>
>>>>
>>>>  NOTE: The previous paragraph is intended to make it conforming for
>>>>  user agents to implement the algorithm in subtly different ways that
>>>>  are equivalent in all ways except that they terminate the connection
>>>>  at earlier or later points.  For example, it enables an
>>>>  implementation to buffer the entire handshake response before
>>>>  checking it, or to verify each field as it is received rather than
>>>>  collecting all the fields and then checking them as a block.
>>>>
>>>> 5.2.  Server-side requirements
>>>>
>>>>
>>> I haven't looked at this section.
>>>
>>> --
>>> Simon Pieters
>>> Opera Software
>>>
>>>
> Cheers,
> --
> Simon Pieters
> Opera Software
>