Re: [hybi] Autobahn test suite

"Len Holgate" <len.holgate@gmail.com> Tue, 30 August 2011 18:47 UTC

Return-Path: <len.holgate@gmail.com>
X-Original-To: hybi@ietfa.amsl.com
Delivered-To: hybi@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0E77021F8D3F for <hybi@ietfa.amsl.com>; Tue, 30 Aug 2011 11:47:29 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.599
X-Spam-Level:
X-Spam-Status: No, score=-3.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id j1ck-kQgsLKO for <hybi@ietfa.amsl.com>; Tue, 30 Aug 2011 11:47:28 -0700 (PDT)
Received: from mail-ww0-f44.google.com (mail-ww0-f44.google.com [74.125.82.44]) by ietfa.amsl.com (Postfix) with ESMTP id 44E2921F8D36 for <hybi@ietf.org>; Tue, 30 Aug 2011 11:47:28 -0700 (PDT)
Received: by wwf5 with SMTP id 5so4746510wwf.13 for <hybi@ietf.org>; Tue, 30 Aug 2011 11:48:55 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:to:cc:references:subject:date:message-id:mime-version :content-type:content-transfer-encoding:x-mailer:in-reply-to :thread-index:x-mimeole; bh=5YnkafplK7eQOJnoozpE0CW3tJ8Fys5zE4EWckbz5ak=; b=rtu6UnC6PMGJd6/Jy/lJaT7dVena3hV+NC9E5b6zIcnWzOcYViSYtMYvH6gMT58jrU TeZdw/G/7/qHU6TFFszRFRAkR52NIk99/WjtDNjoOrDIfVDN+fb4KkOQGbGRvS5di9Lu ChkIWZvZ3rZ+fF/rqxcTGU1kyNwtn2pb6fmIw=
Received: by 10.227.59.73 with SMTP id k9mr5432950wbh.81.1314730135720; Tue, 30 Aug 2011 11:48:55 -0700 (PDT)
Received: from Venus (cpc4-glfd6-2-0-cust201.6-2.cable.virginmedia.com [80.5.68.202]) by mx.google.com with ESMTPS id fm9sm4852081wbb.44.2011.08.30.11.48.54 (version=SSLv3 cipher=OTHER); Tue, 30 Aug 2011 11:48:55 -0700 (PDT)
From: Len Holgate <len.holgate@gmail.com>
To: 'Tobias Oberstein' <tobias.oberstein@tavendo.de>
References: <CA68A4DB.477B%tobias.oberstein@tavendo.de><CAH9hSJamGCGx3Q4jgCji3Xy1fYfRDqkwDvWuxh2h++oEELveNw@mail.gmail.com> <634914A010D0B943A035D226786325D422BFBFBF15@EXVMBX020-12.exch020.serverdata.net> <0cc701cc673c$fec42d70$0a00a8c0@Venus> <634914A010D0B943A035D226786325D422C0DFD271@EXVMBX020-12.exch020.serverdata.net>
Date: Tue, 30 Aug 2011 19:48:22 +0100
Message-ID: <0cfb01cc6745$659994b0$0a00a8c0@Venus>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
X-Mailer: Microsoft Office Outlook 11
In-Reply-To: <634914A010D0B943A035D226786325D422C0DFD271@EXVMBX020-12.exch020.serverdata.net>
Thread-Index: AcxXwXL4sJmmuO3yQXO+aY/Hi+VaIwAYSzfAA8BrwIAABtLQcAABRtyg
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3664
Cc: hybi@ietf.org
Subject: Re: [hybi] Autobahn test suite
X-BeenThere: hybi@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Server-Initiated HTTP <hybi.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/hybi>, <mailto:hybi-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/hybi>
List-Post: <mailto:hybi@ietf.org>
List-Help: <mailto:hybi-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/hybi>, <mailto:hybi-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 30 Aug 2011 18:47:29 -0000

> - those cases are really extreme (fragment size 64 with 
> payload 8M, chopsize = 1 octet)

Yes, I know. The issue doesn't seem to be on my side anymore, but it may
well be.

> - did you run on loopback or remote? you could try different 
> platform and/or loopback/remote ..

So far it's all been on loopback. When I get a moment I may run it remotely
and grab the wireshark logs.

> - don't know your Python background: when you change a 
> CaseXX.py, you need to do
> "python setup.py install" again to get that into 
> site-packages. Also, try add a print timeout
> somewhere to see if the change really happens, because it's 
> strange that it doesnt obey

I worked that out eventually. The timeout does seem to be changed as the
error then states the new timeout value. The actual test still terminates
after 6 secs or so though.

> - you can reenable wirelog for those cases (and decrease the 
> payload to something small;),

I've enabled the wirelog, that didn't show much, except that the test thinks
it has sent everything and started the timeout and then that I close the
connection. 

At present, since it's the only failing test I'm not going to spend that
much time on it. 

Are any of the other tests multi-fragment with 1 byte chopsize? If not I'll
add one that's a bit more manageable and see if I can find out what's wrong
in my code.

> Lastly, your project is commercial, but you might 
> nevertheless be interested in putting it
> onto
> 
> http://en.wikipedia.org/wiki/Comparison_of_WebSocket_implementations

Ok, I wasn't sure if commercial stuff could be added. 

> I'll extend the test suite to cover UTF-8/Handshake 
> (open/close) as soon as I find time
> again .. should those issues persist for you, we can try to 
> nail it down (then, because
> for the moment, I'm a bit overwhelmed with other stuff)

I know the feeling. More tests would be good, but the test suite as it
stands has been very useful to me.

Len