Re: [hybi] With deflate-stream, Close frame doesn't work as an end of data marker

Andy Green <andy@warmcat.com> Wed, 09 March 2011 22:12 UTC

Return-Path: <andy.warmcat.com@googlemail.com>
X-Original-To: hybi@core3.amsl.com
Delivered-To: hybi@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id DC65D3A6AEB for <hybi@core3.amsl.com>; Wed, 9 Mar 2011 14:12:13 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.585
X-Spam-Level:
X-Spam-Status: No, score=-3.585 tagged_above=-999 required=5 tests=[AWL=0.014, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fS8w0IV45phs for <hybi@core3.amsl.com>; Wed, 9 Mar 2011 14:12:12 -0800 (PST)
Received: from mail-ww0-f44.google.com (mail-ww0-f44.google.com [74.125.82.44]) by core3.amsl.com (Postfix) with ESMTP id 39D393A6AE2 for <hybi@ietf.org>; Wed, 9 Mar 2011 14:12:12 -0800 (PST)
Received: by wwa36 with SMTP id 36so857165wwa.13 for <hybi@ietf.org>; Wed, 09 Mar 2011 14:13:28 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:sender:message-id:date:from:user-agent :mime-version:to:cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=i/yiJXK/HJ1/2p79bB6L6F5TwYQmxdUc6b83aAFDzFY=; b=hIqwL0qacDjIqxJWbbP0jg6AiwencXOQGhFxUH6HhnsLQZnyLEHD8gTQXysZ8DmFGr YPoWT9rNm5Fe501VNtDRRfbtlBIQ6SbqmFx56D77SYS5M5LTqwYxDCsfVntDEnUgZmtn Oq/1eZn9zx62QN8TsuxzyI6td7JKZaWqpzong=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; b=bLe/H5V36bvbkeyIqf8UP8kj3Bvg5cP+jw4vOn8oJbLenvNnr5xfhrjV61Fgw5L8Yb O+iR4Ny/KobUP65TxC95UTLjA0DCe+ZD8ia4261DouG7XCZIYTZiai9dNCV5ARjfwt/j 1HT3nwBGPv+zAP3NiPpmGfwOisOBfwMA7qLiA=
Received: by 10.227.139.89 with SMTP id d25mr698333wbu.58.1299708807836; Wed, 09 Mar 2011 14:13:27 -0800 (PST)
Received: from otae.warmcat.com (s15404224.onlinehome-server.info [87.106.134.80]) by mx.google.com with ESMTPS id o6sm1833760wbo.3.2011.03.09.14.13.25 (version=SSLv3 cipher=OTHER); Wed, 09 Mar 2011 14:13:26 -0800 (PST)
Sender: Andy Green <andy.warmcat.com@googlemail.com>
Message-ID: <4D77FB84.9010104@warmcat.com>
Date: Wed, 09 Mar 2011 22:13:24 +0000
From: Andy Green <andy@warmcat.com>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.14) Gecko/20110302 Fedora/3.1.8-3.fc16 Thunderbird/3.1.8
MIME-Version: 1.0
To: Yutaka_Takeda@PlayStation.Sony.Com
References: <OF4821AA00.120197B4-ON8825784E.0076F791-8825784E.00781253@playstation.sony.com>
In-Reply-To: <OF4821AA00.120197B4-ON8825784E.0076F791-8825784E.00781253@playstation.sony.com>
Content-Type: text/plain; charset="ISO-8859-1"; format="flowed"
Content-Transfer-Encoding: 7bit
Cc: hybi@ietf.org
Subject: Re: [hybi] With deflate-stream, Close frame doesn't work as an end of data marker
X-BeenThere: hybi@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: Server-Initiated HTTP <hybi.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/hybi>, <mailto:hybi-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/hybi>
List-Post: <mailto:hybi@ietf.org>
List-Help: <mailto:hybi-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/hybi>, <mailto:hybi-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 09 Mar 2011 22:12:14 -0000

On 03/09/2011 09:51 PM, Somebody in the thread at some point said:

Hi -

>  > Are there other circumstances that can actually cause this I'm
>  > overlooking too easily?
>
> Yes. Since your receive buffer passed in recv(2) has a predetermined
> finite size and you may not be reading all data in the kernel space
> (TCP) buffer. Depending on how it is implemented and timing, the
> receiver may successfully extract Close frame out of data you have just
> read from TCP socket, and then close the socket without bothering to
> read the outstanding data in the kernel.

It's a good point.  I can see how that's effectively fragmenting the 
thing entirely in userland at the receiver.

However that seems simple to solve since the extra junk data is already 
received in kernel.  Just before socket close time we just need to use a 
little poll() POLLIN loop with no timeout to drain it while there's 
stuff immediately pending, right?  Nothing more will come and close is 
always happy.

-Andy