Re: [ppsp] [decade] Object naming in -req and -arch

Songhaibin <haibin.song@huawei.com> Thu, 19 July 2012 02:37 UTC

Return-Path: <haibin.song@huawei.com>
X-Original-To: ppsp@ietfa.amsl.com
Delivered-To: ppsp@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D8DFE21F8680; Wed, 18 Jul 2012 19:37:56 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.353
X-Spam-Level:
X-Spam-Status: No, score=-6.353 tagged_above=-999 required=5 tests=[AWL=0.246, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id INLHuw4t4Spa; Wed, 18 Jul 2012 19:37:56 -0700 (PDT)
Received: from dfwrgout.huawei.com (dfwrgout.huawei.com [206.16.17.72]) by ietfa.amsl.com (Postfix) with ESMTP id 22C8A21F84EA; Wed, 18 Jul 2012 19:37:56 -0700 (PDT)
Received: from 172.18.9.243 (EHLO dfweml202-edg.china.huawei.com) ([172.18.9.243]) by dfwrg02-dlp.huawei.com (MOS 4.2.3-GA FastPath) with ESMTP id AHW57758; Wed, 18 Jul 2012 22:38:48 -0400 (EDT)
Received: from DFWEML407-HUB.china.huawei.com (10.193.5.132) by dfweml202-edg.china.huawei.com (172.18.9.108) with Microsoft SMTP Server (TLS) id 14.1.323.3; Wed, 18 Jul 2012 19:35:47 -0700
Received: from SZXEML408-HUB.china.huawei.com (10.82.67.95) by dfweml407-hub.china.huawei.com (10.193.5.132) with Microsoft SMTP Server (TLS) id 14.1.323.3; Wed, 18 Jul 2012 19:35:46 -0700
Received: from SZXEML534-MBX.china.huawei.com ([169.254.2.243]) by szxeml408-hub.china.huawei.com ([10.82.67.95]) with mapi id 14.01.0323.003; Thu, 19 Jul 2012 10:35:28 +0800
From: Songhaibin <haibin.song@huawei.com>
To: "arno@cs.vu.nl" <arno@cs.vu.nl>, Peng Zhang <pzhang.thu@gmail.com>
Thread-Topic: [decade] [ppsp] Object naming in -req and -arch
Thread-Index: AQHNYGz9SgP+UqsLmUq+tORnpn0WHZcmRtaAgAmX1jA=
Date: Thu, 19 Jul 2012 02:35:29 +0000
Message-ID: <E33E01DFD5BEA24B9F3F18671078951F23AF0FE9@szxeml534-mbx.china.huawei.com>
References: <20120710162606039401143@chinamobile.com> <2039343B-5F6B-4777-864E-B4F00B5A258E@gmail.com> <4FFE6D67.80705@cs.vu.nl> <D282B585-33DD-4A0D-8CD3-9CF525C56446@gmail.com> <4FFFCA6D.7060306@cs.vu.nl>
In-Reply-To: <4FFFCA6D.7060306@cs.vu.nl>
Accept-Language: en-US, zh-CN
Content-Language: zh-CN
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [10.138.41.73]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-CFilter-Loop: Reflected
X-Mailman-Approved-At: Wed, 18 Jul 2012 19:51:06 -0700
Cc: ppsp <ppsp@ietf.org>, decade <decade@ietf.org>
Subject: Re: [ppsp] [decade] Object naming in -req and -arch
X-BeenThere: ppsp@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: discussing to draw up peer to peer streaming protocol <ppsp.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ppsp>, <mailto:ppsp-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/ppsp>
List-Post: <mailto:ppsp@ietf.org>
List-Help: <mailto:ppsp-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ppsp>, <mailto:ppsp-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 19 Jul 2012 02:37:57 -0000

I know little about Merkle Hash Tree. So I searched and learned some from the Internet. My feeling is MHT is useful for pre-stored objects. But it is less useful for live contents. IMO, the co-relation of the objects besides naming the object itself adds extra complexity in DECADE environment. Or is there any good reason to use it?

Just my 2 cents,
-Haibin

> -----Original Message-----
> From: decade-bounces@ietf.org [mailto:decade-bounces@ietf.org] On Behalf Of
> Arno Bakker
> Sent: Friday, July 13, 2012 3:13 PM
> To: Peng Zhang
> Cc: ppsp; decade
> Subject: Re: [decade] [ppsp] Object naming in -req and -arch
> 
> On 12/07/2012 22:28, Peng Zhang wrote:
> >
> > On Jul 12, 2012, at 2:23 AM, Arno Bakker wrote:
> >
> >> The gains of using MHT depend on the chunk size. For PPSP we prefer
> >> chunks of 1K that fit in an UDP packet carried over Ethernet. In
> >> that case, for a 4 GB file, there are 4 M chunks, resulting in 80
> >> MB of leaf hashes when SHA1 is used. Transferring that beforehand
> >> as in BitTorrent definitely increases latency ;o)
> > Yes, if the chunk size is only 1KB, and each chunk is verified
> > individually, we cannot afford to send all hashes beforehand. While
> > in the worst case without optimization, almost 2*80M = 160M hashes
> > needs to be sent to the receiver, will that be a large overhead
> > compared to 4G? Do we really need such a small chunk size? Maybe I
> > miss some previous discussion on this.
> 
> Hi
> 
> For PPSP we want to use UDP as we don't need the in-order and
> reliability features of TCP, and want flexibility to use differnet
> congestion control algorithms and handle NATs. With Ethernet as the
> dominant MAC layer at present and an unreliable transport we don't want
> datagrams to exceed the Ethernet MTU, otherwise the chance of losing a
> datagram increases (an UDP packet taking N IP packets will not be
> delivered when only 1 IP packet is lost). Hence, we use chunks of ~1K.
> 
> A good practice in P2P networks is to not forward data you have not
> verified. So to forward the 1K chunks directly we need to be able to
> verify their integrity at this granularity, enter Merkle Hash Trees.
> We think the resulting overhead due to the size of the tree is
> acceptable, as it is easy to optimize the number of hashes transmitted
> in our use cases.
> 
> CU,
>      Arno