Re: [decade] [ppsp] Object naming in -req and -arch

Arno Bakker <arno@cs.vu.nl> Thu, 12 July 2012 06:23 UTC

Return-Path: <a.bakker@vu.nl>
X-Original-To: decade@ietfa.amsl.com
Delivered-To: decade@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3968321F854E; Wed, 11 Jul 2012 23:23:09 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.543
X-Spam-Level:
X-Spam-Status: No, score=-1.543 tagged_above=-999 required=5 tests=[AWL=-0.039, BAYES_00=-2.599, HELO_EQ_NL=0.55, HOST_EQ_NL=1.545, RCVD_IN_DNSWL_LOW=-1]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IcVisQMk2yKO; Wed, 11 Jul 2012 23:23:08 -0700 (PDT)
Received: from mailin.vu.nl (mailin.vu.nl [130.37.164.18]) by ietfa.amsl.com (Postfix) with ESMTP id 70CEC21F8522; Wed, 11 Jul 2012 23:23:08 -0700 (PDT)
Received: from PEXHB012B.vu.local (130.37.236.67) by mailin.vu.nl (130.37.164.18) with Microsoft SMTP Server (TLS) id 14.2.298.4; Thu, 12 Jul 2012 08:23:38 +0200
Received: from [109.37.58.145] (130.37.238.20) by mails.vu.nl (130.37.236.67) with Microsoft SMTP Server (TLS) id 14.2.298.4; Thu, 12 Jul 2012 08:23:37 +0200
Message-ID: <4FFE6D67.80705@cs.vu.nl>
Date: Thu, 12 Jul 2012 08:23:35 +0200
From: Arno Bakker <arno@cs.vu.nl>
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:11.0) Gecko/20120312 Thunderbird/11.0
MIME-Version: 1.0
To: Peng Zhang <pzhang.thu@gmail.com>
References: <20120710162606039401143@chinamobile.com> <2039343B-5F6B-4777-864E-B4F00B5A258E@gmail.com>
In-Reply-To: <2039343B-5F6B-4777-864E-B4F00B5A258E@gmail.com>
Content-Type: text/plain; charset="ISO-8859-15"; format="flowed"
Content-Transfer-Encoding: 7bit
X-Originating-IP: [130.37.238.20]
X-Mailman-Approved-At: Thu, 12 Jul 2012 07:57:33 -0700
Cc: ppsp <ppsp@ietf.org>, decade <decade@ietf.org>
Subject: Re: [decade] [ppsp] Object naming in -req and -arch
X-BeenThere: decade@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
Reply-To: arno@cs.vu.nl
List-Id: "To start the discussion on DECoupled Application Data Enroute, to discuss the in-network data storage for p2p applications and its access protocol" <decade.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/decade>, <mailto:decade-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/decade>
List-Post: <mailto:decade@ietf.org>
List-Help: <mailto:decade-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/decade>, <mailto:decade-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Jul 2012 06:23:09 -0000

Hi Peng

On 11/07/2012 23:25, Peng Zhang wrote:
> Hi Arno,
>
> Thanks for the clarification. In my understanding, MHT over single hash
> can enable immediate integrity check before the whole media file is
> received, which is critical for streaming applications. But the client
> can also download hashes of every piece, just like in BitTorrent. Does
> it reduce a lot startup latency or server load by using MHT? Thanks.
>

The gains of using MHT depend on the chunk size. For PPSP we prefer 
chunks of 1K that fit in an UDP packet carried over Ethernet. In that 
case, for a 4 GB file, there are 4 M chunks, resulting in 80 MB of leaf 
hashes when SHA1 is used. Transferring that beforehand as in BitTorrent 
definitely increases latency ;o)


> For DECADE, It supports integrity check on piece/chunk level, so that
> client can verify that the received piece corresponds to the piece name.
> DECADE is unaware of what file this piece belongs to, and thus does not
> provide end-to-end integrity check. For file level, we leave the
> integrity check to applications. Thus, imo, we should not include MHT in
> the design of DECADE. But MHT can be built on top of DECADE, which means
> applications can still use MHT to implement integrity check for themselves.
>

You are right, if DECADE doesn't provide "links" between chunks, MHTs at 
the DECADE level make no sense. Using MHTs at the app level instead as 
you say does make sense and is easy, just build all the piece 
names/piece hashes into a tree.

Regards,
       Arno