Re: [ppsp] [decade] Object naming in -req and -arch

Peng Zhang <pzhang.thu@gmail.com> Thu, 12 July 2012 20:28 UTC

Return-Path: <pzhang.thu@gmail.com>
X-Original-To: ppsp@ietfa.amsl.com
Delivered-To: ppsp@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2CBBE11E8096; Thu, 12 Jul 2012 13:28:20 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.304
X-Spam-Level:
X-Spam-Status: No, score=-3.304 tagged_above=-999 required=5 tests=[AWL=0.295, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id edR6Ao8qCdjY; Thu, 12 Jul 2012 13:28:19 -0700 (PDT)
Received: from mail-qa0-f51.google.com (mail-qa0-f51.google.com [209.85.216.51]) by ietfa.amsl.com (Postfix) with ESMTP id 18F9F11E8080; Thu, 12 Jul 2012 13:28:19 -0700 (PDT)
Received: by qaea16 with SMTP id a16so1929456qae.10 for <multiple recipients>; Thu, 12 Jul 2012 13:28:53 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=wT7m4tPFgv3SqXzFZ+I+dR7he8wP5J2g2KzPRXVw0KE=; b=eY4fKZurtz/hSkeKU40jPZB68EgyqoIkSDfJjbdEorJkjEIE0gSD+xLnK5fFe66YS9 PmALRsD7qKHNUYkf0MCbmaILKUsQp+AtCRsyeO3v5cvNg1f5W+phuFvVA6ssYZFZjRlA qrM6LYuRh2SsvQ/h7Qsf2XvUEkyW/0q0bTHVBfDadSi1exCaiEYEctBWF3A5HcCClfGL jgPmE/GGhHbQTvRgnxs2k2qA/EtXdMYviwC5YTHKbC6Oh3wY7+O3ktaa/tMvd4RJ6Byw K3bTgxHilX13u1gs4tn5x4k67rLL9DnuH5CFBcedDLNgaa+G4Yl2kQ1kapF+TYUe1/gL 9cjw==
Received: by 10.229.135.75 with SMTP id m11mr28743911qct.74.1342124932891; Thu, 12 Jul 2012 13:28:52 -0700 (PDT)
Received: from [128.36.160.220] ([128.36.160.220]) by mx.google.com with ESMTPS id w14sm3730489qaz.17.2012.07.12.13.28.51 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 12 Jul 2012 13:28:52 -0700 (PDT)
Mime-Version: 1.0 (Apple Message framework v1278)
Content-Type: text/plain; charset=us-ascii
From: Peng Zhang <pzhang.thu@gmail.com>
In-Reply-To: <4FFE6D67.80705@cs.vu.nl>
Date: Thu, 12 Jul 2012 16:28:50 -0400
Content-Transfer-Encoding: quoted-printable
Message-Id: <D282B585-33DD-4A0D-8CD3-9CF525C56446@gmail.com>
References: <20120710162606039401143@chinamobile.com> <2039343B-5F6B-4777-864E-B4F00B5A258E@gmail.com> <4FFE6D67.80705@cs.vu.nl>
To: <arno@cs.vu.nl>
X-Mailer: Apple Mail (2.1278)
X-Mailman-Approved-At: Sun, 15 Jul 2012 19:10:02 -0700
Cc: ppsp <ppsp@ietf.org>, decade <decade@ietf.org>
Subject: Re: [ppsp] [decade] Object naming in -req and -arch
X-BeenThere: ppsp@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: discussing to draw up peer to peer streaming protocol <ppsp.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ppsp>, <mailto:ppsp-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/ppsp>
List-Post: <mailto:ppsp@ietf.org>
List-Help: <mailto:ppsp-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ppsp>, <mailto:ppsp-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Jul 2012 20:28:20 -0000

On Jul 12, 2012, at 2:23 AM, Arno Bakker wrote:

> Hi Peng
> 
> On 11/07/2012 23:25, Peng Zhang wrote:
>> Hi Arno,
>> 
>> Thanks for the clarification. In my understanding, MHT over single hash
>> can enable immediate integrity check before the whole media file is
>> received, which is critical for streaming applications. But the client
>> can also download hashes of every piece, just like in BitTorrent. Does
>> it reduce a lot startup latency or server load by using MHT? Thanks.
>> 
> 
> The gains of using MHT depend on the chunk size. For PPSP we prefer chunks of 1K that fit in an UDP packet carried over Ethernet. In that case, for a 4 GB file, there are 4 M chunks, resulting in 80 MB of leaf hashes when SHA1 is used. Transferring that beforehand as in BitTorrent definitely increases latency ;o)
Yes, if the chunk size is only 1KB, and each chunk is verified individually, we cannot afford to send all hashes beforehand. While in the worst case without optimization, almost 2*80M = 160M hashes needs to be sent to the receiver, will that be a large overhead compared to 4G? Do we really need such a small chunk size? Maybe I miss some previous discussion on this.
> 
> 
>> For DECADE, It supports integrity check on piece/chunk level, so that
>> client can verify that the received piece corresponds to the piece name.
>> DECADE is unaware of what file this piece belongs to, and thus does not
>> provide end-to-end integrity check. For file level, we leave the
>> integrity check to applications. Thus, imo, we should not include MHT in
>> the design of DECADE. But MHT can be built on top of DECADE, which means
>> applications can still use MHT to implement integrity check for themselves.
>> 
> 
> You are right, if DECADE doesn't provide "links" between chunks, MHTs at the DECADE level make no sense. Using MHTs at the app level instead as you say does make sense and is easy, just build all the piece names/piece hashes into a tree.
Yes, Exactly.
> 
> Regards,
>      Arno