Re: [dtn-interest] DTN architecture issues for handling Huge Content Objects

"Burleigh, Scott C (313B)" <scott.c.burleigh@jpl.nasa.gov> Thu, 06 December 2012 20:46 UTC

Return-Path: <scott.c.burleigh@jpl.nasa.gov>
X-Original-To: dtn-interest@ietfa.amsl.com
Delivered-To: dtn-interest@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0332921F8734 for <dtn-interest@ietfa.amsl.com>; Thu, 6 Dec 2012 12:46:11 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.599
X-Spam-Level:
X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fJcoOkmKeiGY for <dtn-interest@ietfa.amsl.com>; Thu, 6 Dec 2012 12:46:09 -0800 (PST)
Received: from mail.jpl.nasa.gov (mailhost.jpl.nasa.gov [128.149.139.106]) by ietfa.amsl.com (Postfix) with ESMTP id D586421F8798 for <dtn-interest@irtf.org>; Thu, 6 Dec 2012 12:46:09 -0800 (PST)
Received: from mail.jpl.nasa.gov (ap-ehub-sp02.jpl.nasa.gov [128.149.137.149]) by smtp.jpl.nasa.gov (Sentrion-MTA-4.2.2/Sentrion-MTA-4.2.2) with ESMTP id qB6Kk43D005007 (using TLSv1/SSLv3 with cipher AES128-SHA (128 bits) verified NO); Thu, 6 Dec 2012 12:46:05 -0800
Received: from AP-EMBX-SP40.RES.AD.JPL ([169.254.7.220]) by ap-ehub-sp02.RES.AD.JPL ([fe80::dd85:7b07:1e36:7e3c%15]) with mapi id 14.02.0318.001; Thu, 6 Dec 2012 12:46:04 -0800
From: "Burleigh, Scott C (313B)" <scott.c.burleigh@jpl.nasa.gov>
To: John Zinky <jzinky@bbn.com>, "dtn-interest@irtf.org" <dtn-interest@irtf.org>
Thread-Topic: [dtn-interest] DTN architecture issues for handling Huge Content Objects
Thread-Index: AQHN0+6IPWIx70ssaE64g6FYYD+MBZgMNbJw
Date: Thu, 06 Dec 2012 20:46:03 +0000
Message-ID: <A5BEAD028815CB40A32A5669CF737C3B0FA1072F@ap-embx-sp40.RES.AD.JPL>
References: <A5BEAD028815CB40A32A5669CF737C3B0FA0F473@ap-embx-sp40.RES.AD.JPL> <CCE53432.D259%william.d.ivancic@nasa.gov> <A5BEAD028815CB40A32A5669CF737C3B0FA0F77F@ap-embx-sp40.RES.AD.JPL> <CC504B2B-3965-4CCC-812E-5405BB47B721@bbn.com>
In-Reply-To: <CC504B2B-3965-4CCC-812E-5405BB47B721@bbn.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [128.149.137.113]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Source-Sender: scott.c.burleigh@jpl.nasa.gov
X-AUTH: Authorized
Subject: Re: [dtn-interest] DTN architecture issues for handling Huge Content Objects
X-BeenThere: dtn-interest@irtf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "The Delay-Tolerant Networking Research Group \(DTNRG\) - Announce." <dtn-interest.irtf.org>
List-Unsubscribe: <https://www.irtf.org/mailman/options/dtn-interest>, <mailto:dtn-interest-request@irtf.org?subject=unsubscribe>
List-Archive: <http://www.irtf.org/mail-archive/web/dtn-interest>
List-Post: <mailto:dtn-interest@irtf.org>
List-Help: <mailto:dtn-interest-request@irtf.org?subject=help>
List-Subscribe: <https://www.irtf.org/mailman/listinfo/dtn-interest>, <mailto:dtn-interest-request@irtf.org?subject=subscribe>
X-List-Received-Date: Thu, 06 Dec 2012 20:46:11 -0000

To clarify what I was suggesting: BP fragmentation can occur at any point on the end-to-end path, and there's nothing in RFC5050 that says that fragmentation has to result in multiple segments that are all of the same size.  This means it's never necessary for the source to know any sort of global MTU size that applies to the whole path.  When a bundle (which may itself be a fragment) residing at any node along the path (not just the source) is about to be forwarded over a link for which it is too large (as determined by the forwarding BPA from local knowledge, acquired however), the BPA can divide that bundle into two fragments: fragment A that is small enough to be efficiently forwarded over this specific link at this particular moment in time, and fragment B that is the rest of the bundle.  Fragment A can then be forwarded, whereupon the same consideration can then be given to fragment B, and so on.  This is generally efficient, highly adaptive, sensitive to even rapidly changing local conditions, and simple.

In CCSDS we've got the beginnings of a spec for something called "Delay-Tolerant Payload Conditioning" (DTPC) which would be the common site for a variety of optional end-to-end services, somewhat TCP-like.  It would be a standardized application service layer -- similar to a transport layer -- running immediately above BP.  (Not quite ready to post as an Internet Draft, but soon, I hope.)  We've thought about adding erasure coding as a DTPC service, which I suspect would have much the same effect as the procedure proposed in the Erasure Coding draft.  I think running LTP over an erasure-coded link service also would make a lot of sense, and in fact the two would not be mutually exclusive.

Scott

-----Original Message-----
From: dtn-interest-bounces@irtf.org [mailto:dtn-interest-bounces@irtf.org] On Behalf Of John Zinky
Sent: Thursday, December 06, 2012 12:16 PM
To: dtn-interest@irtf.org
Subject: Re: [dtn-interest] DTN architecture issues for handling Huge Content Objects

A deployment context has an effective Maximum Transmission Unit (MTU), whether the limitation comes from a CLA (such as the UDP-Link CLA used on radio broadcast channels) or Contact time. If the source is somehow privy to this runtime constraint (MTU metadata), the Data Object can be divided into appropriate size bundles proactively. As with all QoS adaption the key is to dynamically identify the situation at runtime (effective MTU) and apply the appropriate mechanism (Fragmentation or Encoding). One size does not fit all.

The Eraser Coding draft and the DTN 2.9 implementation offers a different mechanism of reactive fragmentation.  Erasure Coding can be applied to a Big Bundle inside the BPA. The Big Bundle wire-format is treated as a Data Object, encoded by the source BPA,  the Encoding Bundles sent to a destination BPA that decodes the Big Bundle, and the destination reinserts the bundle into the DTN. The group of bundles acts like a tunnel to get the Big Bundle from one place in the DTN to another. The Destination can be chosen based on the runtime situation and does not have to be the same BPA as the destination EID. See Slide 9 in:
  https://dist-systems.bbn.com/papers/2011/Zinky/ErasureCodingExtBlockSpec-IRTF-2011-07-25-Rev1-2-2.pdf



On Dec 5, 2012, at 8:28 PM, "Fall, Kevin" <kfall@qti.qualcomm.com> wrote:

> The DTN fragmentation isn't intended for the same purposes as IP 
> fragmentation. While IP fragmentation is about adapting an abstract datagram (IP datagram) to be carried on various underlying link layers, DTN fragmentation is about dividing a big thing into chunks appropriate to contacts (for proactive fragmentation) and to help in recovery for unexpected outages (for reactive).


On Dec 5, 2012, at 8:54 PM, "Burleigh, Scott C (313B)" <scott.c.burleigh@jpl.nasa.gov> wrote:

> But I'd argue that you can do just as well with proactive fragmentation within the network.  "Would be nice" implies that there is some value in retransmitting 3 MB instead of all 10 MB of the bundle.  How much value is that?  
...
> Okay, suppose your link is only 4800 bps.  Now the extra cost of not 
> doing reactive fragmentation is 11,667 seconds of transmission -- 
> unless you do proactive fragmentation at the time you start sending 
> the bundle, i.e., you break off 600-byte fragments of the bundle one 
> at a time and send them over the link.  Now when the link fails the 
> cost of not doing reactive fragmentation is one entire bundle,
...
> This is of course a topic on which we've been arguing for over a decade in DTNRG and I am confident that I haven't changed many minds here.  I just want to suggest that it's possible to engineer DTN to be relatively efficient over an opportunistic contact without jettisoning bundle authentication.

_______________________________________________
dtn-interest mailing list
dtn-interest@irtf.org
https://www.irtf.org/mailman/listinfo/dtn-interest