Re: [dtn] BPv7 CDDL and CBOR tagging

Brian Sipos <BSipos@rkf-eng.com> Wed, 10 April 2019 16:50 UTC

Return-Path: <BSipos@rkf-eng.com>
X-Original-To: dtn@ietfa.amsl.com
Delivered-To: dtn@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2CDC0120621 for <dtn@ietfa.amsl.com>; Wed, 10 Apr 2019 09:50:57 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.899
X-Spam-Level:
X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=rkfeng.onmicrosoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gfd_cQkqURt3 for <dtn@ietfa.amsl.com>; Wed, 10 Apr 2019 09:50:53 -0700 (PDT)
Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-eopbgr770083.outbound.protection.outlook.com [40.107.77.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B4EBE120632 for <dtn@ietf.org>; Wed, 10 Apr 2019 09:50:52 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rkfeng.onmicrosoft.com; s=selector1-rkfeng-com0i; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PSR6EpzdDjLwsFLlNeF5V78GtwHhs1XrXgFYjtIbMGE=; b=hwowWq4Nj1ibKG0hzaCx0c5Rd+4okF3PFFybGgmpPCix1bIMKqSurFN5hk0uUlTYW9dOFNOQcXgpucXN+vufgUR7/IuVj8lhwt3Ifl5ljsZtKcgpfx5GFIGuByOFLJUbed0Sm/oY24bhtCW3jAeXtai1GeaC3PgApThLbmHGH0E=
Received: from CY4PR1301MB2039.namprd13.prod.outlook.com (10.171.240.14) by CY4PR1301MB2135.namprd13.prod.outlook.com (10.171.240.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1792.10; Wed, 10 Apr 2019 16:50:48 +0000
Received: from CY4PR1301MB2039.namprd13.prod.outlook.com ([fe80::c72:6b85:66ca:845c]) by CY4PR1301MB2039.namprd13.prod.outlook.com ([fe80::c72:6b85:66ca:845c%3]) with mapi id 15.20.1792.009; Wed, 10 Apr 2019 16:50:48 +0000
From: Brian Sipos <BSipos@rkf-eng.com>
To: "Burleigh, Scott C (312B)" <scott.c.burleigh=40jpl.nasa.gov@dmarc.ietf.org>, Carsten Bormann <cabo@tzi.org>
CC: "dtn@ietf.org" <dtn@ietf.org>
Thread-Topic: [dtn] BPv7 CDDL and CBOR tagging
Thread-Index: AQHU6cwkKjg/bVi0S06SSCrlH9F6SKYvG12AgAPhHQCAABXEAIABoVmAgADXHoCAABzBAA==
Date: Wed, 10 Apr 2019 16:50:48 +0000
Message-ID: <3b3fc65dc0f5df2af6d598fc13532b53163f461c.camel@rkf-eng.com>
References: <CY4PR1301MB20399B0DD6B59D2D566257379F570@CY4PR1301MB2039.namprd13.prod.outlook.com> <B8DF4499-2EEC-4680-B1AD-9FEF00A907D7@tzi.org> <6773e0f0e03e6e4ddb935f0f02de7ade2bc606b8.camel@rkf-eng.com> <670af7e65e8c4746947b0833690f4625@jpl.nasa.gov> <1069e920de24c960ca46b3172d99db44802a3b2b.camel@rkf-eng.com> <77a455395d2c4495986cceeef84edd8e@jpl.nasa.gov>
In-Reply-To: <77a455395d2c4495986cceeef84edd8e@jpl.nasa.gov>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [38.100.63.114]
x-mailer: Evolution 3.28.5 (3.28.5-3.fc28)
x-clientproxiedby: DM5PR05CA0018.namprd05.prod.outlook.com (2603:10b6:3:d4::28) To CY4PR1301MB2039.namprd13.prod.outlook.com (2603:10b6:910:48::14)
authentication-results: spf=none (sender IP is ) smtp.mailfrom=BSipos@rkf-eng.com;
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: bacf96ac-a170-484d-d4ad-08d6bdd4ab2f
x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(7021145)(8989299)(4534185)(7022145)(4603075)(4627221)(201702281549075)(8990200)(7048125)(7024125)(7027125)(7023125)(5600139)(711020)(4605104)(2017052603328)(7193020); SRVR:CY4PR1301MB2135;
x-ms-traffictypediagnostic: CY4PR1301MB2135:
x-ms-exchange-purlcount: 1
x-microsoft-antispam-prvs: <CY4PR1301MB2135C7F964352C9D1AC774109F2E0@CY4PR1301MB2135.namprd13.prod.outlook.com>
x-forefront-prvs: 00032065B2
x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(346002)(366004)(39830400003)(376002)(396003)(136003)(13464003)(189003)(54094003)(199004)(81166006)(81156014)(316002)(76176011)(102836004)(186003)(6506007)(30864003)(229853002)(86362001)(26005)(99286004)(97736004)(486006)(52116002)(53546011)(110136005)(6512007)(80792005)(25786009)(7736002)(6246003)(71200400001)(71190400001)(6486002)(236005)(54896002)(8676002)(6306002)(8936002)(4326008)(6436002)(386003)(446003)(50226002)(118296001)(11346002)(2616005)(53936002)(476003)(105586002)(606006)(68736007)(256004)(5660300002)(6116002)(3846002)(36756003)(2906002)(14454004)(66066001)(72206003)(508600001)(93886005)(966005)(106356001)(14444005)(99106002); DIR:OUT; SFP:1101; SCL:1; SRVR:CY4PR1301MB2135; H:CY4PR1301MB2039.namprd13.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1;
received-spf: None (protection.outlook.com: rkf-eng.com does not designate permitted sender hosts)
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam-message-info: 2d1mzBxwmTHsG3OSKLcp4qF6w7xgnrWhMOVpkgGHENwNqqXnFBF1cPqM2Yp34kq6Uncts6hcDnomkM0hlAa6U11/t2YiS8O27j/PSXDusMtfRJLW2iHob90fA7TV91ty6givlNZl8dO1osP/B5qz13mNkHmjGPPGmo2Px8dmMRq42pF3tTqOCeqe3l868LUAmRkF8Xwom10J6Hp+6M+1u3jLALo7RZGB0fbppru1H+P0AWTMDzwAKIwFxq5fVEe2IlW/ebs1x5IpIuQLDk89p1g0LEUpDzzSbTeBBoS6/Ph+itowCCHaTG38C8/LXQAwIUeDXW/+Tv4HX58ZWEGsPPInSZvwu+r2MGYLW7GsdnoT5RTDYbdQs8naJgFw8KNo7bR597QU4GcP0+zrtM+f9GXLNHct0ijphTla7mz2P4Y=
Content-Type: multipart/alternative; boundary="_000_3b3fc65dc0f5df2af6d598fc13532b53163f461ccamelrkfengcom_"
MIME-Version: 1.0
X-OriginatorOrg: rkf-eng.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bacf96ac-a170-484d-d4ad-08d6bdd4ab2f
X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Apr 2019 16:50:48.6914 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4ed8b15b-911f-42bc-8524-d89148858535
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR1301MB2135
Archived-At: <https://mailarchive.ietf.org/arch/msg/dtn/savgXuRElQlG2zjqnhQ1uh9rTH8>
Subject: Re: [dtn] BPv7 CDDL and CBOR tagging
X-BeenThere: dtn@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Delay Tolerant Networking \(DTN\) discussion list at the IETF." <dtn.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/dtn>, <mailto:dtn-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/dtn/>
List-Post: <mailto:dtn@ietf.org>
List-Help: <mailto:dtn-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/dtn>, <mailto:dtn-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 10 Apr 2019 16:51:00 -0000

Scott and Carsten,
I'm not as familiar with some newish behavior of CDDL as I would like to be, so didn't even realize the Generics behavior was available. I agree with the last that the canonical block should be written as a generic that the payload/extension blocks use.

Below is an updated CDDL which simplifies things and adds back in the total application data unit length field. It fully validates randomly generated bundles from a different application, which is a good sign. I also trimmed the line length to fit within 80 characters to avoid some formatting issues. I'm adding CODE tags similar to recommended in RFC8407, I wonder if something similar would be useful for CDDL embeds...


<CODE BEGINS> file "bpv7.cddl"


start = bundle / #6.55799(bundle)


; Times before 2000 are invalid

dtn-time = uint


; CRC enumerated type

crc-type = 0 / 1 / 2

; Either 16-bit or 32-bit

crc-value = (bstr .size 2) / (bstr .size 4)


creation-timestamp = [dtn-time, sequence: uint]


eid = $eid-choice .within eid-structure

eid-structure = [

  uri-code: uint,

  SSP: any

]

$eid-choice /= [

  uri-code: 1,

  SSP: (tstr / 0)

]

$eid-choice /= [

  uri-code: 2,

  SSP: [

    nodenum: uint,

    servicenum: uint

  ]

]


; The root bundle array

bundle = [primary-block, *extension-block, payload-block]


primary-block = [

  version: 7,

  bundle-control-flags,

  crc-type,

  destination: eid,

  source-node: eid,

  report-to: eid,

  creation-timestamp,

  lifetime: uint,

  ? (

    fragment-offset: uint,

    total-application-data-length: uint,

  )

  ? crc-value,

]

bundle-control-flags = uint .bits bundleflagbits

bundleflagbits = &(

  reserved: 15,

  reserved: 14,

  reserved: 13,

  bundle-deletion-status-reports-are-requested: 12,

  bundle-delivery-status-reports-are-requested: 11,

  bundle-forwarding-status-reports-are-requested: 10,

  reserved: 9,

  bundle-reception-status-reports-are-requested: 8,

  bundle-contains-a-Manifest-block: 7,

  status-time-is-requested-in-all-status-reports: 6,

  user-application-acknowledgement-is-requested: 5,

  reserved: 4,

  reserved: 3,

  bundle-must-not-be-fragmented: 2,

  payload-is-an-administrative-record: 1,

  bundle-is-a-fragment: 0

)


; Abstract shared structure of all non-primary blocks

canonical-block-structure = [

  block-type-code: uint,

  block-number: uint,

  block-control-flags,

  crc-type,

  ; Each block type defines the content within the bytestring

  block-type-specific-data,

  ? crc-value

]

block-control-flags = uint .bits blockflagbits

blockflagbits = &(

  reserved: 7,

  reserved: 6,

  reserved: 5,

  reserved: 4,

  bundle-must-be-deleted-if-block-cannot-be-processed: 3,

  status-report-must-be-transmitted-if-block-cannot-be-processed: 2,

  block-must-be-removed-from-bundle-if-it-cannot-be-processed: 1,

  block-must-be-replicated-in-every-fragment: 0

)

block-type-specific-data = bstr / #6.24(bstr)

; Actual CBOR data embedded in a bytestring, with optional tag to indicate so

embedded-cbor<Item> = (bstr .cbor Item) / #6.24(bstr .cbor Item)


; Extension block type, which does not specialize other than the code/number

extension-block = $extension-block-structure .within canonical-block-structure

; Generic shared structure of all non-primary blocks

extension-block-use<CodeValue, BlockData> = [

  block-type-code: CodeValue,

  block-number: (uint .ne 0),

  block-control-flags,

  crc-type,

  BlockData,

  ? crc-value

]


; Payload block type

payload-block = payload-block-structure .within canonical-block-structure

payload-block-structure = [

  block-type-code: 1,

  block-number: 0,

  block-control-flags,

  crc-type,

  $payload-block-data,

  ? crc-value

]


; Arbitrary payload data, including non-CBOR bytestring

$payload-block-data /= block-type-specific-data



; Administrative record as a payload data specialization

$payload-block-data /= embedded-cbor<admin-record>

admin-record = $admin-record .within admin-record-structure

admin-record-structure = [

  record-type-code: uint,

  record-content: any

]

; Only one defined record type

$admin-record /= [1, status-record-content]

status-record-content = [

  bundle-status-information,

  status-report-reason-code: uint,

  source-node-eid: eid,

  subject-creation-timestamp: creation-timestamp,

  ? (

    subject-payload-offset: uint,

    subject-payload-length: uint

  )

]

bundle-status-information = [

  reporting-node-received-bundle: status-info-content,

  reporting-node-forwarded-bundle: status-info-content,

  reporting-node-delivered-bundle: status-info-content,

  reporting-node-deleted-bundle: status-info-content

]

status-info-content = [

  status-indicator: bool,

  ? timestamp: dtn-time

]



; Previous Node extension block

$extension-block-structure /=

  extension-block-use<7, embedded-cbor<ext-data-previous-node>>

ext-data-previous-node = eid


; Bundle Age extension block

$extension-block-structure /=

  extension-block-use<8, embedded-cbor<ext-data-bundle-age>>

ext-data-bundle-age = uint


; Hop Count extension block

$extension-block-structure /=

  extension-block-use<9, embedded-cbor<ext-data-hop-count>>

ext-data-hop-count = [

  hop-limit: uint,

  hop-count: uint

]


<CODE ENDS>



On Wed, 2019-04-10 at 15:07 +0000, Burleigh, Scott C (312B) wrote:
Brian, thanks, this is great!  I'll insert the revised CDDL and re-post.

I think "total application data unit length" was erroneously deleted from section 4.2.2 a couple of revs back, but it was restored as of version 13 (January).

Scott

-----Original Message-----
From: Brian Sipos <BSipos@rkf-eng.com<mailto:BSipos@rkf-eng.com>>
Sent: Tuesday, April 9, 2019 7:18 PM
To: Burleigh, Scott C (312B) <scott.c.burleigh@jpl.nasa.gov<mailto:scott.c.burleigh@jpl.nasa.gov>>; Carsten Bormann <cabo@tzi.org<mailto:cabo@tzi.org>>
Cc: dtn@ietf.org<mailto:dtn@ietf.org>
Subject: [EXTERNAL] Re: [dtn] BPv7 CDDL and CBOR tagging

Scott,
At the bottom of this email is a full CDDL that captures both the encodings of extension / administrative data as well as the binding between extension data types and extension block numbers.
I also added restrictions on CRC type code and CRC value data, and used the socket convention and the "-structure" convention from the pre-RFC CDDL draft.

I'll leave up to you to decide if this is an over-specialized use of CDDL. Using the reference "cddl" program to generate randomized data actually produces near-to-reasonable bundles to my eye, which is nice validation of the CDDL itself.

The reference program can be tested as in:
$ gem install cddl
$ cddl bpv7.cddl.txt generate
[[7, 17174, 0, [2, [3026, 451]], [1, 0], [2, [535, 2091]], [4044, 2891], 1070, h'FCDD'], [8, 2593, 229, 2, h'190836', h'C4F3'], [7, 2533, 177, 2, h'820100', h'0504'], [8, 3239, 179, 2, h'1850'], [8, 398, 237, 2, h'183E', h'31261138'], [1, 0, 175, 0, h'8201848481F581F481F481F51901C6820166766F6C75707482190704190CDA',
h'AFA5']]

One issue that I ran into when testing this: I used this CDDL to validate some test bundles which I had been generating separately to test the TCPCL agent. The original CDDL in the draft contains the field name "total-application-data-length" which also appears several times in the draft as "total application data unit length" in the statments.
But I don't see that field listed in Section 4.2.2. Is there a missing field in that section, or is that field no longer present and the other text is inaccurate?

Thanks,
Brian S.

On Tue, 2019-04-09 at 01:24 +0000, Burleigh, Scott C (312B) wrote:
Guys, please feel free to send me an updated CDDL expression of Bundle
Protocol version 7; I will gladly include it in the document.

My one comment is that the type dtn-time is correctly defined as uint;
it should NOT be changed to int, as negative values are not allowed.
If you can spot an instance in the spec where dtn-time is
characterized as a signed integer, please let me know and I will
correct it.

Scott

start = bundle / #6.55799(bundle)

; Times before 2000 are invalid
dtn-time = uint

; CRC enumerated type
crc-type = 0 / 1 / 2
; Either 16-bit or 32-bit
crc-value = (bstr .size 2) / (bstr .size 4)

creation-timestamp = [dtn-time, sequence: uint]

eid = $eid-choice .within eid-structure
eid-structure = [
  uri-code: uint,
  SSP: any
]
$eid-choice /= [
  uri-code: 1,
  SSP: (tstr / 0)
]
$eid-choice /= [
  uri-code: 2,
  SSP: [
    nodenum: uint,
    servicenum: uint
  ]
]

bundle-control-flags = uint .bits bundleflagbits bundleflagbits = &(
  reserved: 15
  reserved: 14
  reserved: 13
  bundle-deletion-status-reports-are-requested: 12
  bundle-delivery-status-reports-are-requested: 11
  bundle-forwarding-status-reports-are-requested: 10
  reserved: 9
  bundle-reception-status-reports-are-requested: 8
  bundle-contains-a-Manifest-block: 7
  status-time-is-requested-in-all-status-reports: 6
  user-application-acknowledgement-is-requested: 5
  reserved: 4
  reserved: 3
  bundle-must-not-be-fragmented: 2
  payload-is-an-administrative-record: 1
  bundle-is-a-fragment: 0
)

block-control-flags = uint .bits blockflagbits blockflagbits = &(
  reserved: 7
  reserved: 6
  reserved: 5
  reserved: 4
  bundle-must-be-deleted-if-block-cannot-be-processed: 3
  status-report-must-be-transmitted-if-block-cannot-be-processed: 2
  block-must-be-removed-from-bundle-if-it-cannot-be-processed: 1
  block-must-be-replicated-in-every-fragment: 0
)

bundle = [primary-block, *extension-block, payload-block]

primary-block = [
  version: 7,
  bundle-control-flags,
  crc-type,
  destination: eid,
  source-node: eid,
  report-to: eid,
  creation-timestamp,
  lifetime: uint,
  ? (
    fragment-offset: uint,
;    total-application-data-length: uint,
  )
  ? crc-value,
]

; Abstract shared structure of all non-primary blocks canonical-block-structure = [
  block-type-code: uint,
  block-number: uint,
  block-control-flags,
  crc-type,
  ; Each block type defines the content within the bytestring
  block-type-specific-data,
  ? crc-value
]
block-type-specific-data = bstr / #6.24(bstr) ; Extension block type, which does not specialize any more than the code/number extension-block = $extension-block-structure .within canonical-block- structure $extension-block-structure = [
  block-type-code: (uint .gt 1),
  extension-block-number,
  block-control-flags,
  crc-type,
  ($extension-block-data .within block-type-specific-data)
  ? crc-value
]
extension-block-number = (uint .ne 0)
; Payload block type
payload-block = payload-block-structure .within canonical-block- structure payload-block-structure = [
  block-type-code: 1,
  block-number: 0,
  block-control-flags,
  crc-type,
  ($payload-block-data .within block-type-specific-data)
  ? crc-value
]

; Arbitrary payload data
$payload-block-data /= bstr


; Administrative record as a payload data specialization $payload-block-data /= block-type-specific-data .cbor admin-record admin-record = $admin-record .within admin-record-structure admin-record-structure = [
  record-type-code: uint,
  record-content: any
]
; Only one defined record type
$admin-record /= [1, status-record-content] status-record-content = [
  bundle-status-information,
  status-report-reason-code: uint,
  source-node-eid: eid,
  subject-creation-timestamp: creation-timestamp,
  ? (
    subject-payload-offset: uint,
    subject-payload-length: uint
  )
]
bundle-status-information = [
  reporting-node-received-bundle: status-info-content,
  reporting-node-forwarded-bundle: status-info-content,
  reporting-node-delivered-bundle: status-info-content,
  reporting-node-deleted-bundle: status-info-content ] status-info-content = [
  status-indicator: bool,
  ? timestamp: dtn-time
]

; Previous Node extension block
$extension-block-structure /= [
  block-type-code: 7,
  extension-block-number,
  block-control-flags,
  crc-type,
  (block-type-specific-data .cbor ext-data-previous-node)
  ? crc-value
]
ext-data-previous-node = eid

; Bundle Age extension block
$extension-block-structure /= [
  block-type-code: 8,
  extension-block-number,
  block-control-flags,
  crc-type,
  (block-type-specific-data .cbor ext-data-bundle-age)
  ? crc-value
]
ext-data-bundle-age = uint

; Hop Count extension block
$extension-block-structure /= [
  block-type-code: 9,
  extension-block-number,
  block-control-flags,
  crc-type,
  (block-type-specific-data .cbor ext-data-hop-count)
  ? crc-value
]
ext-data-hop-count = [
  hop-limit: uint,
  hop-count: uint
]

_______________________________________________
dtn mailing list
dtn@ietf.org<mailto:dtn@ietf.org>
https://www.ietf.org/mailman/listinfo/dtn