Re: [Deepspace] [dtn] technical remarks

sburleig.sb@gmail.com Tue, 26 September 2023 14:47 UTC

Return-Path: <sburleig.sb@gmail.com>
X-Original-To: deepspace@ietfa.amsl.com
Delivered-To: deepspace@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7D0E3C1519B0; Tue, 26 Sep 2023 07:47:52 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.102
X-Spam-Level:
X-Spam-Status: No, score=-2.102 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, LOTS_OF_MONEY=0.001, RCVD_IN_DNSWL_BLOCKED=0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lZ-3YwG1YzpR; Tue, 26 Sep 2023 07:47:47 -0700 (PDT)
Received: from mail-pf1-x434.google.com (mail-pf1-x434.google.com [IPv6:2607:f8b0:4864:20::434]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C3634C15199F; Tue, 26 Sep 2023 07:47:47 -0700 (PDT)
Received: by mail-pf1-x434.google.com with SMTP id d2e1a72fcca58-690d2e13074so6910870b3a.1; Tue, 26 Sep 2023 07:47:47 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695739666; x=1696344466; darn=ietf.org; h=thread-index:content-language:mime-version:message-id:date:subject :in-reply-to:references:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nyP8bnVU1vfCYgp/ntKiFNrvBrMPjkzZvP5R7aLFrwY=; b=Ui7kfoLq56IftF0OwD6+nuAqUU2NnDrybEQLQ9j4a+NK+wSbj0BU5FoC/7QCI8Nn7a WQhJ11cNsEOL58OilaILNGg87/bLBasabkthf/uGo4Z8+qBrEDHHOxAvMjhfW7ZO2lki AUVR2wRaK+/bIRPWqGtxqrJIWj2L2ot+EtHdbOlFY4suISa/mXZouWbELZA6TxxC5e0Q N78i0UbQEsz+uuVtHQezlqgz4iYOK1ziQRXwPLAXZzZTOmuRCohivYrK+4vbYCxo493w Xn7zHfD383hJ69a8YSK+DYjoQ5OZ7wufsTtUPugOR/5scd8Ku7h8ajdqvIAckUrdORWQ CsNA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695739666; x=1696344466; h=thread-index:content-language:mime-version:message-id:date:subject :in-reply-to:references:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nyP8bnVU1vfCYgp/ntKiFNrvBrMPjkzZvP5R7aLFrwY=; b=unCCYSMu6PxotMDnZG2igSaZRPSxQ6AwwIGlPrQYvbr3mb0zuYerHHO5INGIstSxcQ BLi316paAv7iq2n5DHCuW9eg7Kpor6xJHAmZ+e09YtPDQG5fBs6021WSAjDXtxdT3xtA TCuHcwSLxRZ4+eI8MPlwxfC/dhUkBPnTh0llkMoJlwQ31frcp4fIm6GpHfyeX3McAFiZ B595E44D1yhtDcY/Copapp3OZgXC/2mqHeYk61Vxu+85ALyizt+JmLBEc5TzQscU0HyE i//muWbHnMGX22Af25mBnGqqVKTCGgrMvVHzq7I5tCJtRXPp6q5TYD/NTwgHY845Lgd2 D3Lg==
X-Gm-Message-State: AOJu0YyM9i/8a58/Smx+Uvi8msjOX+IypIvqjZ/VieWlBM1qxDnXzXMH /vY1MKJ+9EmYqcsxgPHjCDKgkEon5wI=
X-Google-Smtp-Source: AGHT+IHfKtMQEQxyiYSv+W386SUX6ehekqLIwnbzVHG6kLq1YQYT0wAYUDDO+OEH8G6nt0R3bLvPHg==
X-Received: by 2002:a05:6a00:2d18:b0:690:d413:ee13 with SMTP id fa24-20020a056a002d1800b00690d413ee13mr8098848pfb.23.1695739666022; Tue, 26 Sep 2023 07:47:46 -0700 (PDT)
Received: from Dorothy ([2600:8801:be14:3100:8d80:dac7:45d7:37cd]) by smtp.gmail.com with ESMTPSA id w3-20020aa78583000000b0068fc48fcaa8sm10075265pfn.155.2023.09.26.07.47.44 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 26 Sep 2023 07:47:45 -0700 (PDT)
From: sburleig.sb@gmail.com
To: 'Marc Blanchet' <marc.blanchet@viagenie.ca>
Cc: deepspace@ietf.org, 'DTN WG' <dtn@ietf.org>
References: <050001d9ea91$d25d55f0$771801d0$@gmail.com> <E127BE58-D0D4-4692-9D19-5E99CA5DA7B5@viagenie.ca> <04a401d9edd3$08c6d0f0$1a5472d0$@gmail.com> <F7B5CEEC-4A89-4F4C-A351-59A6735146E7@viagenie.ca> <024d01d9f018$f62d3720$e287a560$@gmail.com> <142FBCA1-6B75-417E-9B0E-6651736B383F@viagenie.ca>
In-Reply-To: <142FBCA1-6B75-417E-9B0E-6651736B383F@viagenie.ca>
Date: Tue, 26 Sep 2023 07:47:42 -0700
Message-ID: <017601d9f088$680ad860$38208920$@gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="----=_NextPart_000_0177_01D9F04D.BBB32C50"
X-Mailer: Microsoft Outlook 16.0
Content-Language: en-us
Thread-Index: AQHMq5vkylNsA+4sI1/BNWFh9AuXfQJdA4S5Ae1MrFkBpYo7xgHAiHuEAaTPP8Sv/Q+2YA==
Archived-At: <https://mailarchive.ietf.org/arch/msg/deepspace/Pqem27C_vk-kXNb99Bw6eCG9SSM>
Subject: Re: [Deepspace] [dtn] technical remarks
X-BeenThere: deepspace@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: IP protocol stack in deep space <deepspace.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/deepspace>, <mailto:deepspace-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/deepspace/>
List-Post: <mailto:deepspace@ietf.org>
List-Help: <mailto:deepspace-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/deepspace>, <mailto:deepspace-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 26 Sep 2023 14:47:52 -0000

Good idea, Marc.  This time around I will split the thread up into individual issues.  First one coming shortly.

 

Scott

 

From: Marc Blanchet <marc.blanchet@viagenie.ca> 
Sent: Monday, September 25, 2023 8:54 PM
To: Scott Burleigh <sburleig.sb@gmail.com>
Cc: deepspace@ietf.org; DTN WG <dtn@ietf.org>
Subject: Re: [dtn] technical remarks

 

 

Le 26 sept. 2023 à 03:29, <sburleig.sb@gmail.com <mailto:sburleig.sb@gmail.com> > <sburleig.sb@gmail.com <mailto:sburleig.sb@gmail.com> > a écrit :

 

Hi, Marc.  Some comments in-line below.

 

Thanks again for taking the time. Very appreciated.

 

Further responses below.  (I’m getting concerned that the length and the multiple levels of responses becomes unmanageable to read and follow, maybe we shall cut in pieces (by subject or else) next time. Anyway, embedded comments for now.





 

Scott

 

From: Marc Blanchet <marc.blanchet@viagenie.ca <mailto:marc.blanchet@viagenie.ca> > 
Sent: Monday, September 25, 2023 10:32 AM
To: Scott Burleigh <sburleig.sb@gmail.com <mailto:sburleig.sb@gmail.com> >
Cc: deepspace@ietf.org <mailto:deepspace@ietf.org> ; DTN WG <dtn@ietf.org <mailto:dtn@ietf.org> >
Subject: Re: [dtn] technical remarks

 

 






Le 23 sept. 2023 à 06:04, sburleig.sb@gmail.com <mailto:sburleig.sb@gmail.com>  a écrit :

 

Well, maybe I am so out of touch with modern Internet standards that I don’t realize how delay-tolerant the architecture is nowadays.  Here are some of my possibly misplaced concerns.

 

The “late binding” concept built into the BP suite is based on the idea that connectivity lapses between nodes on an end-to-end path may be so lengthy (hours certainly; days for some deep space missions) that the destination node may change its location – its IP address – while traffic is en route to its earlier address.  Will this simply not happen frequently enough to justify late binding measures?  If so, can you explain why?  If not, how will Internet architecture handle these anomalies?

 

The fact that a node changes location does not mean changing its IP address. 

 

[SB]        The words “its location – its IP address –“ were intended to indicate that I meant, precisely, the node’s location in network topology, not its location in physical space.  That may not have been clear.

 

QUIC has the IP address and/or port change managed securely. The peers confirm their new IP address/port change by a crypto change, since they already have established a trust relationship in the first handshake. So it is robust to DOS or man-in-the-middle attacks and it works with IP address/port change.

 

[SB]        What works?  The destination address of the packet is different from the IP address at which the intended destination now resides.  How does the packet get routed to the destination node at its new address?  

 

Routing is not QUIC business. IP is responsible about it.

 

In some ways, BP « does everything ». Here, QUIC does some, IP does other, HTTP does other. It is a layered approach, with responsabilities for each.





And how long does it take to make that happen?

 

A related point: will DNS be used across the solar system to resolve destination names to IP addresses?  If so, will DNS lookup queries traverse interplanetary links 

 

Should not.

 

or will they be resolved by local DNS servers? 

 

yes.

 

If the latter, will DNS information be synchronized across the solar system by zone transfers or something similar?

 

The ones you need.

 

[SB]        What do you mean?  Are you saying that the DNS changes will be synchronized across the solar system in exactly the same way that they are synchronized across the Internet on Earth?  If not, what will be done differently?





 

  In either case, are we confident that the volume of DNS synchronization traffic across the interplanetary network will be so low as not to significantly degrade science traffic?

 

As answered to Keith in another email, there is a draft on DNS deployment which is already written and currently reviewed by DNS experts. I’m awaiting a few more reviews and will publish. Up to now, no reviewer got heartburn.


[SB]        So it is now known that the way in which DNS changes are propagated will work across the solar system without modification and any delays in that information propagation will have no impact on the performance of the network?

 

Very respectfully, I suggest that we just wait for the draft on DNS to be published and continue that conversation there. The reason is that I will have to essentially pull out the whole draft into these email conversations.  I’ll let a few days pass to wait for incoming reviews and will publish anyway at the end of this week. Then I suggest we resume that conversation on DNS. 

 

 

If a packet arrives at a router from which there is currently no onward connectivity, it will be retained in storage until the next onward connection is established, correct? 

 

yes.




Suppose that next onward connection is never established (maybe because the next router in the path has been destroyed by a meteor impact).  The packet is not forwarded, so its hop count does not increase, so its hop limit is never exceeded; would it continue to occupy storage at the router indefinitely?  If not, what is the pure IP functionality that would cause it to be discarded?

 

This is a forwarding policy/AQM discipline. It is already implemented but again might need to be tuned by increasing timers.


[SB]        How will the correct values of these timers be determined?  Will something tell AQM that connection to the next router in the path for a given packet will never be established?  How will AQM ensure that it doesn’t discard packets prematurely and, alternatively, doesn’t retain them so long that congestion is introduced?  Does something new need to be developed?

 

As appropriately written by Kiran in another email, the AQM may be pretty simple and it is more a buffer management work. On the actual timers, establishment, ..., Contact plans using TVR provides  that information to the IP stack.

 

 

I mentioned in an earlier email my misgivings about the ability of QUIC to impose flow control and congestion control over round-trip times that may be measured in hours or (if exercised only at the source and destination) days.  I think those are still open questions.

 

[SB]        I’ll restate those questions in another email, as I don’t think Christian’s response to Keith quite answered them.

 

Are QUIC retransmission timeout intervals going to be learned over time by measuring the interval between transmission and acknowledgment of several packets, or are they going to be provided by management (e.g., known one-way light time as determined from orbital elements)?  If the former, won’t it take a long time to figure out the correct timeout interval for retransmissions if the round-trip times are measured in hours?  If the latter, is that capability already defined in the QUIC specification?  And does it include automatic adjustment of the timeout interval when the end of a period of bidirectional connectivity is reached?

 

[SB]        This one too.

 

The automatic securing of QUIC connections by TLS is terrific, but it is only in effect while the packet is in transit, correct?  That is, once the packet has reached its destination it is no longer protected?

 

Well, it is decrypted by the QUIC stack at the destination and then pushed to the application. Like TLS today. I’m not sure I get your point here.

 

[SB]        Yes, you have to read the next sentence to get the point.




This is fine if the QUIC destination is the packet’s final destination, but in that case the packet retransmission timeout interval at the source becomes hard to compute: the packet may reside in storage at one or more routers along the path for various periods of time, which can only be determined if the entire path to the destination (not just the next-hop router) is known at the moment the packet is sourced.  

 

I’m not parsing you here, specially: "the QUIC destination is the packet’s final destination »: it is always the case, unless you do some proxies. And even that, the last leg of the last proxy will be the packet’s final destination. So I’m not sure.

 

[SB]        The QUIC destination is not the packet’s final destination if transmission along the path is accomplished by a string of QUIC proxies as suggested in the deepspace-ip-assessment draft.  So not always - unless that option in the draft is being abandoned. 

 

I’m not sure I understand why you are sticking on that. It seems to me that deploying proxies or not is really a decision of the operation of the service/network as today on Internet. The paragraph in the draft is pretty clear in stating that a string of QUIC proxies has his own issues. The intent of the paragraph and the draft was to state various possibilities with the pros and cons. I’ve just re-read the paragraph and it seems very clear that proxies are described with issues.





But sure, if the packet travels by a string of proxies then the destination of the transmission from the last proxy will be the packet’s final destination, no argument.

 

If, on the other hand, transmission along the path is accomplished by a string of QUIC proxies as suggested in the deepspace-ip-assessment draft,

 

Actually, the string of QUIC proxies was discussed but not recommended. I’ll look again at the text. It may be useful in some deployment scenarios, but we don’t need it.

 

[SB]        You don’t need it unless you care about end-to-end delivery latency.  Suppose a message is being sent by an instrument operator in Houston to a rover on the surface of Mars; the message will travel from Houston to a DSN station (e.g., Goldstone), from there to a Mars relay satellite, and from the Mars relay satellite to the rover.  Suppose the distance from Earth to Mars is 12 light minutes (about average), and suppose the message suffers a couple of bit errors in transmission from the relay satellite to the rover.  Transmission of the message has taken a little over 12 minutes so far, and all that time has been wasted; likewise the additional 12 minutes the instrument operator’s machine is now waiting for the acknowledgement that never arrives.  The source node’s retransmission timer now expires and the message is sent again, and maybe this time it makes it to the rover without incident; transmission of the message to the rover has taken 36 minutes, which could have been reduced to 12 minutes if the relay satellite itself had been able to detect the message transmission failure and retransmit locally.  Is this unimportant?

 

There are pros and cons on each approach. Actually the draft describes two solutions. The operator of the network decides which one he prefers for his own use case.  









then the packet will be delivered by QUIC multiple times while traversing the path.  In this case it might be resident at multiple routers while awaiting onward connections and, while resident, would not be protected by QUIC/TLS.  Does this mean that additional application-layer security is always going to be needed for all deep space IP traffic?

 

[SB]        Do we have an answer for this question?

 

(A note on your remarks regarding LTP: you say that “to get its benefits, one has to make sure that all links are using LTP.”  That is simply false.  Every hop on a BP end-to-end path can use a completely different convergence-layer protocol.  The ability to exploit – and/or adapt to – the different characteristics of different types of links on the end-to-end path, instead of expecting a single set of communication properties to hold everywhere along the route, is one of the advantages of the BP architecture.)

 

Sorry, I should have rephrased as: every link should have a reliable transport under BP to achieve an end to end reliability.


[SB]        Exactly – QUIC or LTP or TCP or something similar.

 

I understand that HTTP itself can be operated in a RESTful manner, so that it is no longer a client/server protocol – very helpful!  

 

Well, it is a client-server protocol. RESTFul does not change anything about that. 

 

[SB]        You’re right, I should have said “query/response” instead of “client/server”.




What about HTTPS?  Will HTTP always be run over QUIC, so that it is always automatically secured by TLS?  Or will TLS handshakes be necessary when establishing a HTTPS session?

 

See QUIC (RFC9000, RFC9001,…). QUIC includes TLS and TLS is negotiated together with QUiC parameters in a single RTT. QUIC can’t run without TLS.

 

[SB]        So you’re saying that HTTP will always be run over QUIC, yes?  That’s the answer to my question?

 

QUIC is a transport like TCP or UDP, so it is under HTTP. HTTP runs over QUIC. Yes.  There is a good tutorial on QUIC that the primary authors and implementors have done during one of the IETF some time ago. It is available on Youtube. I suggest to listen to it, very clear, good, understandable. 





 

I understand too that, thanks to zero-RT, only connection to a new or re-located peer (new IP address) or to a socket with a different port number would force a QUIC connection handshake, resulting in a communication hiatus of no more than 40 minutes at Mars, assuming no data loss.  You are certainly correct that new spacecraft are not likely to be added to the network very often, but of course not all nodes of the space network are going to be spacecraft – ground nodes are going to be included in the network as well, right?  Those are likely to cost closer to $2000 than $100,000,000, so there could be a good many of them, with new ones added frequently.  Also, wouldn’t applications running on spacecraft open new sockets relatively often?

 

A QUIC connection can be established between 2 peers for an almost infinite amount of time, say for 2 years if you want. A QUIC connection enables multiple streams (connections/sockets in the traditional TCP/UDP sense) to be carried over that « pipe ». So one can have multiple applications, using HTTP or anything, still carried over a single connection between 2 peers.


[SB]        Right, I think I got that.  But suppose you’ve got 20 nodes on Mars (rovers, landers, helicopters) and 2000 nodes on Earth (mission operators); maybe each mission operator only talks to 5 Mars nodes on average, so 10000 connections in all.  Each connection took (on average) 24 minutes to establish, so 4000 hours of transmission time (about 5.5 months) was consumed in setting up the connections. 

 

No. Again, think QUIC is a pipe. It establishes a connection between the peers. Inside the QUIC connection, you have streams, which are the equivalent of TCP connections. Those do not need any handshake as the QUIC connection is already established and secured.

 

I should find the graphics that people are using for describing QUIC. Much easier to understand graphically.

 





Every time a new mission operations node comes online or an existing mission operations node moves to another network location (getting a new IP address) another 24 minutes of transmission opportunity are spent on making the new connection.  Is this okay?

 

How would files be transferred over deep space IP?  Would we use FTP and SFTP and TFTP as in the Internet?  If so, don’t we pay for a connection handshake (1 to 3 round trips of up to 40 minutes each for FTP, more for SFTP) whenever starting a session, plus a query/response round trip for each file?  If not, what will we use instead?

 

File transfer can be done in a myriad of ways. Nowadays, if you transfer a file over Internet, it is ~98% over HTTP.


[SB]        Okay, good answer.

 

Establishing an SSH session requires a multi-round-trip handshake, but maybe we won’t need SSH in the deep space Internet?  

 

SSH is an example of an interactive application protocol not well suited for space use. As you know, SSH, like Telnet, sends each keystroke in a single packet. I’m pretty sure no one was planning to have that kind of application over BP, because it does not make sense. So don’t use SSH as is. However, one can configure SSH to send the line in a single packet (aka after the CR/LF). That would be more possible, again like any application protocol, if you configure the timeout properly. In a line mode, it would be similar to Command and Control currently used. 

 

[SB]        But timeout isn’t the issue, is it?  The issue is the round-trip time that elapses between issuing a command and getting an acknowledgment.  You’re right, though, SSH simply is not an application you want to run over deep space links.




Will we just use RESTCONF instead? 

 

That is one possibility yes.

 

Does RESTCONF let you do everything you’d otherwise do with SSH?

 

RESTCONF can be more compared to DTNMA. So BP will be managed by DTNMA, IP will be managed by RESTCONF. 


[SB]        Sure.

 

I think we’re going to want to send email over this network. 

 

Great! 

 

Do I understand correctly that SMTP, POP3, and IMAP require connections, and POP3 and IMAP servers deliver mail only in response to queries, all of which require round-trip exchanges?

 

This is more a deployment discussion than the actual protocol. But I remember that we did not discuss it in the draft, so I guess we should add it. In a nutshell, the deep space links are just one link. SMTP can be carried over and then mail sits in the mail server on the celestial body network infrastructure. The user then just use regular IMAP (or even web-based mail) locally on the celestial body. 

 

[SB]        Doesn’t SMTP need to make connections between sending and receiving servers? 

 

Yes. So?





How is this going to scale up, in the event that the deep space internet becomes widely used for settlement and industry?  Does something new need to be developed?

 

What needs to be done? I can’t follow you. We have pretty good experience in wide scale SMTP deployment. 





 

How about RTP, DHCP, NTP?  Will their session establishment procedures work over interplanetary signal propagation delays?

 

Good questions. As the draft says, there is a need to assess various application protocols to see which one: works as is, works with an adjustable timer, does not work, needs a protocol change, …  >From your list:

- RTP:  I was surprised to see that space agencies in CCSDS have standardized RTP over BP. So I guess if RTP works over BP, it shall work over IP, but I haven’t looked at it.  Cisco and Lockheed Martin have made a presentation on 2-way videoconferencing in space a few IETFs ago. One should look at it.

 

[SB]        So the answer to this one is pending.

 

Well, the draft does not pretend to answer all questions!  And if you re-read again section 6, it says just that.





 

- DHCP: I see no use for DHCP over deep space links. Could you tell me why you need it?

 

[SB]        Good point, another protocol that shouldn’t be exercised over the deep space internet.

 

DHCP is for local configuration of an attaching device on a link (mostly broadcast links). We typically don’t do DHCP over P2P links on IP networks, does not make any sense. So I can’t see the use of DHCP over deep space links. However, on celestial body networks, of course, it makes total sense to configure nodes using DHCP. But again, this is also the choice of the operator of the network. One can decide to do everything manually if it fits its use case.





 

- NTP: interesting question. On celestial body networks, no issue. Over deep space link, don’t know. Question is do we need it. How is time sync done in BP?

 

[SB}        We upload clock correction offset messages in bundles. 

 

So nothing prevents that to be done over HTTP/QUIC. 





 

I’d also be worried about the routing protocols – RIP, OSPF, BGP.  If routing in the deep space IP network is going to rely on current information about the topology, as in the Internet, won’t we see a lot of routing errors due to lengthy delays in the arrival of new information about discovered topology changes?

 

Yes. Again, TVR wg is looking at this problem. And this is written in the draft.


[SB]        But the fact that TVR is looking at the problem is not an answer, is it? 

 

Well, it says that we are trying not to duplicate work. In other words, what I’m saying is that this discussion should be in the TVR working group. 





Does something new need to be developed?

 

Maybe. The good thing here is that the TVR work encompasses various other use cases, such as energy conservation by shutting down links, nodes, …. When they are « not needed ». What that means is that there will be many implementations of TVR available that we could leverage.





 

There are probably a few other issues I would need to understand better, but this is all I can think of for now.

 

Thanks for taking the time to review the draft.  Hope I answer your questions.

 

[SB]        Some of them.  But it seems to me that the deepspace IP framework as currently conceived relies on the network being small, heavily managed, and static; what happens if the idea catches on?  Deep space IP seems to lack solutions to the problems of using (expensive) transmission opportunities efficiently, managing storage queues effectively, imposing delay-tolerant flow control, controlling congestion, and propagating information about dynamic changes in network topology in a timely manner.  

 

Its advantages are not yet clear to me.

 

Well, I’m not abandoning to convince you.  But it is healthy to have those conversations and that you question the « framework ». Thanks for taking the time.

 

Marc.





 

Regards, Marc. 

 

 

From: Marc Blanchet <marc.blanchet@viagenie.ca <mailto:marc.blanchet@viagenie.ca> > 
Sent: Tuesday, September 19, 2023 11:29 AM
To: Scott Burleigh <sburleig.sb@gmail.com <mailto:sburleig.sb@gmail.com> >
Cc: deepspace@ietf.org <mailto:deepspace@ietf.org> ; DTN WG <dtn@ietf.org <mailto:dtn@ietf.org> >
Subject: Re: [dtn] technical remarks

 

 

Le 18 sept. 2023 à 20:39, sburleig.sb@gmail.com <mailto:sburleig.sb@gmail.com>  a écrit :

 

Hi, Marc.

 

Well, I’ve been pushing that idea. The actual draft has 2 co-authors. So I guess this should be discussed in general, not targeted to me specifically.







  To get this started, here are some remarks on the new draft from a generally technical perspective.

 

Thanks for your great review. Very appreciated.

 

 

In my mind, the central conceptual premise of RFC 4838 and the DTN work that grew out of it is that communication round trips must be rigorously excluded from the communication protocols used to operate an internet over interplanetary links.  This is because long signal propagation latencies and potentially frequent and lengthy lapses in connectivity between nodes may make round-trip times so long that the information received on the return path is obsolete or otherwise no longer usable and/or arrived so late that opportunities for further transmission had to be foregone - in all cases, an unacceptable waste of expensive communication resources.

 

Right. Deep space has specific characteristics (long delays, interruptions) that some applications/protocols/… are not designed for that use case, whatever we use IP or BP.







 

That concept was (I believe) the rationale for moving beyond the IP stack in advocating for a solar system internet: virtually every important protocol in the IP stack (IP itself being the main exception) required a handshake to negotiate communication parameters, issued data from a server in response to a query from a client, or both.

 

We start to disagree. Yes, there are some application protocols (or some applications) that were designed with the use of instant negotiations and handshakes.  That does not preclude them to be unusable in a less « friendly » environment. It depends. Our initial findings is that it is not that quite bad, on the contrary. An example. While not all applications we are using on Internet make sense in deep space (this is an important point), a very large majority is using HTTP as its « application transport ». HTTP does not have a notion of time by itself, unless one is using some specific headers (see the draft for details), in which one should be careful about the values you put in; consequence is no change in HTTP needed. And HTTP is used a lot, a lot, in current applications with the RESTful paradigm, or other variations/enhancements such as gRPC, …. I verified it: it works just fine as is, if you are changing default timers in configs of clients and servers. No change in code, no change in protocol, no change in software. Many/Most? Internet applications we are using today are not well designed for deep space use case: of course, you won’t buy a Taylor Swift ticket over a deep space link (at least for now ;-). But, if one is developing an application for a relevant use in space, then HTTP is able to carry your application no issue. That is a big big win for getting any current developer to develop space applications, reusing many frameworks in use today. Of course, they need a few best current practice (some are already in the draft, but obviously not comprehensive) to be careful to use, for example, a completely asynchronous design (which is in fact the BCP even for Internet applications) and be careful about timers. That is mostly it. 

 

To summarize, while there are some application protocols that are not suited for space use case, we have a great framework nowadays that do work well in this environment with simple but careful config.







 

The keystone of your draft seems to me to be the substitution of QUIC for TCP. 

 

Yes, it is an important part of it. But as you have read the draft, we looked at the different levels of the protocol stack to make sure it overall works. Looking into only one piece would not work.







QUIC in itself seems to me quite cool: QUIC clearly accelerates connection establishment and solves TCP's head-of-line blocking problem.  In the BP world, I think QUIC would make an outstanding convergence-layer protocol.

 

When you say QUIC CLA, you mean BP over QUIC over IP. Yes, it could be defined and it would work. The current design for deep space comm is to use BP over LTP over space links. So I guess the QUIC CLA will be used in terrestrial networks (or celestial body local networks) as a replacement of the TCP CLA. Fine, but that is not the proposal in the draft. To me, while possibly better, replacing the TCP CLA with a QUIC CLA for terrestrial networks is IMHO a marginal advantage, as many of the TCP CLA used today is between space agencies private high-speed networks. I would welcome the dtn wg to look at the QUIC CLA. Now back to the proposal, which is not about a QUIC CLA.







 

Where the draft goes wrong, I think, is in asserting that substituting QUIC for TCP would make the BP stack unnecessary.  

 

I for one tried hard to not say BP is unnecessary. I’m more saying that there is an alternative proposal that may also work. It will be up to the space organizations to decide which one (or both) they want to use. QUIC vs TCP is not the full story. It is all the pieces together that make an alternative to a BP infrastructure.







I think that's incorrect:

*	Supporting multiple concurrent flows over a single connection would indeed achieve the same sort of continuous transmission that we already get with LTP, a major plus.

 

We can do pros and cons of LTP vs QUIC. However, it should be noted that there is an important difference between the two:

- QUIC is end to end, provides reliable transport over non reliable links, TLS security end to end (setup in a single RTT), … so whatever the number and the characteristics of the links in the network from one endpoint to the other, it maintains its properties.

- LTP on the other end is on a per link basis. Therefore, to get its benefits, one has to make sure that all links are using LTP. Applications won’t be able to know that, they just have to believe it is the case. (For those reading and not familiar, LTP is RFC5326 (Experimental status from the dtnrg time; I know that a new version is being worked on in CCSDS).







*	 
*	Also, configuring QUIC for zero-RT would eliminate round trips in re-establishing a previously established connection, another big plus.  But every connection with a new peer, 

 

Given that a typical peer in space costs around $100M, that event won’t happen often. But even if it happens « often », it is a one RTT to setup. 

 

Let’s say a ground software station wants to establish a connection to a not yet known device (I have a hard time thinking that it would be the case typically…) which is on Mars, then it would take a 40 minutes max (1RTT) to establish the secured connection. Time it takes is not because of IP or QUIC, but just the light time. Given that typically, these devices last for quite some time, sometimes years, the actual « cost » of a single RTT to establish the connection is marginal. Moreover, if you know (which is pretty much the case!) in advance that you would be « talking » to this device, one can establish the connection between the two peers before the spacecraft leaves Earth, therefore it would be already « connected » together when resuming communications later in space.







*	or with a peer whose location in the topology has changed, would still require a handshake. 

 

Only if there is a change in IP address or port. It remains to be seen how such things (change of IP address/port) will happen « often » in space. But even, It is a single RTT to confirm securely the two peers. Avoid DOS or man in the middle or else attacks. Very secured. Tested on a wild environment called the Internet… ;-)







*	LTP, on the other hand, can always begin transmission to a brand-new or relocated node immediately upon the start of a communication opportunity, as the parameters of that communication are always pre-established when the LTP engine is notified of the change.

 

Per link basis. QUIC won’t need to care about link status changes, as it is end to end. The link state changes are handled by lower levels (aka TVR).

 

*	 
*	QUIC, like TCP, thinks that flow control is implemented upon reception of transmission authorizations.  Christian Huitema proposed asynchronous transmission of anticipatory "flow control credits" as a way to make this work in space, and sure, you could modify the QUIC protocol in this way.  Just using rate control informed by contact plans - as LTP does - is simpler.
*	QUIC apparently thinks that measuring the link data rate and sticking to that rate controls congestion.  It does not.  DTN congestion control is a matter of limiting storage occupancy, not of matching the transmission rate to the acknowledgment rate.
*	Christian Huitema's experiment showed that QUIC can indeed convey data reliably over 20-minute one-way light times, once you revise the code and lengthen the timer intervals.  But that is by far the simplest part of reliable transmission over interplanetary links: accommodating continuous changes in connectivity, link character, data rates, network topology, forwarding rules, and required quality of service was not considered in any way.

 

I let that cited person to respond to these.

 

 

Obviously all of this stuff is software, and you can always modify software to do anything you want.  It is certainly possible to spend some time and money modifying all of the many widely exercised IP protocols 

 

Again, that I think is a false statement. Not that much of changes. The draft talks about some « major » IP protocols in used today (probably for the Internet, covering 99% of the traffic), but also says that some investigation might be needed for other protocols. 







in such a way that you reinvent what is already in the BP stack 

 

I would claim the reverse. I found over years that the BP framework was reinventing a lot of stuff that was already well defined in Internet protocols. I’ll take one example. I would have preferred not to compare, but I think we need to. (IP) Network management has gone through roughly two iterations (I’m oversimplifying here) of protocol frameworks: SNMP+MIB and then NETCONF-RESTCONF+YANG. There is quite a wealth of implementations, data models, etc… for almost every kind of device. One can look at the yang data model github repo. NETCONF is defined to be used over TCP or SSH: that won’t work as is in deep space. NETCONF over QUIC is specified in a draft. But RESTCONF is over HTTP. So right off, you can use everything defined (including the YANG models being discussed in TVR at this time) in this realm without changes (make sure you configure in the implementation the timer for expiration to be longer than typical). On the other hand, I found DTNMA (https://datatracker.ietf.org/doc/draft-ietf-dtn-dtnma/) to be very complex, to read, to implement, and kind of reinventing the wheel. It may have its own use, but again, if NETCONF-RESTCONF+YANG works, why redo? I’m sure Ed will chime in on this. ;-)

 

Another example. We tried some time ago to develop a native application over BP: there was a project interested in looking at using BP. From the application perspective, it just pass blobs of data to the BP node with the destination endpoint id (i.e. « address »). There is no notion or abstraction of a connection (on purpose). We end up doing a lot of connection identification, state, payload ids and managing reordering, etc… in the application layer. It is difficult to do that right and reliably in an application. And then if you add the need for security, one has to rely on BP security to be implemented from end to end and made it « known » to the application: I don’t know of any BP implementation that enables this, but I guess it could be done. While QUIC does all this, including security, for you, free, in the single connection, with all the certificate/keys framework we are using today.






– eliminating all connections and all client/server interactions – 

 

Actually, I would, on the contrary, not eliminate any connection or client/server interaction. I would keep them, but make sure they are tolerant to delay and disruption. Some of that « dirty work » is done at the transport and IP layer, so it is mostly invisible to the application protocols and applications (with some care about timers as discussed above), which is a big big plus.







and call it IP, and maybe that is what the industry will decide it prefers.

 

What's unfortunate about such an initiative is that it would sap time and resources from development and deployment of a protocol stack that we already know will work at Mars without significant modification, since it was shown to work at interplanetary distances (on the order of 1 light minute, with link outages resulting in round-trip times on the order of days), in space, 15 years ago.

 

I can’t respond to this except that I’m sorry to bring this now. It is also the fact that things/protocols have evolved in parallel and got, IMHO, an overall better solution. But I understand that I need to work hard to convince you.







 

As best I can tell, the only advantage of the proposed initiative is that the IP stack's "implementations have been exercised on real traffic orders of magnitude more than what has been achieved with Bundle Protocol stacks."  This is undeniable, but of course the same could have been said about SNA or DECnet in the days before IP became widely supported.

 

I hope that argument (being exercised widely) to be taken the right way. I think you are right that this should not be the only or primary reason for choosing this new proposal. However, it secures a lot people using a technology, when software has been exercised so much: way less chance to have memory issues, crashes, … For a difficult environment such as space, it is IMHO a very big plus.







 

More to the point, we are not here talking about deploying that widely exercised stack in space.  We are talking about deploying an IP stack that is operationally very different from what we use on Earth and which has not yet been specified in detail, much less implemented, much less demonstrated, much less exercised on even a tiny fraction of the traffic conveyed between Earth and the International Space Station over the last seven years of continuous science instrument support. 

 

IP has been used in space. Just an example: it has been presented at a recent IETF, by Cisco and Lockheed Martin (see: https://www.youtube.com/watch?v=V8Xdr0DvGKE).  Webex Earth to Orion spacecraft. Don’t know if they were using QUIC, but WebRTC so SIP-RTP-UDP-HTTPwebsockets. Some careful considerations, especially the use of the right codec. But this is real-time two-way communications. Not sure we want to do this as is in « far » deep space (aka Mars). 

 

I am doubtful.

 

Understood. Hope that I, not by myself, but hopefully with the help of others, will be able to convince you at some point.

 

Regards, Marc.

 

PS. Don’t want to overpromise, but we are working on a testbed to show the whole stack with simulation of the various conditions as near to real situations as possible. Will report results when done.

 

 

Scott

_______________________________________________
dtn mailing list
 <mailto:dtn@ietf.org> dtn@ietf.org
 <https://www.ietf.org/mailman/listinfo/dtn> https://www.ietf.org/mailman/listinfo/dtn