Re: [alto] ALTO Draft ReCharter WG review

Qin Wu <> Thu, 04 March 2021 13:22 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 7BE713A0809 for <>; Thu, 4 Mar 2021 05:22:42 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.766
X-Spam-Status: No, score=-1.766 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_BLOCKED=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, URI_DOTEDU=0.132] autolearn=no autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 2aPrBpQ5viD9 for <>; Thu, 4 Mar 2021 05:22:38 -0800 (PST)
Received: from ( []) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 89A853A07F7 for <>; Thu, 4 Mar 2021 05:22:37 -0800 (PST)
Received: from (unknown []) by (SkyGuard) with ESMTP id 4Drrts1bwWz67pQT; Thu, 4 Mar 2021 21:14:45 +0800 (CST)
Received: from ( by ( with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2106.2; Thu, 4 Mar 2021 14:22:29 +0100
Received: from ( by ( with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2106.2; Thu, 4 Mar 2021 14:22:29 +0100
Received: from ( by ( with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.1.2106.2 via Frontend Transport; Thu, 4 Mar 2021 14:22:28 +0100
Received: from ([]) by ([fe80::b177:a243:7a69:5ab8%31]) with mapi id 14.03.0513.000; Thu, 4 Mar 2021 21:22:22 +0800
From: Qin Wu <>
To: Qiao Xiang <>, 刘鹏 <>
CC: "Y. Richard Yang" <>, IETF ALTO <>
Thread-Topic: [alto] ALTO Draft ReCharter WG review
Thread-Index: AdcQ86X2uxA+1lGHQomVpMdQzi/GcQ==
Date: Thu, 04 Mar 2021 13:22:21 +0000
Message-ID: <>
Accept-Language: zh-CN, en-US
Content-Language: zh-CN
x-originating-ip: []
Content-Type: multipart/alternative; boundary="_000_B8F9A780D330094D99AF023C5877DABAADE3BFF4dggeml511mbschi_"
MIME-Version: 1.0
X-CFilter-Loop: Reflected
Archived-At: <>
Subject: Re: [alto] ALTO Draft ReCharter WG review
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Application-Layer Traffic Optimization \(alto\) WG mailing list" <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 04 Mar 2021 13:22:43 -0000

Thanks Qiao for sharing your project on Unicorn and thought on multi-domain setting.
My impression in your implementation is each domain name and first ingress node in such domain will be carried in the ALTO response message.
One thing I want to highlight is
Unicorn has already been deployed in several cities of USA in 2018 and implemented in the ODL open source.
Quote from
   The authors build an ALTO server on top of the OpenDaylight Software
   Defined Network controller.  The ALTO server collects the network
   state information from the OpenDaylight controller, e.g., topology,
   policy and traffic statistics, processes the collected information
   into resource abstraction, and sends the abstraction back to the ALTO
   client at the resource orchestrator.

The Unicorn framework has been deployed and
   demonstrated in small federation networks connecting Dallas, Texas,
   Los Angles, California, and Denver, Colorado at different
   conventions.  For example, in 2018, the federation in the
   demonstration is composed of three member networks.  Network 1 is in
   Dallas, Texas, and Network 2 and network 3 are in Los Angeles,
   California.  Network 1 is connected to network 2 through a layer-2
   WAN circuit with a 100 Gbps bandwidth, provisioned by several
   providers such as SCinet, CenturyLink and CENIC.  Network 1 is a
   temporal science network in the CMS experiment, while network 2 and 3
   are long-running CMS Tier-2 sites.
I believe your case require server to server communication for end-to-end interdomain routing.

Secondly, as for network information exposure in multi-domain setting, I think
1. 3GPP Network Exposure Function is a good example for network information exposure, but it is part of 5G core which enables data exchange between UE and application server and does not extend to other domain.
2. ZSM multi-domain network and service management can be another concrete example for multiple domain network information exposure which can be used to have a quick
response to network anomaly or reroute the traffic to the less congested path [].
3. PCE has similar design for multi-domain setting, which allows PCE to PCE communication.

发件人: Qiao Xiang []
发送时间: 2021年3月3日 0:18
收件人: 刘鹏 <>
抄送: Y. Richard Yang <>; IETF ALTO <>; Qin Wu <>
主题: Re: [alto] ALTO Draft ReCharter WG review

Hi Peng, Qin and Richard,

Very good discussion! Richard and I have been working with folks from CMS and ESNet (a large global multi-domain science network) to design network information exposure abstractions and mechanisms in multi-domain networks, with privacy requirements considered. The basic idea stems from the ALTO path-vector extension but goes beyond to take privacy into consideration. The following are some pointers.

[1] "Toward Fine-Grained, Privacy-Preserving, Efficient Multi-Domain Network Resource Discovery", IEEE JSAC, 2019. (
[2] "Resource Orchestration for Multi-Domain, Exascale, Geo-Distributed Data Analytics", (

For the pointers above, the privacy requirement considered in this work is that the network information of multiple domains should be exposed to applications as a complete, unified aggregation, appearing as much as possible as from a single (virtual) network. We design a network information obfuscation mechanism so that the application is not able to associate any network resource bottleneck information to any domain, reducing the risk of exposing network vulnerability.

In addition, we also studied how to control the routing across multiple domains to achieve more flexible end-to-end interdomain routing. Essentially, we propose a mechanism that allows networks to expose their available interdomain routes, just as BGP looking glasses, so that applications can control them. In this setting, we consider the privacy setting where each network's BGP export policies are private, and design interesting algorithms for applications to select the best policy-compliant routes without knowing the export policies. The following is the pointer for this study:

[3] "Toward Optimal Software-Defined Interdomain Routing". INFOCOM 2020 (

Above are our current efforts on extending ALTO to multi-domain settings. It would be great if we can know more about the industry efforts on network information exposure in multi-domain settings, and the privacy requirements of operators. This would be extremely helpful to push this extension forward! :-)


On Tue, Mar 2, 2021 at 1:14 PM 刘鹏 <<>> wrote:
Hi Richard,

Thank you. please see my reply inline below.

Peng Liu | 刘鹏
China Mobile | 移动研究院
mobile phone:13810146105

发件人: Y. Richard Yang<>
时间: 2021/03/02(星期二)07:36
收件人: 刘鹏<>;
抄送人: IETF ALTO<>;Qin Wu<>;
主题: Re: [alto] ALTO Draft ReCharter WG review
Dear Peng,

Thank you so much for the feedback. Please see below.

On Fri, Feb 26, 2021 at 9:23 PM 刘鹏 <<>> wrote:
Hi WG,

Here are some considerations of recharter:
I believe that the multi domain problem is worthy of attention.

It is good info.

At present, operators also research in it, which may involve guaranteeing end-to-end network service in the future, such as delay, bandwidth, etc. There are some researches on cross domain deterministic network in the industry, which need some support from management and control plane.

 Do you want to share some pointers?

[Peng] As Qin said, it is hard to collect information across network borders.
Just taking deterministic network as an example, it is hard to applying synchronization, unified forwarding strategy in multi domain, so there are some works need to be done with management plane. Due to the large scale and multi domains or operators, the management system may be distributed.
A potential way is to consider negotiating the forwarding time of each domain in advance and carrying time stamp in the message to control the forwarding path of each domain. While it needs some agreements like contracts to prevent one party from tampering with and denying the management content.
Beside this, there may be others use case. I'm not sure if Alto servers are willing to do those work, but it may be helpful to collect or configure some key information.

Who is the provider of Alto service is related to the deployment and cooperation mode. It may be difficult for operators to give too much detailed network information now. If the Alto service belongs to the operator, it may be used to help manage its own network. If Alto service belong to non operators, I think the issue of how to cooperate needs further discussion.

It looks that you want to consider both modes: multidomains but single operator (i.e., intra-cooperation) and multidomains and multiple operators. Regardless, I agree that it is important for the work to clarify on the privacy requirements.

[Peng] Yes, agree.



Peng Liu | 刘鹏
China Mobile | 移动研究院
mobile phone:13810146105

发件人: Qin Wu<>
时间: 2021/02/22(星期一)21:45
收件人: IETF ALTO<>;
抄送人: alto-chairs<>;alto-ads<>;
主题: [alto] ALTO Draft ReCharter WG review
Hi, :
We have requested one hour session for ALTO WG meeting in the upcoming IETF 110, which is arranged on Friday, March 12, 14:30-15:30(UTC).
The goal is to boil down ALTO recharter and have consensus on charter contents in IETF 110.
To get this goal, an updated inline draft charter text for ALTO has just been posted to this list,

This charter has received a couple of rounds of informal review from WG members, chairs and our Ads from brief to deep thorough, 5 new chartered items have been listed.
We would like to solicit feedback on these new chartered items and your use case, deployment, idea corresponding to these new chartered items.
Sharing your past deployment story will also be appreciated.

The ALTO working group was established in 2008 to devise a request/response protocol to allow a host to benefit from a server that is more cognizant of the network infrastructure than the host is.

The working group has developed an HTTP-based protocol and recent work has reported large-scale deployment of ALTO based solutions supporting applications such as content distribution networks (CDN).

ALTO is now proposed as a component for cloud-based interactive applications, large-scale data analytics, multi-cloud SD-WAN deployment, and distributed
computing. In all these cases, exposing network information such as abstract topologies and network function deployment location helps applications.

To support these emerging uses, extensions are needed, and additional functional and architectural features need to be considered as follows:

o Protocol extensions to support a richer and extensible set of policy attributes in ALTO information update request and response. Such policy attributes may indicate information dependency (e.g., ALTO path-cost/QoS properties with dependency on real-time network  indications), optimization criteria (e.g., lowest latency/throughput network performance objective), and constraints (e.g., relaxation bound of optimization criteria, domain or network node to be traversed, diversity and redundancy of paths).

o Protocol extensions for facilitating operational automation tasks and improving transport efficiency. In particular, extensions to provide "pub/sub" mechanisms to allow the client to request and receive a diverse types (such as event-triggered/sporadic, continuous), continuous, customized feed of publisher-generated information. Efforts developed in other working groups such as MQTT Publish / Subscribe Architecture, WebSub, Subscription to YANG Notifications will be considered, and issues such as scalability (e.g., using unicast or broadcast/multicast, and periodicity of object updates) should be considered.

o The working group will investigate the configuration, management, and operation of ALTO systems and may develop suitable data models.

o Extensions to ALTO services to support multi-domain settings. ALTO is currently specified for a single ALTO server in a single administrative domain, but a network may consist of
multiple domains and the potential information sources may not be limited to a certain domain. The working group will investigate extending the ALTO framework to (1) specify multi-ALTO-server protocol flow and usage guidelines when an ALTO service involves network paths spanning multiple domains with multiple ALTO servers, and (2) extend or introduce ALTO
services allowing east-west interfaces for multiple ALTO server integration and collaboration. The specifications and extensions should use existing services whenever possible. The specifications and extensions should consider realistic complexities including incremental deployment, dynamicity, and security issues such as access control, authorization (e.g., an ALTO server provides information for a network that the server has no authorization), and privacy protection in multi-domain settings.

o The working group will update RFC 7971 to provide operational considerations for recent protocol extensions (e.g., cost calendar, unified properties, and path vector) and new extensions that the WG develops. New considerations will include decisions about the set of information resources (e.g., what metrics to use), notification of changes either in proactive or reactive mode (e.g., pull the backend, or trigger just-in-time measurements), aggregation/processing of the collected information  (e.g., compute information and network information )according to the clients’ requests, and integration with new transport mechanisms (e.g., HTTP/2 and HTTP/3).

When the WG considers standardizing information that the ALTO server could provide, the following criteria are important
to ensure real feasibility:

- Can the ALTO server realistically provide (measure or derive) that information?

- Is it information that the ALTO client cannot find easily some other way?

- Is the distribution of the information allowed by the operator of the network? Does the exposure of the information introduce privacy and information leakage concerns?

Issues related to the specific content exchanged in systems that make use of ALTO are excluded from the WG's scope, as is the issue of dealing with enforcing the legality of the content. The WG will also not propose standards on how congestion is signaled, remediated, or avoided.

-Qin Wu (on behalf of chairs)
alto mailing list<>

| Y. Richard Yang <<>>   |
| Professor of Computer Science       |
|        |
alto mailing list<>

Qiao Xiang
School of Informatics,
Xiamen University