Re: [alto] ALTO Draft ReCharter WG review

[Qiao Xiang <[e-mail]>]

Hi Qin,

Thank you so much for the feedback. Please see my responses inline.

On Thu, Mar 4, 2021 at 9:22 PM Qin Wu <[e-mail]> wrote:

> Thanks Qiao for sharing your project on Unicorn and thought on
> multi-domain setting.
>
> My impression in your implementation is each domain name and first ingress
> node in such domain will be carried in the ALTO response message.
>
First, for domain name, I do not believe we need that in the ALTO response
message. Our setting here is each domain has an ALTO server, and the ALTO
clients at the aggregator have separate connections to different ALTO
servers. In this way, by maintaining a (domain, ALTO  server) mapping, the
aggregator can differentiate responses from different domains. Second, for
ingress node, the client needs to specify the ingress and the (srcIP,
dstIP) pair in the query first so that the ALTO server knows what
information to return. And I should also clarify that although we use
ODL-ALTO for our system, but the path vector extension is not implemented
exactly following the specification since it was not fully stabilized then.

@Jensen is the core developer and can comment further on this, as well as
the ODL implementation.


> One thing I want to highlight is
>
> Unicorn has already been deployed in several cities of USA in 2018 and
> implemented in the ODL open source.
>
I should clarify that this deployment is pre-production, and the
demonstration scenario is at SC18 and SC19, where we are at the conference
cities (Dallas and Denver), and orchestrate traffic from there back to a
Caltech LHC site (we manually separate this site into two domains to create
a 3-domain scenario for demonstration purpose.)


> Quote from
> https://datatracker.ietf.org/doc/draft-xiang-alto-multidomain-analytics/
>
> “
>
>    The authors build an ALTO server on top of the OpenDaylight Software
>
>    Defined Network controller.  The ALTO server collects the network
>
>    state information from the OpenDaylight controller, e.g., topology,
>
>    policy and traffic statistics, processes the collected information
>
>    into resource abstraction, and sends the abstraction back to the ALTO
>
>    client at the resource orchestrator.
>
>
>
> The Unicorn framework has been deployed and
>
>    demonstrated in small federation networks connecting Dallas, Texas,
>
>    Los Angles, California, and Denver, Colorado at different
>
>    conventions.  For example, in 2018, the federation in the
>
>    demonstration is composed of three member networks.  Network 1 is in
>
>    Dallas, Texas, and Network 2 and network 3 are in Los Angeles,
>
>    California.  Network 1 is connected to network 2 through a layer-2
>
>    WAN circuit with a 100 Gbps bandwidth, provisioned by several
>
>    providers such as SCinet, CenturyLink and CENIC.  Network 1 is a
>
>    temporal science network in the CMS experiment, while network 2 and 3
>
>    are long-running CMS Tier-2 sites.
>
> ”
>
> I believe your case require server to server communication for end-to-end
> interdomain routing.
>
>
>
> Secondly, as for network information exposure in multi-domain setting, I
> think
>
> 1. 3GPP Network Exposure Function is a good example for network
> information exposure, but it is part of 5G core which enables data exchange
> between UE and application server and does not extend to other domain.
>
> 2. ZSM multi-domain network and service management can be another concrete
> example for multiple domain network information exposure which can be used
> to have a quick
>
> response to network anomaly or reroute the traffic to the less congested
> path [
> https://www.etsi.org/deliver/etsi_gs/ZSM/001_099/002/01.01.01_60/gs_ZSM002v010101p.pdf
> ].
>
> 3. PCE has similar design for multi-domain setting, which allows PCE to
> PCE communication.
>
>
> Thank you again for the pointers. I'll take a look at ZSM and PCE soon.


Best
Qiao


> -Qin
>
> *发件人:* Qiao Xiang [mailto:[e-mail]]
> *发送时间:* 2021年3月3日 0:18
> *收件人:* 刘鹏 <[e-mail]>
> *抄送:* Y. Richard Yang <[e-mail]>; IETF ALTO <[e-mail]>; Qin
> Wu <[e-mail]>
> *主题:* Re: [alto] ALTO Draft ReCharter WG review
>
>
>
> Hi Peng, Qin and Richard,
>
>
>
> Very good discussion! Richard and I have been working with folks from CMS
> and ESNet (a large global multi-domain science network) to design network
> information exposure abstractions and mechanisms in multi-domain
> networks, with privacy requirements considered. The basic idea stems from
> the ALTO path-vector extension but goes beyond to take privacy into
> consideration. The following are some pointers.
>
>
>
> [1] "Toward Fine-Grained, Privacy-Preserving, Efficient Multi-Domain
> Network Resource Discovery", IEEE JSAC, 2019. (
> https://ieeexplore.ieee.org/abstract/document/8756056)
> [2] "Resource Orchestration for Multi-Domain, Exascale, Geo-Distributed
> Data Analytics", (
> https://datatracker.ietf.org/doc/draft-xiang-alto-multidomain-analytics/)
>
>
>
> For the pointers above, the privacy requirement considered in this work is
> that the network information of multiple domains should be exposed to
> applications as a complete, unified aggregation, appearing as much as
> possible as from a single (virtual) network. We design a network
> information obfuscation mechanism so that the application is not able to
> associate any network resource bottleneck information to any domain,
> reducing the risk of exposing network vulnerability.
>
>
>
> In addition, we also studied how to control the routing across multiple
> domains to achieve more flexible end-to-end interdomain routing.
> Essentially, we propose a mechanism that allows networks to expose their
> available interdomain routes, just as BGP looking glasses, so that
> applications can control them. In this setting, we consider the privacy
> setting where each network's BGP export policies are private, and design
> interesting algorithms for applications to select the best policy-compliant
> routes without knowing the export policies. The following is the pointer
> for this study:
>
>
>
> [3] "Toward Optimal Software-Defined Interdomain Routing". INFOCOM 2020 (
> https://ieeexplore.ieee.org/abstract/document/9155486)
>
>
>
> Above are our current efforts on extending ALTO to multi-domain settings.
> It would be great if we can know more about the industry efforts on network
> information exposure in multi-domain settings, and the privacy requirements
> of operators. This would be extremely helpful to push this extension
> forward! :-)
>
>
>
>
>
>
>
> Best
>
> Qiao
>
>
>
> On Tue, Mar 2, 2021 at 1:14 PM 刘鹏 <[e-mail]> wrote:
>
> Hi Richard,
>
>
>
> Thank you. please see my reply inline below.
>
>
>
>
>
> Peng Liu | 刘鹏
>
> China Mobile | 移动研究院
>
> mobile phone：[phone]
>
> email: * [e-mail] <[e-mail]>*
>
>
>
> 发件人: Y. Richard Yang <[e-mail]>
>
> 时间: 2021/03/02(星期二)07:36
>
> 收件人: 刘鹏 <[e-mail]>;
>
> 抄送人: IETF ALTO <[e-mail]>;Qin Wu <[e-mail]>;
>
> 主题: Re: [alto] ALTO Draft ReCharter WG review
>
> Dear Peng,
>
>
>
> Thank you so much for the feedback. Please see below.
>
>
>
> On Fri, Feb 26, 2021 at 9:23 PM 刘鹏 <[e-mail]> wrote:
>
> Hi WG,
>
>
>
> Here are some considerations of recharter:
>
> I believe that the multi domain problem is worthy of attention.
>
>
>
> It is good info.
>
>
>
> At present, operators also research in it, which may involve guaranteeing
> end-to-end network service in the future, such as delay, bandwidth, etc.
> There are some researches on cross domain deterministic network in the
> industry, which need some support from management and control plane.
>
>
>
>  Do you want to share some pointers?
>
>
>
> [Peng] As Qin said, it is hard to collect information across network
> borders.
>
> Just taking deterministic network as an example, it is hard to applying
> synchronization, unified forwarding strategy in multi domain, so there are
> some works need to be done with management plane. Due to the large scale
> and multi domains or operators, the management system may be distributed.
>
> A potential way is to consider negotiating the forwarding time of each
> domain in advance and carrying time stamp in the message to control the
> forwarding path of each domain. While it needs some agreements like
> contracts to prevent one party from tampering with and denying the
> management content.
>
> Beside this, there may be others use case. I'm not sure if Alto servers
> are willing to do those work, but it may be helpful to collect or configure
> some key information.
>
>
>
> Who is the provider of Alto service is related to the deployment and
> cooperation mode. It may be difficult for operators to give too much
> detailed network information now. If the Alto service belongs to the
> operator, it may be used to help manage its own network. If Alto service
> belong to non operators, I think the issue of how to cooperate needs
> further discussion.
>
>
>
> It looks that you want to consider both modes: multidomains but single
> operator (i.e., intra-cooperation) and multidomains and multiple operators.
> Regardless, I agree that it is important for the work to clarify on the
> privacy requirements.
>
>
>
> [Peng] Yes, agree.
>
>
>
> Richard
>
>
>
>
>
>
>
> Regards,
>
> Peng
>
>
>
> Peng Liu | 刘鹏
>
> China Mobile | 移动研究院
>
> mobile phone：[phone]
>
> email: * [e-mail] <[e-mail]>*
>
>
>
> 发件人: Qin Wu <[e-mail]>
>
> 时间: 2021/02/22(星期一)21:45
>
> 收件人: IETF ALTO <[e-mail]>;
>
> 抄送人: alto-chairs <[e-mail]>;alto-ads <[e-mail]>;
>
> 主题: [alto] ALTO Draft ReCharter WG review
>
> Hi, :
>
> We have requested one hour session for ALTO WG meeting in the upcoming
> IETF 110, which is arranged on Friday, March 12, 14:30-15:30(UTC).
>
> The goal is to boil down ALTO recharter and have consensus on charter
> contents in IETF 110.
>
> To get this goal, an updated inline draft charter text for ALTO has just
> been posted to this list,
>
> This charter has received a couple of rounds of informal review from WG members, chairs and our Ads from brief to deep thorough, 5 new chartered items have been listed.
>
> We would like to solicit feedback on these new chartered items and your
> use case, deployment, idea corresponding to these new chartered items.
>
> Sharing your past deployment story will also be appreciated.
>
>
>
>
> ============================================================================================
>
> The ALTO working group was established in 2008 to devise a
> request/response protocol to allow a host to benefit from a server that is
> more cognizant of the network infrastructure than the host is.
>
>
>
> The working group has developed an HTTP-based protocol and recent work has
> reported large-scale deployment of ALTO based solutions supporting
> applications such as content distribution networks (CDN).
>
>
>
> ALTO is now proposed as a component for cloud-based interactive
> applications, large-scale data analytics, multi-cloud SD-WAN deployment,
> and distributed
>
> computing. In all these cases, exposing network information such as
> abstract topologies and network function deployment location helps
> applications.
>
>
>
> To support these emerging uses, extensions are needed, and additional
> functional and architectural features need to be considered as follows:
>
>
>
> o Protocol extensions to support a richer and extensible set of policy
> attributes in ALTO information update request and response. Such policy
> attributes may indicate information dependency (e.g., ALTO path-cost/QoS
> properties with dependency on real-time network  indications), optimization
> criteria (e.g., lowest latency/throughput network performance objective),
> and constraints (e.g., relaxation bound of optimization criteria, domain or
> network node to be traversed, diversity and redundancy of paths).
>
>
>
> o Protocol extensions for facilitating operational automation tasks and
> improving transport efficiency. In particular, extensions to provide
> "pub/sub" mechanisms to allow the client to request and receive a diverse
> types (such as event-triggered/sporadic, continuous), continuous,
> customized feed of publisher-generated information. Efforts developed in
> other working groups such as MQTT Publish / Subscribe Architecture, WebSub,
> Subscription to YANG Notifications will be considered, and issues such as
> scalability (e.g., using unicast or broadcast/multicast, and periodicity of
> object updates) should be considered.
>
>
>
> o The working group will investigate the configuration, management, and
> operation of ALTO systems and may develop suitable data models.
>
>
>
> o Extensions to ALTO services to support multi-domain settings. ALTO is
> currently specified for a single ALTO server in a single administrative
> domain, but a network may consist of
>
> multiple domains and the potential information sources may not be limited
> to a certain domain. The working group will investigate extending the ALTO
> framework to (1) specify multi-ALTO-server protocol flow and usage
> guidelines when an ALTO service involves network paths spanning multiple
> domains with multiple ALTO servers, and (2) extend or introduce ALTO
>
> services allowing east-west interfaces for multiple ALTO server
> integration and collaboration. The specifications and extensions should use
> existing services whenever possible. The specifications and extensions
> should consider realistic complexities including incremental deployment,
> dynamicity, and security issues such as access control, authorization
> (e.g., an ALTO server provides information for a network that the server
> has no authorization), and privacy protection in multi-domain settings.
>
>
>
> o The working group will update RFC 7971 to provide operational
> considerations for recent protocol extensions (e.g., cost calendar, unified
> properties, and path vector) and new extensions that the WG develops. New
> considerations will include decisions about the set of information
> resources (e.g., what metrics to use), notification of changes either in
> proactive or reactive mode (e.g., pull the backend, or trigger just-in-time
> measurements), aggregation/processing of the collected information  (e.g.,
> compute information and network information )according to the clients’
> requests, and integration with new transport mechanisms (e.g., HTTP/2 and
> HTTP/3).
>
>
>
> When the WG considers standardizing information that the ALTO server could
> provide, the following criteria are important
>
> to ensure real feasibility:
>
>
>
> - Can the ALTO server realistically provide (measure or derive) that
> information?
>
>
>
> - Is it information that the ALTO client cannot find easily some other way?
>
>
>
> - Is the distribution of the information allowed by the operator of the
> network? Does the exposure of the information introduce privacy and
> information leakage concerns?
>
>
>
> Issues related to the specific content exchanged in systems that make use
> of ALTO are excluded from the WG's scope, as is the issue of dealing with
> enforcing the legality of the content. The WG will also not propose
> standards on how congestion is signaled, remediated, or avoided.
>
>
>
> -Qin Wu (on behalf of chairs)
>
> _______________________________________________
> alto mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/alto
>
>
>
>
> --
>
> --
>
>  =====================================
>
> | Y. Richard Yang <[e-mail]>   |
>
> | Professor of Computer Science       |
>
> | http://www.cs.yale.edu/~yry/        |
>
>  =====================================
>
> _______________________________________________
> alto mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/alto
>
>
>
>
> --
>
> Qiao Xiang
> Professor,
>
> School of Informatics,
>
> Xiamen University
>


-- 
Qiao Xiang
Associate Research Scientist,
Department of Computer Science,
Yale University