Re: [alto] BGP-LS as replicated IGP computing and BGP route collection

"Y. Richard Yang" <> Sun, 23 July 2023 09:52 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 34077C151522 for <>; Sun, 23 Jul 2023 02:52:36 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: 0.451
X-Spam-Status: No, score=0.451 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DC_PNG_UNO_LARGO=0.001, FREEMAIL_FORGED_FROMDOMAIN=0.096, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.25, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001, URI_DOTEDU=1.999] autolearn=no autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id pRTA_EYb9gSk for <>; Sun, 23 Jul 2023 02:52:35 -0700 (PDT)
Received: from ( []) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by (Postfix) with ESMTPS id ACC1EC151095 for <>; Sun, 23 Jul 2023 02:52:34 -0700 (PDT)
Received: by with SMTP id 71dfb90a1353d-48133dc9820so1158342e0c.3 for <>; Sun, 23 Jul 2023 02:52:34 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20221208; t=1690105953; x=1690710753; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=oLD/qGjjTDD4Uxv5p3ibGGb7RllLNFS68J88gEBVh1A=; b=NvWHfE2iZGdmB3ak9TgI03QwszX0cA7Uoup1/5d1dJpnuvqB5L2MNzfqKlfilWR+O+ ENpbaf1Pi98BvlOhzUOJ+O81YqlLxxDrEemqdTntCLbLIQXE/J3x0J6TDgXHDdY54XQ8 XLU8HnHKIPJIE/LMrJuQoAPkg8TRZq4bgiM5QgIs+2HA8o4iRoAlCDkCz5KLnz5zefWT yDx7x/OXOBw5FR5gw1481DsJ7ZO73rEENQWGoo9/WSru4NQ04zwlLNMBqlo4vCjtShtF O7+WYgkrROdvupRX1DzDGyOR4NHoEA9hnE+LKq7VjmQCnmbqm+MSCXFI+Bmo+lTgNnxj XXvA==
X-Gm-Message-State: ABy/qLZqM2yU4BVI4SfOmd92EYWjQ8rhI9OAMsU/YE5IWDNh0NUM/kOo XzNo9/nBolQL0Ixt2aRVYOsMBy6nbYKzYW+ZmTI=
X-Google-Smtp-Source: APBJJlGgeHkAA0tqjX2NEdL3PC+5U20h064giTaa8qsr2g7xVw9/+0Susg27BvCrK6Hgj2KxhGnXHUcu/ONEBSsrmPY=
X-Received: by 2002:a67:eec9:0:b0:443:6c48:4d49 with SMTP id o9-20020a67eec9000000b004436c484d49mr928081vsp.10.1690105953119; Sun, 23 Jul 2023 02:52:33 -0700 (PDT)
MIME-Version: 1.0
References: <> <009c01d9bd00$05ece080$11c6a180$>
In-Reply-To: <009c01d9bd00$05ece080$11c6a180$>
From: "Y. Richard Yang" <>
Date: Sun, 23 Jul 2023 11:52:21 +0200
Message-ID: <>
Cc: "Randriamasy, Sabine (Nokia - FR)" <>,, stefano previdi <>, IETF ALTO <>
Content-Type: multipart/related; boundary="000000000000f8114d0601247419"
Archived-At: <>
Subject: Re: [alto] BGP-LS as replicated IGP computing and BGP route collection
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: "Application-Layer Traffic Optimization \(alto\) WG mailing list" <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sun, 23 Jul 2023 09:52:36 -0000

Hi Adrian,

Thanks for the excellent info and discussion. As suggested, loop in the WG
mailing list as this should be of interest to some.

On Sun, Jul 23, 2023 at 2:53 AM Adrian Farrel <> wrote:

> Hello Richard,
> One of the principles underlying BGP-LS was [1] that BGP is very good at
> applying filters. Thus. BGP-LS could be sourced by a network node that
> participates (at least passively) in the IGP and chooses to report a subset
> of the information (and abstraction, if you will) to give a view of the
> network that may be a summary or a policy-based filter.

Thanks for the background info. Looking back, I agree that a BGP-based
state replication system is a quite reasonable design. Beyond filtering
(redistribution) capabilities, BGP has other desirable features such as
(incremental) updates. IGP also conducts (topology) state replication, but
an IGP node conceptually is a router, which should be a node in the graph.
BGP has more of a design of network information replication guided by

> BGP-LS was originally conceived as an east-west sharing of network
> topology between domains to facilitate inter-domain TE. This was largely
> killed by an understanding of the scaling and confidentiality concerns of
> such an approach. It was realised that it was better to provide the
> information northbound to a controller that could either span multiple
> domains or cooperate with controllers for other domains.

Providing (individual-domain) topology appears to be a more modular design,
where modularity is a basic principle of IETF. Such a module can serve not
only (TE) control (RFC7752 motivation 1, but also
information exposure (RFC7752 motivation 2)

> From an ALTO point of view, I am not sure that there is a large difference
> between participating passively in the IGP, and receiving info via BGP-LS.

This is a good discussion. One of the main successes is the work by Benocs,
which uses IGP indeed: below is their architecture chart []
[image: image.png]

> I think, probably, the key factor is where the policy points are in the
> network architecture: should ALTO server have visibility into the whole
> network and apply its own policies, or network it be up to the to decide
> what information it exports with the policy being inside the network?

Right now, one focus of some of us is BGP-LS as the data source, and one
motivation is more engineering benefit: with wide deployment of BGP-LS, and
if we focus on BGP-LS as the unifying (of OSPF, IS-IS, SR, Flexible Alg and
growing) data source, we reduce the implementation complexity by
considering BGP-LS as the "thin-waist" of state collection, and hence
implementing the interaction with a single type of data source. As the
state of a network is typically distributed (e.g., a single IGP domain can
have different areas), we leave the deployment and management to BGP-LS to
make sure that a complete, less-redundant view is collected (by one, or a
set of BGP-LS nodes), with minimal efforts to aggregate all data to build
the right data source.

The "where policy-point is" discussion is beyond the preceding engineering
consideration and is an excellent architectural point.  As of now, the ALTO
design is that the ALTO server has visibility into the whole network and
applies its own policies. We forsee more flexibility and building multiple
ALTO servers representing the same network, from different points of view.
But we are keeping things simple at this point.

> From a philosophical point of view: is ALTO part of the provider’s network
> or running on top of it? Thus: does the ALTO server apply policies, or does
> the network apply the policies?

This is an excellent philosophical point. As of now, I will say that a
basic ALTO representing a single network is part of the network, and for
larger networks, for example, an autonomous cluster (LHCONE is a good
example), ALTO can benefit from and hence should take the second view: ALTO
is on top of it.

> Now, let’s take two points from your question:
>    1. Latency. Assuming that hop-by-hop latencies are available and
>    advertised in the IGP, you’d certainly get this information by listening to
>    the IGP. Of course, end-to-end latency is about **far** more than just
>    the link lengths (something that could be known from configuration and
>    would be a waste of IGP resources) and is not a static commodity when you
>    consider router queues and router throughput latency. But these things are
>    highly variable and not a good thing to try to report on in the IGP. There
>    has been some work to try to measure and report end-to-end (well,
>    edge-to-edge) path latency both in IPPM and TEAS – I find it a bit
>    suspicious! Nevertheless, if this information was available, it might
>    usefully be passed to and processed by an ALTO server. Whether you use
>    BGP-LS to collet the information largely depends on where the information
>    is, what granularity and churn the information has, and who
>    selects/establishes the edge-to-edge paths that you want to report on.
I share this point. I take the view that ALTO is an information aggregation
service, from wherever information is. For static (propagation) latency,
IGP can carry it as a TE metric (to be interop, adaptive), and we target to
collect it from BGP-LS, if possible, for engineering purposes. We have
recently build an ALTO server on top of a PerfSonar infrastructure, who
collects data also from IPPM work.

>    1. Computed paths. While the internal workings of routers are not
>    standardised, the SPF algorithms used in OSPF and ISIS **are**
>    well-known modulo how ECMP is handled. Thus, it seems unnecessary to export
>    the results of these computations (the FIB) which is different on each
>    node, when this can be easily computed using the RIB. In fact, this is the
>    basic assumption of PCE – that so long as you can see the network topology,
>    you can compute any path you want between any two points. In general,
>    computation is cheap, data export is expensive.
> These are very reasonable points, and we are planning to follow this path.
One consideration of collecting FIB directly is that it is more "evolution"
robust. The computation logic can continue to evolve and be distributed at
multiple points. To know what is computed and installed, collecting the
info directly, by a standard approach, appears to be more robust, for an
abstraction service (of what has been done) such as ALTO and maybe
other services such as control plane verification. Data export is indeed
expensive and hence should be a major consideration in such a route state
(RS) protocol, if designed. Make sense?

Thank you so much!

> Cheers,
> Adrian
> [1] Times change, and foundational engineering principles do not
> necessarily remain untouched.
> *From:* Y. Richard Yang <>
> *Sent:* 22 July 2023 23:44
> *To:* Adrian Farrel <>; Randriamasy, Sabine (Nokia -
> FR) <>;; stefano
> previdi <>
> *Subject:* BGP-LS as replicated IGP computing and BGP route collection
> Hi Stefano, Adrian,
> I have become increasingly more appreciative of the BGP-LS work started by
> your RFC7752.
> After close to 9 years, we start to see some deployments. One main hurdle
> of ALTO deployment I see is the difficulty to obtain alto information, such
> as basic end-to-end delay maps. Hence, the increasing deployment of BGP-LS
> is a great catalyst.
> But I have two questions:
> 1. Is there evaluation/report on whether BGP-LS has enough info to
> replicate the computed paths by IGP such as OSPF and IS-IS? Using BGP-LS
> collection as topology for TE does not require the consistency between IGP
> computed route, but as a compact model does.
> 2. What is your reaction on using BGP(-LS) to collect computed routes that
> includes all routing processes from each router? I see scalability is a
> potential objection in that the information size is number of routers times
> number of prefixes, where topology collection does not have such
> multiplication. But this depends on the details and we may investigate
> optimizations. Is there any related investigation?
> Thank you so much!
> Richard
> --
> Richard