Re: [alto] ALTO at IETF-108: Finishing the current milestones and discussion on re-chartering the WG

[Sebastian Kiesel <[e-mail]>]

Hi Jensen,

On Fri, Jul 17, 2020 at 07:08:13PM -0400, Jensen Zhang wrote:
> Hi Sebastian, Ingmar, Danny, all,
> 
> First, thanks for all your discussions. I also have several comments.
> Please see inline.
> 
> On Tue, Jul 14, 2020 at 2:39 PM Sebastian Kiesel <[e-mail]> wrote:
> >
> > Hi Ingmar, all,
> >
> > On Tue, Jul 14, 2020 at 01:33:37PM +0200, Ingmar Poese wrote:
> > > Am 14.07.20 um 11:56 schrieb Sebastian Kiesel:
> > > > On Mon, Jun 29, 2020 at 08:18:04AM -0300, Danny Alex Lachos Perez wrote:
> > > > > > Even the very first ALTO use case was multi-domain in some sense:
> > > > > > the bittorrent clients in the access networks of Comcast and Sprint,
> > > > > > the pirate bay tracker in Sweden, that could be seen as three domains.
> > > > > > The examples in the drafts you have listed are obviously different and
> > > > > > more complex, but I am still struggling to tell what exactly is new and
> > > > > > needs to be done.
> > > > >
> > > > > Yes, I remember your question. I will try to clarify our multi-domain
> > > > > approach.
> > > > >  From your P2P example, we have the next figure:
> > > > > [image: multi-domain-ALTO.jpg]
> > > > > An ALTO server in each domain will provide only local information to ALTO
> > > > > clients. In the example, the tracker (ALTO client) receives
> > > > > topology-/policy-related information of a single domain (AS1 or AS2). Let's
> > > > > call it "single-domain information exposure".
> > > >
> > > > While your observation is probably true in most deployments we have seen
> > > > so far, I do not completely agree on an architectural level.
> > > >
> > > > We have designed the ALTO protocol such that an ALTO client may ask one
> > > > ALTO server *any* question, e.g., "give me network map and cost map for
> > > > the whole Internet address space" or ECS(IP1, IP2) where IP1 and IP2
> > > > are arbitrary IP addresses in whatever domain.
> > > > The idea that there might be more than one ALTO server, and that one
> > > > ALTO server might give better answers to some specific questions while
> > > > another ALTO server might give better answers to other questions,
> > > > was in our mind when we worked on the protocol, but it did not result
> > > > in any specific feature of the base protocol that would deal with that
> > > > situation.
> > >
> > > I would actually like to go a little further on this point. At the early
> > > stages, if i remember correctly, there was the discussion on "my view on the
> > > Internet".
> >
> > You remember correctly.
> >
> > And if I remember correctly, we also had the idea that (at least for the
> > P2P use case), the ALTO information generally should flow in the
> > opposite direction of the money:  If, for example, I am a paying
> > customer of a specific ISP, I should listen to my ISP's guidance
> > (disseminated through my ISP's ALTO server), in order to reduce the
> > ressource consumption in their network and their costs for their
> > peerings or upstream, so they won't have to increase my monthly bill
> > and/or enforce volume caps or bandwidth throttling.
> >
> > As long as the assumption "bigger pipes = lower cost per byte = less
> > congestion = higher per-flow throughput" holds true, this strategy will
> > also give better application performance to me, the famous win-win
> > situation (or assumption). But there are also counter-examples: If I am
> > a mobile subscriber, my mobile network operator might be tempted to say
> > (for the P2P use case): any destination peer outside of my network is
> > better than pushing the same data twice through my radio access network,
> > to a another customer in my network running the P2P application. If
> > the ALTO server gives such simplistic guidance (low preference for local
> > network prefixes, uniformly higher preference for all other prefixes),
> > I might end up contacting a mobile user in a different part of the
> > world, thus going through 2 wireless links plus a transcontinental link.
> >
> > So, multi-domain optimization might be challenging. And I am still not
> > so sure what are the domains in the multi-domain draft ...
> 
> I agree. The term "domain" is not defined clearly. There have been
> multiple (active/expired) multi-domain related drafts in WG. They may
> talk about different use cases but all claim they are "multi-domain".
> But in general, I think a "domain" means a partial network that
> provides an ALTO server. Those domains can be ASes of the same ISP,
> IXPs, the backbone networks of different ISPs, different access
> networks, or even mobile edge clouds. Each domain may only have a
> partial view (routing and performance) of the user-intended traffic
> (not the complete view of partial traffic). So they need some
> server-to-server information exchange to get a more accurate view of
> traffic for their end-users. But for different kinds of "domains",
> there may be different requirements. e.g., the internal routing
> information can be shared among domains of the same ISP, but may not
> for different ISPs.
> 
> > > I would like to point this out again and the fact that "The
> > > Internet" looks different from different vantage points. Depending on where
> > > you are, decisions are different. Furthermore, a client is not capable of
> > > deciding which information to use from what ALTO server - and it if get
> > > multiple answers, which one is "more" correct.
> >
> > The notion of "more correct" is difficult as this indeed highly depends
> > on the point of view.
> >
> > > If the ALTO Server now only has information about the local network, or only
> > > gives out information about the local network, matching this to interAS
> > > client (or even clients in the same confederation) almost impossible - and
> > > this gets worse if not all Networks offer an ALTO service.
> > > I think an ALTO server should always give an indication on the entire
> > > Internet from it's point of view, even if much of it is aggregated. This
> > > reduces the problems to "what ALTO server to ask" for the client and can be
> > > much easier deployed and maintained on a large scale.
> >
> > I agree.
> 
> I also agree. I think the term "local network" is equal to the term
> "domain" that I used above, right?

probably yes. maybe we should not think of BGP ASes or so when we try
to figure out what a domain in the context of ALTO could be.

What we have so far: The operator of a "local network" may populate his
DHCP and DNS servers with information to direct the clients in his
network to an ALTO server of his choice [RFC 7286].  He may also
configure his DNS servers in a way that ALTO clients outside his
network, that want to optimize traffic that originates or terminates in
his network, can discover an ALTO server of his choice [RFC 8686].
Using this ALTO server (most likely run by said network operator) the
operator may publish his the ALTO information (preferences or policies,
as you like), from * his point of view *.

Preferences or policies may be different, even conflicting, see my
example above. So the main challenge might not be just a common data
model and getting all data everywhere, but finding an "single truth"
that is merged from conflicting points of view...

> But the problem is how to make the
> ALTO server for a domain (local network) have enough information to
> give an indication on the entire Internet? Do you think existing
> routing protocols and measurement tools are enough?

If we really talk about the entire Internet, I'm not only wondering
about technical feasibility, in the sense of protocols and algorithms,
but also: Who would be willing to run the infrastructure that collects
the data, computes and disseminates that single truth?  Would they also
accept the legal liability if overlay applications following that advice
cause a traffic meltdown in some part of the world?


So, how to move forward?  Can we find a realistic use case, that someone
actually wants to implement?  Maybe not for the whole Internet, but
with some domains, having different (maybe even conflicting) policies?
What additional standardization would be needed to support it?

> > > > This is different for the discovery mechansims: there we had the idea
> > > > that an ALTO server that is "near" to the endpoint of the data
> > > > transmissions to be optimized, typically announced by the operator of
> > > > the access network, will probably give good answers.
> > > > If the "peer selection" (or any other routing decision in the overlay)
> > > > is done in the endpoint of the data transmission itself, use RFC 7286;
> > > > if it is done at a remote entity such as a tracker, use RFC 8686 to find
> > > > ALTO server that is nearby the actual endpoint.
> > > >
> > > >
> > > > > On the other hand, if we want an ALTO client to receive merged information
> > > > > from multiple domains, ALTO servers need to be able to exchange
> > > > > information. In the example, the tracker will receive AS1’s and AS2’s
> > > > > merged information. Let's call it "multi-domain information exposure".
> > > > > As we identified [0], different use cases could benefit from multi-domain
> > > > > information exposure using ALTO. However, it also places requirements that
> > > > > the current ALTO design do not satisfy [1].
> > > >
> > > > There is definitively room for extensions. We have so many architectural
> > > > options (side note: what is now just the "ALTO protocol" has been called
> > > > in RFC 5693 and 6708 the "ALTO client protocol", in anticipation that
> > > > there might be an ALTO server-to-server protocol in the future).
> > > >
> > > >
> > > > First area of questions: Data model.  If every ALTO server defines the
> > > > network map on its own and uses the default smaller-value-is-better
> > > > routingcost metric, which does not have a well-defined base unit, it
> > > > will be hard to impossible to merge information from different servers.
> > > > Deprecate them?  Replace with what? Or go for an architectural option
> > > > (see below) that does not require information merging on the ALTO
> > > > protocol side.
> > >
> > > This is an interesting problem that i would like to go into further. But
> > > then, i have a very narrow Agenda on our use case, so maybe i am
> > > overshooting here.
> 
> Thanks for pointing out this problem. Personally, I think simply using
> the same data model of the ALTO client protocol is not good enough for
> the server-to-server information exchange. But we may not need to
> fully deprecate them. But we do need some extensions. e.g., include
> the well-defined base unit. And I still remember the "ingress point"
> issue that Ingmar mentioned in the last IETF meeting. This information
> should also be considered.
> 
> > > > Second area of questions: placement of entities and protocols between
> > > > them. This is also not completely new, see Sec. 2.2.3 of RFC 7971
> > > >
> > > > Ideas:
> > > >
> > > > 1. Define a new server discovery mechanism.  An ALTO client would
> > > > discover several ALTO servers from different domains, ask each of them,
> > > > and then aggregate the information on the client-side.
> > > >
> > > > 2. Mesh-style ALTO server-to-server communication.  Do we need a new
> > > > s2s protocol?  Is a small extension to the ALTO protocol sufficient?
> > > >
> > > > 3. Hierarchical structure with backend information sources and frontend
> > > > servers towards the ALTO clients. Aggregation of the information from
> > > > different backends takes place in the frontends. How to ensure
> > > > consistency? Do we need a new protocol for backend to frontend
> > > > communication? Is a small extension to the ALTO protocol sufficient?
> > > >
> > > >
> > > >  From an academic perspective, I'd love to investigate and specify each
> > > > of these options and many more.  But from an IETF perspective: who is
> > > > willing to deploy anything on a scale that requires and justifies the
> > > > effort for standardization?  If there is one: what is their preference
> > > > which way to go?  Is your group planning a deployment?
> > >
> > > We (BENOCS) have had that on our ToDo list for about a year now, but we have
> > > not come around to implementing it so far. The deployment would encase about
> > > 100 million subscribers towards a few Hypergiants across Europe.
> >
> > This really sounds impressive!
> 
> Very impressive. I'm fully from academia. So I would like to see the
> real requirements and issues coming from real industrial deployment.
> But one debate is whether we use a hierarchical structure like DNS, or
> a collaborative structure like BGP, or a delegated structure like the
> master-slave cluster?
> 
> > > I have a fairly good plan on how to do it - although we were not going to
> > > use ALTO for cross domain ALTO (we call it cascading internally), but rather
> > > implement something ourselves that fits our use case of
> > > Hypergiant<->end-user mapping (we do not do P2P).
> >
> > Do you already have an idea which parts of your solution require
> > standardization and are understood well enough so that we can put
> > them on the deliverables list for our rechartering discussion?
> >
> > > I would be open to discussing this in a broader sense to standardize the
> > > approach.
> >
> > We definitively should have a venue in the context of IETF or IRTF
> > for discussing this, but is is already ripe for standardization?
> 
> I agree. I cannot say it is already ripe. But I believe there are
> already many people interested in this topic. But to understand its
> feasibility, we need more discussions.
> 
> Ingmar, Sebastian, and Danny, many thanks for your great discussions.
> As WG chairs suggest, would some of you like to give a single slide
> presentation to discuss this potential rechartering item during the
> IETF 108 ALTO session?

My feeling is that this is a topic for further research, but not for
being put on the deliverables list in next week's re-chartering
discussion.  If someone else has a clearer view what is already ripe for
standardization, I'll be more than happy to comment and contribute.

Thanks
Sebastian