Re: [Anima-signaling] GRASP issue 51: Flooded objectives

[Toerless Eckert <[e-mail]>]

To validate the proposal:

Can you write down pseudo-code for registrar discovery, eg:

  code for registrar ASA:
   API-call(s) and JSON parameters

  code for bootstrap-proxy ASA:
    API-call(s) and JSON parameters

  I would primarily like to validate that the parameters we brainstormed
  in Berlin will be carried and how they are put into the API,
  specifically:

     loop-count, remaining-loopcount
  
  And just for the fun of it, i would love to have in the example
  the parameters "prio", "weight" because we could show then how
  we could map mDNS service announcements to GRASP - because prio and
  weight are the two standard mDNS service parameters.

  Eg: algorithm on bootstrap proxy:

On Fri, Aug 05, 2016 at 03:24:43PM +1200, Brian E Carpenter wrote:
> So, here is where my head is now on flooding.
> 
> 1. We specify that the source address is cached with the flooded objective, and that
> multiple copies are kept if there are multiple sources.

Yes, but "source" really must be the ASA transport address, aka: 
   source = {ACP-ULA-IPv6-address, proto=TCP, ASA-GRASP-TCP-port}
  
Example why:

   I want to support "ISSU" - In-service-software-upgrade without interruption,
   meaning: i have version 1 of the ASA running, i now want to change to
   version 2 of the ASA without ever having service interruption.

      - Start version 2 of ASA
      - Version 2 ASA calls API to announce its objectives
        these are likely the same objectives as those from version 1,
        so we now have for a while 2 ASA on the same host (same ACP-ULA IPv6-address)
        that announce the same objective(s). So we need those transport
        addresses as "source" to distinguish them.
      - Version 2 ASA indicates to version 1 of ASA: stop service
      - Version 1 ASA calls API to stop announcing its ojectives
      - Version 1 ASA waits until it has finished all open services
        (eg: Registrar-ASA: all ongoing EST connections).
      - Version 1 ASA (optionally) indicates to Version 2 ASA
        that it's now done. exists
      - lots of variations. For example, instead of stopping announcement
        immediately, Version 1 could first announce with lower priority
        (when there is priority).

> 2. To cover the common case, if a regular ASA calls synchronize(obj) it will
> by default get back the first entry in the flood cache, and if there is nothing
> in the flood cache, GRASP will discover and synchronize the objective.

a) Whether or not to do discovery&sync if the flood-cache is empty should be
a decision of the ASA, not a hardcoded GRASP function. In general it would
be a logic defined by the objective. Eg: For objectives with a dense set
of clients, i would define to not do it. Instead, a callback-API would
be nice so the client will get an API callback when the first entry shows
up in the flood-cache.

> 3. To cover the case where we want multiple flooded values, we'll add a function
> to the API called something like get_flood(obj) which will return a list
> of versions of the objective, each tagged with its source address.

Yes. It would be nice to have example (pseudo?) code to vet if the
API is complete. Eg: registrar announce & discover with functions as we discussed.

  Registrar-ASA:
    # get TCP port for EST
    s = socket(TCP,...)
    bind(s, ACP-ULA-address)
    est-port = getsockaddr(s).port

    API-call grasp.flood-syn("AN registrar") with parameters
      - loopcount = orig-loopcount = maxloopcount
      - service-address = { EST-v1, est-port } // details TBD
      # Even if we decide we do not want prio and weight, i would love to
      # use them in the example, because prio/weight are the mandatory DNS-SD
      # parameters, so it would be good if we showed how easy we could translate
      # them into GRASP:
      - weight = n
      - prio = m
  
  Bootstrap-proxy-ASA:

   try-to-connect:

     bestprio = maxprio+1
     bestdistance = maxdistance+1
     bestsources = {}
     forall instance in grasp.get_flood("AN registrar")
         instance.distance = instance.orig_loopcount - instance.loopcount
         if instance.distance = bestdistance
             if instance.prio = bestprio
                 bestsources += instance
             if instance.prio < bestprio
                 bestprio = instance.prio
                 bestsources = instance
         if instance.distance < bestdistance
             bestdistance = instance.distance
             bestsources = instance
     totalweight = 0
     forall instance in bestsources 
         totalweight += instance.weight
     weight = rnd(0,totalweight)
     forall instance in bestsources 
         weight -= instance.weight
         if weight <= 0
             break instance
    
     if instance.service-address = "EST"
         addr = instance.addr.ipv6addr
         port =  instance.service-address.est-port
         open TCP socket to destination {addr, port}
         if connection_timed_out
             # ASA seems to be dead. Do API call to grasp to declare this
             # ASA objective dead:
             grasp.delete_objetive_source(instance)
             goto try_to_connect.
         else
             # continue with EST

     else
         HELP, unsupported protocol
  


> Text? I have put some words into my working version of the GRASP spec.
> I will push the latest XML to GitHub shortly.

Great.

> Running code? Yes, I've coded all that in my prototype and tested all the code
> paths. (I don't intend to publish that code until we get a -07 draft out.)

Great.

> In terms of freshness, here's some text:
> 
>    If the value of an objective has time-sensitive properties such as a
>    defined lifetime, a defined expiry time, or a defined refresh rate,
>    these SHOULD be expressed as part of the definition of the objective
>    itself; they are not foreseen in the GRASP protocol.  For example, an
>    objective that is flooded using Flood Synchronization might include a
>    parameter specifying its expiry time as a UTC string.

If the announcer of a flood sync announce dies, how would its cache entries
ever be deleted ? On nodes that have an ASA interested in the objective,
the ASA can look at expiry time and tell GRASP to expire the entries. But
how about nodes without such ASA ?

  Registrar is a bad example, because every node should run a bootstrap-proxy.
  But think for example ASA that are only required by nodes connecting to hosts.
  Only autonomic access-switches would be interested in those objectives,
  not any of the intervening core/distribution routers. But those would still
  need to have caches to make flooding work correctly. Nut they wold love
  not to maintain obsolete months old objectives that are stale.

Second discussion point:
A node without prior state (sleeping or booted) becomes active, it wants
to know  specific objectives that are only flood synchronized. How does it
do that ?
  a) Wait until it's periodically repeated
  b) Multicast a discovery to speed up discovery.
     This can be IMHO quite complex: The most simple option is
     b.1) The discovery is flooded network wide without taking
          cache entries into account. Every node that has the objective
          and sees such a discovery request will trigger a new 
          flood discovery. So the client would then need to
          convert to examine after a while the flood discovery cache.
     b.2) If there are a lot of periodically sleeping/waking up
          nodes then this will be inefficient. so we would need to
          take the cache into account and stop flooding of the
          discovery once we have cache entries - and generate new
          flood sync messages only to the link with the just woken-up
          device.

b) is complex. So IMHO the easiest approach right now is to use a) and
make ASA periodically repeated flood sync.

Aka: expiry_time looks to me like a logical GRASP header parameter so that
we can GRASP without ASA expire flood sync cache entries.

Cheers
    Toerless

> Regards
>    Brian
> 
> On 01/08/2016 20:55, Toerless Eckert wrote:
> > Inline.
> > 
> > On Mon, Aug 01, 2016 at 08:47:22AM +1200, Brian E Carpenter wrote:
> >> Hi,
> >>
> >> Prior to raising this on the WG list, I'd like feedback from the design team.
> >>
> >> Following side discussion with the bootstrap and ACP teams in Berlin, there
> >> is a new issue about flooded objectives (not really about the Flood message
> >> itself). It has two aspects:
> >>
> >> 51.1 Should flooded objectives have a time-to-live before they are deleted from
> >> the flood cache? I think the answer is no, because in any case the source can repeat
> >> the Flood message if the value changes. If anybody sees a use case for a TTL,
> >> please explain it.
> > 
> > We discussed in Berlin methods to discover sources that went away:
> > 
> > a) IMHO, we do not want to send rapid updates from sources to discover quickly when
> >    they die because of the absence of those announcements. (aka: no fast aliveness).
> > b) IMHO we do not want to work out the details of the GRASP infra quickly detecting
> >    when a sources goes away and flood that information (no IGP like fast source revocation).
> > c) I suggestested that GRASP clients could help:
> >    - GRASP thinks source is alive (long or no TTL)
> >    - GRASP API client tries to connect to objective, and fails
> >    - GRASP API client tells GRASP "objective seems dead".
> >    - GRASP removes objective source from cache
> >    - When GRASP receives another flood message from objective source,
> >      it repopulates cache and tells client.
> > 
> > With c) in mind, we might not need a TTL, but I am not aware of existing service
> > announcement mechanisms that tried to rely on this "c)" approach, so i am a bit
> > weary to solely rely on it:
> > 
> > When there is a network disconnect between objective and client, the client would not
> > be able to reach the objective and the objective would be declared dead. Aka: If we
> > do "c)", we do need to periodically flood from the objective source the announcement
> > to recover from this situation. The periodicity should be determined by the source
> > itself. But it would be good if the source also announced this periodicity.
> > 
> > Aka: I would suggest that intead of TTL, the source could simply announce the periodicity
> > of it's announcements (in seconds). Default 30 seconds.
> > 
> > If a client does not perform "c)", then it can use the periodicity value to
> > calculate when it can declare the source dead - eg: after three periods non-receipt
> > of a message.
> > 
> > There is also a fundamental limitation with "c)", and that is when the announcer
> > is not tightly coupled with GRASP:
> >    - GRASP could announce the objective, but the process providing the objective
> >      could be running on a different system and could have died. Or maybe this is
> >      even possible on the same system.
> >    - Client connecting to the objective  would declare it dead to GRASP, but
> >      GRASP would continue to see announcement from the source.
> > 
> > Ideally, we would not have this case, aka: GRASP API should always know when
> > the actual source of the objective is really alive. But i am not sure if it's possible
> > to always guarantee this. So it would be good to have in the flooded objective
> > some indiciation that the GRASP announcement is not coupled to the objective aliveness
> > itself.
> > 
> > I am extrapolating this case from what we do with mDNS: Most services today do
> > not couple to mDNS via API. Instead when we use something like Avahi, then there
> > are Avahi config-files that describe the services running on the same system (or
> > on another system). Which of course does not guarantee that each of those services
> > is actually working fine. So i't seasy to announce services when they do have
> > problems (eg: can't start, fail, ...).
> > 
> > Aka: objective parameter "GraspObjectiveCoupled" (or come up with a better name).
> > If true, it means that the GraspSource and Objective Owner feel that it's safe
> > to do c) on the client side to discover death of the service, and false means it is not.
> > 
> > 
> >> 51.2 We do have a potential use case where more than one source floods the same
> >> objective, and the recipient ASAs need to distinguish the results. That use case
> >> is when there are multiple bootstrap proxies advertising on the same LAN, and the
> >> joining nodes (pledges) can choose which one to try. I'm sure there will be others.
> > 
> > The most important one being multiple registrars announcing the registrar service,
> > and the bootstrap proxies want to discover them, and then choose the best one based
> > on network distance (aka: remaining loop count).
> > 
> >> My proposal is to specify that a node receiving a Flood message will cache
> >> the received objective plus its source locator (which is already in the message
> >> for other reasons). We'll also have to upgrade the API so that the recipient
> >> ASA can get multiple flooded answers.
> > 
> > Yes.
> > 
> > Thanks!
> >     Toerless
> >>
> >> Comments?
> >>
> >> Regards
> >>    Brian
> >>
> >> _______________________________________________
> >> Anima-signaling mailing list
> >> [e-mail]
> >> https://www.ietf.org/mailman/listinfo/anima-signaling
> > 

-- 
---
Toerless Eckert, [e-mail]