[Anima] ACP interfaces (was: Use of GRSP in discovery)

[Brian E Carpenter <[e-mail]>]

I put this aside just before the IETF:

On 05/11/2016 04:02, Toerless Eckert wrote:
> On Fri, Nov 04, 2016 at 08:33:43PM +1300, Brian E Carpenter wrote:
>>> GRASP should have a separate socket for each interface 
>>
>> How many interfaces do you mean? With no ACP, GRASP knows a reasonably
>> small number of physical interfaces, and has to open a socket for each of
>> the to send LL multicasts, and it has to listen for LL multicast from
>> all of them, which as far as I know requires a separate socket that is bound
>> to the relevant multicast address.
> 
> If each physcial interface connects to one ACP neighbor, the number
> of logical ACP interfaces would be the same as the number of physical
> interfaces. If an interface is a LAN connecting to N ACP neighbors,
> you would have N ACP virtual interfaces, one for each such neighbor.
> In an implementation of ACP that i could imagine.

Yes, that makes sense.

> Of course, on a LAN with N ACP neighbors, those would be eg: routers
> interconnected via a switch, so if that switch was autonomic itself, then
> you should only see one ACP neighbor, the switch. That the point of the
> discussion about GRASP instead of mDNS for ACP discovery, and of course,
> on the switch, we'd need to figure out how to intercept the ACP messages
> to make that work.
> 
>> In addition it has to open unicast sockets as and when needed, which may
>> be bound to any kind of address. (In the limited instances they will
>> be restricted to link-local unicast, but that's a special case.)
>>
>> I am still missing enough detail about the interfaces and socket options
>> that the ACP will provide to know how this will change when the ACP
>> is there. If there are, e.g., 400 autonomic nodes in the ACP and my node
>> has 4 physical interfaces, each of which has 10 link-local neighbors,
>> how many interfaces will the ACP offer me? 4, 40, 400?
> 
> Never 400. 4 if the L2 switches on the LANs are ACP capable, 40
> otherwise.

Fair enough. 40 is a bit scary (the most my prototype has ever seen is
2, but actually the code should work with 40). I think we should
RECOMMEND that the L2 switches support the ACP.

> 
>>> - but use that
>>> socket both for sending and receiving. No shared socket for reception.
>>
>> Why not? There's no problem identifying the source interface for a multicast,
>> it comes right back in recvfrom() (at least on Windows or Linux).
> 
> I probably should not have used "should" when i was recommending
> a separate socket per interface but instead:
> 
> -> To me the logic of dealing with multiple interfaces wold be most
>    easy if an implementation could have a separate socket for each
>    interface, physcial and ACP-channel (aka: per ACP neighbor).
>    With that approach, its clear at the socket level, which sockets
>    belong to which security domain, and you do not have to
>    enforce that via filtering when you receive messages.

Not sure it's an issue, but certainly I don't see a problem in doing
that - but again, better with 4 interfaces than 40.

> Socket API is a lot of confusing complexity. I haven't played with LL
> IP6 myself on linux, so i can only guess:
> 
> - Lets assume you bind your socket to the LL IPv6 address of an interface
>   via what the API doc calls the "scope ID" (which really is a zone ID).

Yes, Zone ID is the correct IPv6 terminology, but in any case it's
much cleaner to use interface numbers IMHO. My code figures that out
on both Windows and Linux, but they are quite different. In this area
Winsock seems to be superior to POSIX, even when viewed through
the Python library.

>   You should then be able to send/receive only from/to that interface.
> - I am not sure what impact this has on multicast. I was hoping that you
>   could still do the setsockopt to add the GRASP LL IPv6 multicasr group,
>   and in that API call you'd equally indicate your LL IPv6 addr and
>   would then send and receive IPv6 LL multicast only on that interface.

Yes, you must do IPV6_JOIN_GROUP for each interface separately,
with the right mreq structure. If I can inflict more Python on you,
this is how I join the group on all interfaces with the same socket
(slightly simplified):

for i in _ll_zone_ids:
    mreq = ALL_GRASP_NEIGHBOR_6.packed + struct.pack('@I', i)
    mcrsock.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_JOIN_GROUP, mreq)

so it would be easy to do that with separate sockets. You'd need a
thread per socket, however. With a single socket, you don't need
that.

> I have not tested this, so this is right now just conjecture.

No, it's correct for both POSIX and Winsock.

>>> GARSP only needs to figure out which interface is physcial and which
>>> is ACP. On router-OSs, each interface would be assigned to a VRF, so 
>>> it's easy to enumerate the interfaces of eg: the ACP being one VRF.

VRFs on Linux may turn out to be a dark art; a quick Google didn't
help much. I didn't get much out of
https://www.kernel.org/doc/Documentation/networking/vrf.txt

>>
>> I would assume that the adjacency table would provide that sort of clue.
>> To make my code agnostic about Winsock vs POSIX, I just use the interface
>> numbers, not the names, and they should be system-wide unique.
> 
> Have to look at your code..
> 
>>> Right. But this is not a new ACP thing. Any existing IETF protocol
>>> implementation that uses link-local addresses and runs on any type of
>>> any form of physical or virtual interface needs some form of interface
>>> representation at the API level to send/receive LL packets. Thats why
>>> i am somewhat hesitant to add explanatory text for this to the ACP
>>> draft. It's supposed to be implied by the IPv6 addressing model and
>>> common practice in IETF documents that its not regurgitated too much.
>>
>> No, it doesn't need much text but it really isn't obvious at all from
>> the ACP draft, I assure you.
> 
> Can you propose text ?

Not sure I can. But something along the lines of:

The ACP will appear to its users as a VRF instance that supports
various network interfaces which in turn support standard TCP/IP
sockets. These sockets can support link-local or global-scope
IPv6 addresses. Multicast support SHOULD be limited to link-local
operations. The way in which a program using the ACP detects that
the ACP is present and attaches itself to the VRF instance is
implementation-dependent.

IMHO you should also recommend an API for the detection/attachment
and for accessing the adjacency table, but like the GRASP API
that should be a separate document.

Regards
    Brian

> 
> Cheers
>     Toerless
> 
>>>> If you want, I can point you the exact Python code that handles this
>>>> in the prototype.
>>>
>>> Please do. Just not sure when i'll find time.
>>
>> Let me do that when I'm back from my short vacation.
>>
>> Rgds
>>    Brian
>>
>> _______________________________________________
>> Anima mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/anima
>