Re: [Taps] call for agenda items at TAPS IETF-104

["Holland, Jake" <[e-mail]>]

Hi Aaron,

Let’s take 15 minutes on the agenda for just YANG.

If there’s time, I would also be happy to take 10 minutes on multicast, but with a lower priority on the request.  There’s a better chance I can cover that on the list or in offline discussion.

Thanks,
Jake

From: Aaron Falk <[e-mail]>
Date: 2019-03-07 at 12:54
To: "Holland, Jake" <[e-mail]>
Cc: taps WG <[e-mail]>, Michael Welzl <[e-mail]>
Subject: Re: [Taps] call for agenda items at TAPS IETF-104


Jake-

I’m unclear, do you want to discuss multicast in Prague or just YANG?

--aaron

On 7 Mar 2019, at 14:10, Holland, Jake wrote:

Hi Michael,

Thanks for responding, much appreciated. (And also to Tommy
and Aaron.)

Based on the replies from you, Tommy, and Aaron, I'll plan
to present something in Prague, and can you please put me
on the schedule, Aaron?

More responses inline with <jh></jh>, and thanks, these
questions were very helpful to help me focus on what needs
explaining vs. what's obvious.


On 2019-03-07, 00:32, "Michael Welzl" <[e-mail]> wrote:

Hi,

Very sorry for the silence. I can only speak for myself, but here's an example of why this one person was silent:
- When you created your issue on multicast in github, I thought of answering (positively), but then thought that the repo is about to move, and it would probably be better to delay things until the move is finished.
- Then, the issue was overruled by your email. I read it and found it interesting, but hoped for someone else to answer because, frankly, I was afraid to make a fool of myself ... because I know almost nothing about YANG.

<jh>
Thanks for explaining, I'm sympathetic and grateful that you
overcame your concerns. I was in the same place on YANG not
too long ago, and I still have a ways to go, though I now think
I see some of its promise.

But I don't think I created a multicast issue on github? I just
noticed a few nits while reading and sent a pull request, nothing
really substantial yet.
</jh>

But now I'll be brave :) I'll go ahead and ask: how exactly is this YANG proposal more than just a syntax change? What would it give us?
(I understand that YANG can be automatically parsed / checked by some tools, but... what does THAT give us?)

<jh>
Syntax plus tool support is the only difference between machine code
and <your favorite programming language>, so even if that's the only
thing we got, it's not to be sneezed at. (And there is a lot of room
for growth here, but there are already tools that can generate API
header code from yang, e.g:
https://wiki.onosproject.org/display/ONOS/YANG+Tools<https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.onosproject.org_display_ONOS_YANG-2BTools&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=bqnFROivDo_4iF8Z3R4DyNWKbbMeXr0LOgLnElT1Ook&m=LczPM-Wc-j2Kbqd3CAeDoTyKXiXoMzepR7MDds1ApyA&s=a5YJn64q0PfyskIelpOtJkb5cHSgBA4NNTQZKqq6LbI&e=> )

But I'm actually pushing from the other direction; my understanding
of the current docs is that taps-arch is a high level design and goals
document, and taps-interface (and taps-minset) make an attempt to
lay out the concrete implementation details that would make taps-arch a
reality, if an implementation covers them.

My claim here is twofold:
- that taps-interface as it stands today is another high level design
document, which will have a necessarily fuzzy mapping to any actual
implementation.
- that taps-interface is _really_ _close_ to laying out the concrete
implementation details in a non-fuzzy way that can be directly used
by implementors, so I'm suggesting that taking that extra step is
highly valuable.

On the question of benefits: I'll lay out some of the ones
that seem important to me, but I'll first say that in theory,
these benefits could be realized by any concrete configuration
format. The benefit of YANG specifically comes from tool
support and the confidence from existing work that make it clear
it's a reasonable format for networking configuration.

Basically, because there's so much YANG work already done for
networking, using YANG lets us leverage some of that work in a way
other options don't.

In particular, there are a bunch of networking-specific concepts
that are already defined in detail. For example, the ipv6-address
type has a complicated regex that many implementors are likely to
get wrong if they try to roll their own (at least, I've certainly
done so, and seen it a bunch of times):
https://tools.ietf.org/html/rfc6991#page-19<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_rfc6991-23page-2D19&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=bqnFROivDo_4iF8Z3R4DyNWKbbMeXr0LOgLnElT1Ook&m=LczPM-Wc-j2Kbqd3CAeDoTyKXiXoMzepR7MDds1ApyA&s=XKOw2joxRICUZelSqbFB59Lhy1KKcRkCadcLwLUixfo&e=>

But by making the obvious choice of using yang tools to parse yang
stuff, implementors automatically get checking against that
(normative!) regex for free without having to notice and check it
themselves.

There's other side benefits, like that interfaces are themselves
explicitly defined (https://tools.ietf.org/html/rfc8343#section-5<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_rfc8343-23section-2D5&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=bqnFROivDo_4iF8Z3R4DyNWKbbMeXr0LOgLnElT1Ook&m=LczPM-Wc-j2Kbqd3CAeDoTyKXiXoMzepR7MDds1ApyA&s=FpQPWQE_tm7FaoIJkOwbuVzyAieHIl_tJYX3jtPfn5c&e=>),
and so where things like interface types might be used, the past
and future work of people who formalize and extend the set of
interface types gets automatically imported by referencing the
normative yang definition for network interfaces.

But to list a few of the specific benefits from having an explicit
config format:

- cross-platform portability:
If I've got a program using a taps api that talks to some service,
and now I want to write a new program in a different language to talk
to that service, I can export the config and load it in my new
program (even in a new language), and I can be reasonably sure
it'll form the same kind of connection.

Although it's possible to make it work anyway as long as the APIs
have a 1-to-1 mapping, normalizing on a config file format instead
of a list of features seems to me like it'll save me perhaps up to a
day or so of effort each time I have to do this, which when multiplied
by all the implementors in the world over the next decade or 2, starts
to add up.

- testability:
Again, this mostly saves time, but by having a defined input format,
it's possible for different implementations to use the same suite of
test configuration files to verify the same interpretation of their
contents. If each implementation has its own way of configuring the
features, it's not impossible to design a suite that can test them,
but it's vastly harder.

- completeness:
One of the points of confusion I had with taps-interface was that the
Local/RemoteEndpoints seemed both critical to an implementation and
very underspecified. Although it's possible to address that by raising
concerns specifically about the Local/Remote Endpoints, I'm not sure
there aren't other underspecified areas of the taps-interface draft.

I expect that a reference implementation based on parsing a config
format would uncover all such underspecified areas of configuration,
if the config format examples are used to describe all the supported
use cases.

(So part of my goal with this proposal is to solve this problem in
the general case, which implies that it'll also be solved for the
specific cases of Local/Remote Endpoints, though of course the
details for each instance of incompleteness are separate topics.)

There might be some other benefits, but I'll stop there, hoping the
point is sufficiently clear.
</jh>

Also, I actually see 3 separable things being proposed here:

1) the YANG model
<jh>
I'd phrase the current status of the proposal as an in-principle
"use a YANG model", rather than "use this specific YANG model".

The current draft is kind of a throw-away example proof of concept,
which is certainly very incomplete, and some of my reading yesterday
suggests that it's actually not very well structured either. I've
probably already made a fool of myself to any actual YANG experts,
that ever try to read it, but such are the risks of posting public
comments... :)
</jh>

2) multicast support (I find your conclusion that not much needs to change interesting! Though the example you're giving (joining an SSM channel) is only a part of what we'd need, as you also say...)
<jh>
Fair point, this really is a separate topic. I brought it up
really only because it's a motivating factor in my participation
here, not because it's a primary factor for taps in general. Let's
drop it for this thread, and I'll plan to raise it as a separate
issue when the yang questions are settled and I've got something
more thought out.
</jh>

3) applying preferences to addresses and port numbers (which you seem to take for granted in your draft, but which I don't think is supported by our current document).

<jh>
Yes, this is a case in point for my concerns about incompleteness
on the Endpoint specifications.

I wasn't sure how to treat this, so I'm not surprised if I got it
wrong. I was sort of pattern-matching on the rest of what the spec
does, plus guessing at intended meanings of this paragraph from
https://tools.ietf.org/html/draft-ietf-taps-interface-02#section-5.1<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dietf-2Dtaps-2Dinterface-2D02-23section-2D5.1&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=bqnFROivDo_4iF8Z3R4DyNWKbbMeXr0LOgLnElT1Ook&m=LczPM-Wc-j2Kbqd3CAeDoTyKXiXoMzepR7MDds1ApyA&s=Rjjn3OmruCXDbKwgDmAMqHvne-UHNqa_gmIk_g9z6ag&e=>:

Multiple endpoint identifiers can be specified for each Local
Endpoint and Remote Endpoint. For example, a Local Endpoint could be
configured with two interface names, or a Remote Endpoint could be
specified via both IPv4 and IPv6 addresses. These multiple
identifiers refer to the same transport endpoint.

By the time I finished making the examples, I had noticed I was
maybe doing it wrong, but I didn't want to bother fixing it unless
people agree it's worth spending effort in this general direction, and
I didn't want to hold off on opening the discussion. Of course, it
should be changed to match the consensus of the wg on what belongs
here.
</jh>

Side note: unless I'm mistaken, this wouldn't fit our structure well: e.g. a port number would then be a Transport Property that has a certain value, but also has a preference, but currently we say that a Transport Property has "one of a set of data types", one of which is a Preference. Isn't that structure too limiting? Or am I missing something?

<jh>
I'm not sure. This is part of what I mean when I say I find the
current taps-interface a little too loose. I _think_ what I posted
is a valid (though perhaps not optimal) interpretation of the current
text. I might be wrong about that, so I'd be grateful for a pointer
to the text that prohibits this interpretation.

But really, that's beside the point. I'd be even more grateful for an
alternate proposal that more closely matches the intended design for
a taps API, with the same level of concreteness.
</jh>

I guess that 2) needs 3), but perhaps it's useful to see 2) and 3) as separate... maybe there are other use cases for 3) alone ?

<jh>
Yes, I think so. I think I understand an ultimate goal for taps to
be "as an app designer, don't worry about which transport, just
send the magic API to go discover one that works".

However, it seems to me in the interim that it'll be necessary
to specify port numbers in order to interoperate with a lot of
existing services, especially in cases like RTP (or a lot of
other generalized UDP stuff), where there's not an explicit
service-to-port mapping. (Also to interoperate with a lot of
other deployed systems like firewalls.)

I have a few nebulous ideas about how explicit layering might be a
good way to solve this, but I think in order to make the ideas sharp
enough to evaluate, I thought I'd first try the easier task of making
taps sharp enough that I can express the ideas as proposed changes
to an existing well-formed concept.
</jh>

IMO, all of these things are interesting, and would be good to discuss on site. However, I doubt that we can deal with them all in only 15 minutes :-)

<jh>
Fair point.

I propose during the taps meeting to check preliminary consensus about
moving forward with a YANG model within taps, and if so whether to
integrate, replace, or make a paired doc with the existing taps-interface.
Separately, I'll encourage those interested and available to spend some
time in offline clarifying discussion during the week prior.

Cheers,
Jake
</jh>

Cheers,
Michael

PS: Travis is down, or something. At least the "Editor's Copy" links don't currently work.



> On 7 Mar 2019, at 04:55, Holland, Jake <[e-mail]> wrote:
>
> Hi taps,
>
> (Trying again, but simpler.)
>
> I'm looking for a consensus yes or no answer:
>
> Is a normative config input format an interesting and useful
> direction?
>
>
> The idea is to add functionality like this, in taps-interface:
>
> newPre = PreConnection.NewFromJson('''{
> "remote-endpoint":{
> "hosts":[
> { "host":"example.com" }
> ]
> }
> }''')
>
> With a fully specified json input format that can provide all the
> configurable values.
>
>
> If no, I'll move on and assume I just don't understand taps goals.
>
> If yes, I'd like 15 minutes to discuss in Prague, and keep reading:
>
> I think a full definition of an input json format can be exactly
> specified by a YANG model. (With xml for free, if you want it.)
>
> I tried to sketch a start at what a YANG model for this might look like:
> https://tools.ietf.org/html/draft-jholland-taps-api-yang-00#section-3<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Djholland-2Dtaps-2Dapi-2Dyang-2D00-23section-2D3&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=bqnFROivDo_4iF8Z3R4DyNWKbbMeXr0LOgLnElT1Ook&m=LczPM-Wc-j2Kbqd3CAeDoTyKXiXoMzepR7MDds1ApyA&s=1Wpxcyfu6zxEvt5HoIG3vnP_p00PGCpQb3mYYvXJ7SY&e=>
>
> I'm not a YANG expert yet, and it's not much of a model. And it's very
> far from complete. But it compiles, and all the examples in the draft
> validate against that model with libyang. If it's worthwhile, I think
> something like this can be good (and extensible!), if done right.
>
> I believe something along these lines would sharpen up taps-interface
> a lot. (After filling in all the taps-interface properties.)
>
>
> The reason I'm asking is because right now, taps-interface to me
> seems _almost_ really good, except too confusing and vague to actually
> build an API that can replace BSD sockets. I think with a solid config
> format with normative and testable examples, that could be fixed.
>
> If consensus says "interesting", I'll want 15 minutes to discuss it, and
> to start digging into how to make it good. (And also to add multicast
> support to the model and at least one implementation.)
>
> Thanks for your consideration.
>
>
> Cheers,
> Jake
>
>
>
> On 2019-03-04, 17:19, "Holland, Jake" <[e-mail]> wrote:
>
> Hi taps folks,
>
> TL;DR: I think the spec should have a YANG model and I want to use it
> to get multicast support in an implementation.
>
>
> I recently started digging into the taps API with the intent of adding
> multicast support, but it looks like it's basically already there, as
> far as it goes[*].
>
> But I ended up with a higher-level comment, so I thought I'd raise it
> to the wg and ask what you all think.
>
> I found the whole "abstract interface" approach a little too
> loosey-goosey, so I thought I'd try to suggest a way to tighten it up.
>
> My goal with this is to make it much more clear (to the point of being
> mechanically checkable) precisely what a compliant API provides.
>
> I'm not attached to the structure I'm proposing or to any of the
> particulars in the straw-man I've posted, but if it's not tightened up
> with something at a similar level of concreteness, I'm concerned that
> different implementations will be not only incompatible in random
> underspecified corner cases (like BSD sockets today when you try to make
> cross-platform C code), but also are likely to end up with very many
> important differences that would make the whole taps effort more or less
> moot.
>
> In a world where we end up with a doc at the level of abstraction I'm
> currently seeing in draft-taps-interface, it seems to me that if 2
> different API implementations were written in the same language, it'll
> be prohibitively difficult for an app to migrate from using one to using
> the other, just because so many aspects of it are left open to the
> implementors.
>
> In that context, I thought a YANG model would be useful here to
> provide a cross-platform way to specify what exact properties and
> objects exist, an exact format in which the values can be specified, and
> what exact semantics they have, while still allowing for a sane
> extension path and language-specific implementation details.
>
> I'm thinking some language a bit like the first bullet in Section 4.2 of
> taps-interface:
>
> A compliant implementation SHOULD provide a language-appropriate way to
> configure a PreConnection using YANG instance data for this model, and
> SHOULD provide an API that outputs the YANG instance data for an
> established Connection.
>
> An implementation MAY also provide appropriate APIs for directly editing
> the objects without using YANG. It's RECOMMENDED where possible to use
> names that mechanically translate to the names in the YANG data model,
> using capitalization and punctuation conventions as expected for the
> language of the implementation.
>
> And then of course a YANG model:
> https://datatracker.ietf.org/doc/draft-jholland-taps-api-yang/<https://urldefense.proofpoint.com/v2/url?u=https-3A__datatracker.ietf.org_doc_draft-2Djholland-2Dtaps-2Dapi-2Dyang_&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=bqnFROivDo_4iF8Z3R4DyNWKbbMeXr0LOgLnElT1Ook&m=LczPM-Wc-j2Kbqd3CAeDoTyKXiXoMzepR7MDds1ApyA&s=9-wM860yuimc8fbNdS7OKt9ViZprAvUD_sSM6YJNy4E&e=>
> (draft-jholland-taps-api-yang)
>
> If this seems useful, it will need lots of refining. I'll be surprised
> if any part of it survives exactly as written. It's a quick and
> dirty attempt to concretize a few of the things I saw listed in
> draft-ietf-taps-interface, as a starting point to fill out if it seems
> useful.
>
> But the model parses, and the example data instances in the draft
> parse against the model (all with libyang).
>
> One of the main reasons I'm doing this is because it seems to me what's
> specified in taps-interface-02 today is missing some key features, like
> an enumeration of the properties in the Local/RemoteEndpoint examples in
> section 5.1. And I don't see that listed as an open issue in github,
> which surprised me a bit.
>
> I think oversights like this will become immediately and painfully
> obvious when there's a reference implementation that includes a YANG
> parser and an explicit data model, as opposed to the combing of the
> document and a sort of eyeballed comparison to NEAT that I tried
> this week to reach that conclusion (which I found challenging even
> though I thought both the document and the library were mostly pretty
> well written).
>
> The whole thing at this point just smells to me much more abstract than
> it really has to be or than it's really useful to be, which bothers me
> because the idea of replacing BSD sockets with something usable seems
> like such a great idea. I'd like this to be something I can actually
> use in a way that makes my life easier someday soon.
>
> But I think I'm at the point where I need a sanity check to see if I'm
> just missing something, or if this seems like a useful direction.
>
> Thoughts? Suggestions? Worth discussing in Prague? (If so, can I
> get a slot?)
>
> Cheers,
> Jake
>
> *:
> I concluded that there's no reason multicast couldn't be supported
> today, if there were an implementation that could reasonably claim to be
> compliant, by just adapting some of the examples in
> draft-ietf-taps-interface-02 and understanding the semantic meaning of
> multicast address spaces inside the API.
>
> For example, I couldn't find any reason this can't be expected to set
> up an SSM channel subscription without any further ado, given a sane
> implementation that supports it:
>
> RemoteSpecifier := NewRemoteEndpoint()
> RemoteSpecifier.WithIPv4Address(192.0.2.21)
>
> LocalSpecifier := NewLocalEndpoint()
> LocalSpecifier.WithPort(30000)
> LocalSpecifier.WithIPv4Address(232.252.0.2)
>
> NewPreconnection(RemoteSpecifier, LocalSpecifier).Listen(...)
>
> Maybe there's some value in specifying a "JoinSSM()" to override defaults
> in the PreConnection, just to make sure you're specifically asking for
> multicast. I think that would be fine for native multicast, but like I
> said, a much smaller point than the looseness of the API.
>
> Where it gets a bit more complicated is trying to handle multiple options
> for discovering a usable unicast tunnel for multicast traffic, as
> described in Section 2.4.1 of draft-ietf-mboned-driad-amt-discovery-01.
>
> I'd like to have a decent place to tack on an extension to this API that
> can transparently, within the connection API, discover the best available
> AMT relay and start using it when native joining is unavailable (and also
> to provide normative controls for configuring it when there's
> administrative configuration to be added).
>
> But that's a 2nd order question for me at this point, because in the
> current TAPS API I don't see any obviously good spot to put selection
> controls for that kind of tunnel discovery selection, or really a good
> way to explain what it's supposed to do, if I tried to add controls to
> something that's there.
>
> Solving that is my main motivation for being here. (Well, and that
> the BSD socket API for multicast is kind of a disaster today.)
>
> Anyway, if taps finds the whole YANG suggestion useful, I'll probably
> suggest some new extensions about this, and maybe a few other points,
> especially maybe around trying to put in a structure that can support
> some kind of sane explicit layering.
>
> But I'm not sure I can articulate those suggestions in a way I'm sure is
> meaningful without first getting a more clear specification nailed down
> about what's actually in the taps spec. Because right now I'm mostly
> just confused about what an API implementation would really look like,
> and how you could tell whether it matches the taps-interface spec.
>
>
>
> From: Aaron Falk <[e-mail]>
> Date: 2019-03-04 at 10:53
> To: taps WG <[e-mail]>
> Cc: Zaheduzzaman Sarker <[e-mail]>
> Subject: [Taps] call for agenda items at TAPS IETF-104
>
> Hi All-
> What should we use our time to discuss? Let’s focus on things that would benefit from f2f discussion, consensus building, or just argument. :)
> • TAPS docs: are there open topics that need group attention? Seems like we settled most of the remainder at the interim.
> • TAPS security: this seems nearly done. Anything to discuss?
> • Implementations: a good topic for information sharing but less important than anything needing agreement
> • Mobility: https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dietf-2Ddmm-2Dondemand-2Dmobility-2D17&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=bqnFROivDo_4iF8Z3R4DyNWKbbMeXr0LOgLnElT1Ook&m=pLW5gSyetfVe_ixG4_u7qKX_VcjIqzN7Ju2BgM2rpQo&s=HUsBVBF_GhNiOk3gqY_m5qZMD-sPmBJ93GE5wd3D5_s&e= and https://urldefense.proofpoint.com/v2/url?u=https-3A__datatracker.ietf.org_doc_review-2Dietf-2Ddmm-2Dondemand-2Dmobility-2D15-2Dtsvart-2Dlc-2Dwesterlund-2D2019-2D01-2D08_&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=bqnFROivDo_4iF8Z3R4DyNWKbbMeXr0LOgLnElT1Ook&m=pLW5gSyetfVe_ixG4_u7qKX_VcjIqzN7Ju2BgM2rpQo&s=0YmC_XCAu4_GVYdFi0HxiKaKBpan2COYqBL1mB6bXrY&e=
> --aaron
>
>
>
> _______________________________________________
> Taps mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/taps<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_taps&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=bqnFROivDo_4iF8Z3R4DyNWKbbMeXr0LOgLnElT1Ook&m=LczPM-Wc-j2Kbqd3CAeDoTyKXiXoMzepR7MDds1ApyA&s=FgaZPxNY9pMUtlexpSvPSGEsLVa0Pk7aLVN7OA9sgX0&e=>



_______________________________________________
Taps mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/taps<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_taps&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=bqnFROivDo_4iF8Z3R4DyNWKbbMeXr0LOgLnElT1Ook&m=LczPM-Wc-j2Kbqd3CAeDoTyKXiXoMzepR7MDds1ApyA&s=FgaZPxNY9pMUtlexpSvPSGEsLVa0Pk7aLVN7OA9sgX0&e=>