Re: [Bpf] IETF BPF working group draft charter

[Dave Thaler <[e-mail]>]

> -----Original Message-----
> From: David Vernet <[e-mail]>
> Sent: Wednesday, May 17, 2023 2:00 PM
> To: Dave Thaler <[e-mail]>
> Cc: Jose E. Marchesi <[e-mail]>; Dave Thaler
> <dthaler=[e-mail]>; [e-mail]; bpf
> <[e-mail]>
> Subject: Re: [Bpf] IETF BPF working group draft charter
> 
> On Wed, May 17, 2023 at 06:19:42PM +0000, Dave Thaler wrote:
> > Jose E. Marchesi wrote:
> > > As I mentioned during your talk at LSF/MM/BPF, I think that two
> > > items may be a bit confusing, and worth to clarify:
> > >
> > >   * the eBPF bindings for the ELF executable file format,
> > >
> > > What does "eBPF bindings" mean in this context?  I think there are
> > > at least two possible interpretations:
> > >
> > > 1) The way BPF uses ELF, not impacting internal ELF structures.  For
> > >    example the special section names that a conformant BPF loader
> > >    expects and understands, such as ".probes", or rules on how to use
> > >    the symbols visibility, or how notes are used (if they are used)
> > > etc
> > >
> > > 2) The ELF extensions that BPF introduces (and may introduce at some
> > >    point) as an architecture, such as machine number, section types,
> > >    special section indices, segment types, relocation types, symbol
> > >    types, symbol bindings, additional section and segment flags, file
> > >    flags, and perhaps structures of the contents of some special
> > >    sections.
> >
> > See
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.
> > ietf.org%2Farchive%2Fid%2Fdraft-thaler-bpf-elf-
> 00.html&data=05%7C01%7C
> >
> dthaler%40microsoft.com%7Cddd426f86cb4489c17d108db5719c1cc%7C72f9
> 88bf8
> >
> 6f141af91ab2d7cd011db47%7C1%7C0%7C638199540383552382%7CUnkno
> wn%7CTWFpb
> >
> GZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6
> Mn0
> >
> %3D%7C3000%7C%7C%7C&sdata=hM81uNUjJP2J46eSs6tLPi7B82Z7EhlJrGUS
> 2YNo9Bk%
> > 3D&reserved=0 It includes the values used in the ELF header, section
> > naming, use of the "license" and "version" sections, meaning of "maps"
> > and ".maps" sections, etc.
> 
> I have what may be a silly question: The document you linked specifies,
> 
> "This specification is a [sic] extension to the ELF file format as specified in
> Chapter 4 of the System V Application Binary Interface [ELF]."
> 
> What does it mean exactly for an IETF-published BPF ELF extension to be an
> extension of a specification from a totally separate standards body (in this case,
> the System V release 4 ABI / Tool Interface Standard (TIS))? 

Perhaps "extension" is not the correct word.   At least that word is not in the
draft charter.  If the document is changed to say "bindings" then it would be
consistent with the draft charter, but there may be a better term.

As Alexei explained at LSF/MM/BPF last week, this isn't a change to the ELF
spec, and my understanding is that it needn't be done by the ELF standards body,
but I am not the expert there.

> In other words, is it
> normal for extensions to be specified in external / separate standards bodies
> from where the original specification is defined? It seems like that could
> potentially result in a confusing outcome if the original standards body could
> itself eventually choose to publish its own extension which conflicts with the
> IETF. That won't happen for Sys V of course, but in general it seems odd for us
> to publish an extension for a specification that was defined in the System V ABI
> instead of the IETF, and it seems like a situation for which following the existing
> contours for x86_64, ARM, etc might make sense.
> 
> > > If the intended meaning of that point in the draft is 1), then I
> > > would suggest to change the wording to something like:
> > >
> > > * the requirements and expectations that ELF files shall fulfill so they
> > >   can be handled by conformant eBPF implementations.
> >
> > My own opinion is to leave the more detailed definition of what
> > belongs in the ELF spec vs another document up to the WG to define
> > rather than baking it into the charter.
> 
> I tend to agree, but that seems to suggest that we should remove this line from
> the charter, and instead leave it up to the WG to determine if it should be
> included:
> 
> * the eBPF bindings for the ELF executable file format,

Removing it would be problematic in my view (unless we remove a phrase
Warren commented on) since the draft charter says:
> The working group shall not adopt new work until these
> documents have progressed to working group last call.

So if the WG does need to do it, it couldn't adopt it without keeping this bullet.

> Or is your point rather that the line in the charter as it exists now is really
> saying that discussing ELF *in general* is in scope for the working group, but
> that we may or may not end up actually producing a document for it depending
> on how discussions go in the WG, and that if we did produce a document, the
> scope would be decided by the WG?

Yes, that would be another valid interpretation.  My opinion is that the IETF
should produce a document.

> More broadly, would you mind please clarifying exactly what this section will
> imply for the WG (see below for more details on my question):
> 
> > The working group will produce one or more documents on the following
> > work item topics:
> >
> > * The eBPF instruction set architecture (ISA) that defines the
> >   instructions and low-level virtual machine for eBPF programs,
> >
> > * Verifier expectations and building blocks for allowing safe execution
> >   of untrusted eBPF programs,
> >
> > * the BPF Type Format (BTF) that defines debug information and
> >   introspection capabilities for eBPF programs,
> >
> > * the eBPF bindings for the ELF executable file format,
> >
> > * the platform support ABI, including calling convention, linker
> >   requirements, and relocations,
> >
> > * Cross-platform map types allowing native data structure access from
> >   eBPF programs,
> >
> > * Cross-platform helper functions, such as for manipulation of maps,
> >
> > * Cross-platform eBPF program types that define the higher level
> >   execution environment for eBPF programs,
> >
> > * and an architecture and framework informational document.
> 
> As far as I understand, if a topic is missing from this section, it doesn't
> automatically mean that it's out of scope for the WG to produce a document
> for it. If that's the case, and part of the job of the WG will be to specify what is
> actually in scope regardless of what's enumerated here, I'm not quite following
> why this section is necessary beyond providing the reader with some informal
> context on what BPF is in general.

The above section is typical in IETF WG charters.  And the bit I quoted earlier
about not doing additional things as work items until this list is done, is also
somewhat typical.
 
> Thanks in advance for explaining these concepts to an IETF noobie.
> 
> > > Otherwise, if the intended meaning in the draft charter is to cover
> > > 2), I would like to note that, usually and conventionally ELF
> > > extensions introduced by architectures (and operating systems in the
> > > ELF sense)
> > > are:
> > >
> > > - Part of the psABI (chapter Object Files).
> > >
> > > - Not standards, in the sense that these are not handled by
> > >   standardization bodies.
> > >
> > > - Maintained by corporations, associations, and/or community groups, and
> > >   published in one form or another.  A few examples of both arch and os
> > >   extensions:
> > >
> > >   + The x86_64 psABI, including the ELF bits, is maintained by Intel
> > >     (mainly by HJ Lu, a toolchain hacker) and available in a git repo in
> > >     gitlab [1].
> > >
> > >   + The risc-v psABI, including the ELF bits, is maintained by I believe
> > >     RISC-V International and the community, and is available in a git
> > >     repo in github [2].
> > >
> > >   + The GNU extensions to the gABI, including the ELF bits, is
> > >     maintained by GNU hackers and available in a git repo in sourceware
> > >     [3].
> > >
> > >   + The llvm extensions to ELF, which in this case take the form of an
> > >     "os" in the ELF sense even if it is not an operating system, are
> > >     maintained by the LLVM project and available in the
> > >     docs/Extensions.rst file in the llvm source distribution.
> > >
> > >   Note that more often than not this is kept quite informally, without
> > >   the need of much bureocratic overhead.  A git repo in github or the
> > >   like, maintained by the eBPF foundation or similar, would be more than
> > >   enough IMO.
> >
> > To ensure interoperability, I'd want a slightly more formal specification.
> 
> I understand the desire and need for ensuring interoperability, but if specifying
> a BPF ELF extension would be the exception to the rule for the entirety of the
> rest of the industry when it comes to ELF, I think we should consider also being
> explicit about what's different for BPF.

Others probably understand ELF processes better than I do.  But I think
the belief is that it's not an extension.  If there is, and there's another place
it needs to be done, then acknowledging that in terms of who the WG will
interact with might be helpful.

> > > - Open to suggestions and contributions from the community, vendors,
> > >   implementors, etc.  This usually involves having a mailing list where
> > >   such suggestions can be sent an discussed.  Almost always very little
> > >   discussion is required, if any, because the proposed extension has
> > >   already been agreed and worked on by the involved parties: toolchains,
> > >   consumers, etc.
> > >
> > > - Continuously evolving.
> > >
> > > So, would the IETF working group be able to accomodate something
> > > like the above?  For example, once a document is officially
> > > published by the working group, how easy is it to modify it and make
> > > a new version to incorporate something new, like a new relocation type for
> example?
> > > (Apologies for my total ignorance of IETF business :/)
> >
> > There's 3 ways:
> > 1) The IETF can publish an extension spec with additional optional features.
> > 2) The IETF can publish a replacement to the original (not usually
> > desirable)
> > 3) The IETF can define a process for other organizations or vendors to
> > create their own extensions, and some mechanism for ensuring that two
> > such extensions don't collide using the same codepoint.  This is what
> > the charter implies the WG should do.
> 
> This certainly seems useful, but it also feels like ELF is kind of a special case
> here given that it was originally published as part of Sys V, and there are no
> formally specified extensions for other much larger architectures. I may be
> missing a lot of context here, so thanks in advance again for filling in any gaps.
> 
> - David

Dave