Re: [Bpf] IETF BPF working group draft charter

[David Vernet <[e-mail]>]

On Wed, May 17, 2023 at 06:19:42PM +0000, Dave Thaler wrote:
> Jose E. Marchesi wrote:
> > As I mentioned during your talk at LSF/MM/BPF, I think that two items may be
> > a bit confusing, and worth to clarify:
> >
> >   * the eBPF bindings for the ELF executable file format,
> >
> > What does "eBPF bindings" mean in this context?  I think there are at least two
> > possible interpretations:
> >
> > 1) The way BPF uses ELF, not impacting internal ELF structures.  For
> >    example the special section names that a conformant BPF loader
> >    expects and understands, such as ".probes", or rules on how to use
> >    the symbols visibility, or how notes are used (if they are used) etc
> >
> > 2) The ELF extensions that BPF introduces (and may introduce at some
> >    point) as an architecture, such as machine number, section types,
> >    special section indices, segment types, relocation types, symbol
> >    types, symbol bindings, additional section and segment flags, file
> >    flags, and perhaps structures of the contents of some special
> >    sections.
> 
> See https://www.ietf.org/archive/id/draft-thaler-bpf-elf-00.html
> It includes the values used in the ELF header, section naming,
> use of the "license" and "version" sections, meaning of "maps" and
> ".maps" sections, etc.

I have what may be a silly question: The document you linked specifies,

"This specification is a [sic] extension to the ELF file format as
specified in Chapter 4 of the System V Application Binary Interface
[ELF]."

What does it mean exactly for an IETF-published BPF ELF extension to be
an extension of a specification from a totally separate standards body
(in this case, the System V release 4 ABI / Tool Interface Standard
(TIS))? In other words, is it normal for extensions to be specified in
external / separate standards bodies from where the original
specification is defined? It seems like that could potentially result in
a confusing outcome if the original standards body could itself
eventually choose to publish its own extension which conflicts with the
IETF. That won't happen for Sys V of course, but in general it seems odd
for us to publish an extension for a specification that was defined in
the System V ABI instead of the IETF, and it seems like a situation for
which following the existing contours for x86_64, ARM, etc might make
sense.

> > If the intended meaning of that point in the draft is 1), then I would suggest to
> > change the wording to something like:
> >
> > * the requirements and expectations that ELF files shall fulfill so they
> >   can be handled by conformant eBPF implementations.
> 
> My own opinion is to leave the more detailed definition of what belongs
> in the ELF spec vs another document up to the WG to define rather than
> baking it into the charter.

I tend to agree, but that seems to suggest that we should remove this
line from the charter, and instead leave it up to the WG to determine if
it should be included:

* the eBPF bindings for the ELF executable file format,

Or is your point rather that the line in the charter as it exists now is
really saying that discussing ELF *in general* is in scope for the
working group, but that we may or may not end up actually producing a
document for it depending on how discussions go in the WG, and that if
we did produce a document, the scope would be decided by the WG?

More broadly, would you mind please clarifying exactly what this section
will imply for the WG (see below for more details on my question):

> The working group will produce one or more documents on the following
> work item topics:
>
> * The eBPF instruction set architecture (ISA) that defines the
>   instructions and low-level virtual machine for eBPF programs,
>
> * Verifier expectations and building blocks for allowing safe execution
>   of untrusted eBPF programs,
>
> * the BPF Type Format (BTF) that defines debug information and
>   introspection capabilities for eBPF programs,
>
> * the eBPF bindings for the ELF executable file format,
>
> * the platform support ABI, including calling convention, linker
>   requirements, and relocations,
>
> * Cross-platform map types allowing native data structure access from
>   eBPF programs,
>
> * Cross-platform helper functions, such as for manipulation of maps,
>
> * Cross-platform eBPF program types that define the higher level
>   execution environment for eBPF programs,
>
> * and an architecture and framework informational document.

As far as I understand, if a topic is missing from this section, it
doesn't automatically mean that it's out of scope for the WG to produce
a document for it. If that's the case, and part of the job of the WG
will be to specify what is actually in scope regardless of what's
enumerated here, I'm not quite following why this section is necessary
beyond providing the reader with some informal context on what BPF is in
general.

Thanks in advance for explaining these concepts to an IETF noobie.

> > Otherwise, if the intended meaning in the draft charter is to cover 2), I would
> > like to note that, usually and conventionally ELF extensions introduced by
> > architectures (and operating systems in the ELF sense)
> > are:
> >
> > - Part of the psABI (chapter Object Files).
> >
> > - Not standards, in the sense that these are not handled by
> >   standardization bodies.
> >
> > - Maintained by corporations, associations, and/or community groups, and
> >   published in one form or another.  A few examples of both arch and os
> >   extensions:
> >
> >   + The x86_64 psABI, including the ELF bits, is maintained by Intel
> >     (mainly by HJ Lu, a toolchain hacker) and available in a git repo in
> >     gitlab [1].
> >
> >   + The risc-v psABI, including the ELF bits, is maintained by I believe
> >     RISC-V International and the community, and is available in a git
> >     repo in github [2].
> >
> >   + The GNU extensions to the gABI, including the ELF bits, is
> >     maintained by GNU hackers and available in a git repo in sourceware
> >     [3].
> >
> >   + The llvm extensions to ELF, which in this case take the form of an
> >     "os" in the ELF sense even if it is not an operating system, are
> >     maintained by the LLVM project and available in the
> >     docs/Extensions.rst file in the llvm source distribution.
> >
> >   Note that more often than not this is kept quite informally, without
> >   the need of much bureocratic overhead.  A git repo in github or the
> >   like, maintained by the eBPF foundation or similar, would be more than
> >   enough IMO.
> 
> To ensure interoperability, I'd want a slightly more formal specification.

I understand the desire and need for ensuring interoperability, but if
specifying a BPF ELF extension would be the exception to the rule for
the entirety of the rest of the industry when it comes to ELF, I think
we should consider also being explicit about what's different for BPF.

> > - Open to suggestions and contributions from the community, vendors,
> >   implementors, etc.  This usually involves having a mailing list where
> >   such suggestions can be sent an discussed.  Almost always very little
> >   discussion is required, if any, because the proposed extension has
> >   already been agreed and worked on by the involved parties: toolchains,
> >   consumers, etc.
> >
> > - Continuously evolving.
> >
> > So, would the IETF working group be able to accomodate something like the
> > above?  For example, once a document is officially published by the working
> > group, how easy is it to modify it and make a new version to incorporate
> > something new, like a new relocation type for example?
> > (Apologies for my total ignorance of IETF business :/)
> 
> There's 3 ways:
> 1) The IETF can publish an extension spec with additional optional features.
> 2) The IETF can publish a replacement to the original (not usually desirable)
> 3) The IETF can define a process for other organizations or vendors to create
> their own extensions, and some mechanism for ensuring that two such
> extensions don't collide using the same codepoint.  This is what the charter
> implies the WG should do.

This certainly seems useful, but it also feels like ELF is kind of a
special case here given that it was originally published as part of Sys
V, and there are no formally specified extensions for other much larger
architectures. I may be missing a lot of context here, so thanks in
advance again for filling in any gaps.

- David