Re: [arch-d] Comments on draft-iab-protocol-maintenance-03

["Martin Thomson" <[e-mail]>]

Hey SM,

On Thu, May 9, 2019, at 11:11, S Moonesamy wrote:
> How can one determine whether critical details have been omitted 
> (Section 3) when the protocol designer interacts with the 
> implementer?  The specification may be clear to the reviewer as what 
> he/she considers as obvious might be viewed as a critical detail by 
> someone who does not regularly follow IETF discussions.  

I don't think there is any real science here.  People can try to talk to each other.  As you say, you need feedback to identify those things.  Yes, some things seem obvious to some and not others, but a conversation can go a long way to understanding and reconciling these little miscommunications.

See https://github.com/quicwg/base-drafts/issues/2671 for a recent example of how obvious isn't always very obvious.  

> According to one of the examples given in the draft the 
> protocol maintenance took eight years.  Why weren't the specification 
> shortcomings addressed within a few years after publication?

Mostly because no one bothered for a long time.  There was no community and no maintenance for many years.  When a group of people finally decided to fix the rotten thing, the pile of work was massive and the deployed base crufty.  We're talking HTTP/1.1 here, which is a big protocol with lots of woolly bits.  It has more woolly bits thanks to that period of neglect, but the community is slowly and carefully clawing away the worst of the rot and I think that good progress has been made in the past 10 years.  It's slow and deliberate, but I think that's the right approach.

> Fatal conditions (Section 3) can cause a product failure.  One of the 
> issues is getting two or more parties to agree on who is responsible 
> for fixing the fatal condition.  It is simpler to reduce the 
> likelihood of a failure in comparison with convincing the protocol 
> designer or the implementer to fix the issue.

Yes it is indeed simpler.  But that's what builds up the bad mojo and so I would have us reject that notion as a first option.  FWIW, I've found it a fairly pleasant experience dealing with people on the other end of protocols.  If we agree that there is a bug (which is often the case), it gets fixed.  I've had a few cases where it didn't get fixed very fast, but we tend to work out a plan for managing the problem.  Things usually work out better in the end that way.

> It is difficult to get approval for a protocol change (Section 4) as 
> there is very little interest in revisiting the specification once it 
> has been published.

Yes.  That's a huge problem.  That's why I'm a big fan of efforts from the IESG to smooth the path for updates (draft-roach-bis-documents, the work on finding referenceable interoperable targets, and the associated work).  Institutionally, the IETF is not very well suited to this.  Despite that, we've been able to make the process work in a few groups.

> The draft mentions the ecosystem (Section 4).  Is there a 
> relationship between interoperability and competition [1][2] in the ecosystem?

Yes, but I really didn't want to get into that.  To a first approximation, I find that the desire for interoperability is the overriding concern in most cases.  However, we routinely get into situations where competitive pressures might affect the process of reaching an amicable conclusion.  That's why we have the IETF and other such places, and the process

> What would be the impact if every piece of software had telemetry 
> built in (Section 7.1)?

Probably a whole lot of better-informed developers and a whole lot of software that is built for use cases that matter.  There's a privacy risk, of course, but I know that Mozilla wouldn't survive without telemetry.  We have a little bit, but it is never enough.

> Shouldn't those errors be caught during 
> testing instead of the software collecting information globally?

Testing is awesome, but tests are fallible just like specifications.  Test the wrong thing and you get the wrong behaviour.  And it's possible that tests miss something entirely.  Telemetry can give you insight into problems if you ask the right questions (a difficult skill, to be sure), and provide a useful barometer for transient problems.  I doubt that you would find any server operator out there who could operate without a feed of CPU and network utilization from server instances.  What is measured differs depending on what your system does, but it's all the same principle: the longer and looser the feedback loop, the harder the system is to operate.

> As an overall comment, the protocol examples in the draft are mainly 
> web related.  How does protocol maintenance work in other areas?  Was 
> interoperability [3] more of an advantage instead of a disadvantage?

Nah, this is mostly just my own bias.  As others have pointed out, we don't know if that's a problem.  The next version will probably have some sort of caveat on it to that effect.  There are lots of good examples out there, but it's hard finding concise descriptions of them.  (They tend to read like Paul Wouters' tale of IPsec woe.)

I don't quite know what you refer to in Mozilla's FTC submission there.  There's a lot going on.