Re: [httpapi] RateLimit Header client implementations

[Alex Martinez <[e-mail]>]

Hi Darrel,

On Tue, Jan 5, 2021 at 4:17 PM Darrel Miller <[e-mail]>
wrote:

>
> My assertion is that if we discouraged API providers from punishing
> consumer for reaching a rate limit, and we promote the use of the
> retry-after as a signal to the client to back-off before making another
> call, then there are fewer scenarios where a client is required to
> constantly monitor "the number of calls left before reaching the limit".
> Returning and monitoring "calls remaining" is potentially an expensive
> operation, especially when multiple clients can consume the same quota,
> e.g. in the case of calls per user that is using multiple applications.
>

That's a valid observation. That's why clients receiving these headers
should consider them a hint and the next request they perform could be
responded with new data in the headers that would make little sense if
taken in isolation without further knowledge of what could be consuming the
quotas. Service specific policies specified in the headers could help
reduce that uncertainty, but those are too broad and diverse, so there's
been little discussion on the topic other than allowing people to specify
those so service specific clients can make use of those extra bits of
information.


> > The immediate benefit of these headers to service providers is being
> able to signal expectations
> > to clients so they can adapt.
>
> The question I have is with the existing non-standard headers, how many
> people have actually bothered to write this adaptive client code?  Do we
> believe that lack of standardization has been sufficient to discourage
> people from writing that code?  Are the only people writing that code the
> ones who are trying to avoid punitive behavior?  If so, maybe a best
> practices document that recommends not doing that would be more
> constructive.
>

Yes, I believe that because there is no standardization there's been little
to no effort in writing such adaptive code in generic HTTP clients, but
this is pure speculation. I think it is very likely that most people doing
so are the ones writing service specific clients, and in many cases they
could be the same people that wrote the service in the first place, unless
the service happens to see lots of usage across the board with many
independent clients. At a minimum, retry-after could be used to signal
generic clients that no request should be performed for a while, but that
falls short of the minimum information that generic clients could use with
these headers.


> >  Sometimes it is just a matter of adjusting to some specific rate of
> > resource consumption. Some other times the only way to adapt is just to
> stop using a service
> > entirely for a while or use another endpoint, and that's more useful
> than just returning a 403
> > or 50X without further information, ie. during a maintenance window you
> can use the headers
> > to signal that window with 0 quota units and avoid useless traffic.
>
> Why would a 503 with a retry-after header with a period that matches the
> maintenance window not be sufficient here?
>

You are right, the immediate effect is the same. That said, you would be
lacking some extra information about the nominal quota units and duration
of the current window and the windows applying after this one is elapsed
(ie. after recovery you'd have information about the expected rates, and
those could also signal some sort of punitive policy), though that
information might be next to useless in the current window since the
intended effect is to stop traffic altogether for a while. Yet it would not
work in a closely related scenario in which you would have some partial
maintenance work but keeping the ability to serve clients in a degraded
service manner -- that is, unless you kept giving clients shorter
no-service windows interleaved with service responses as usual, hence
generating a lot more wasteful traffic.

Cheers,
  Alex