Re: [Id-event] Push Delivery: working group last call

[Phil Hunt <[e-mail]>]

Looking at the error codes, a couple of concerns:

   | authentication_failed | The SET Recipient could not authenticate  |
   |                       | the SET Transmitter from the contents of  |
   |                       | the request.                              |
   | access_denied         | The SET Transmitter is not authorized to  |
   |                       | transmit the provided SET to the SET      |
   |                       | Recipient.        

These are also HTTP Level errors. I would suggest putting in stronger language indicating that these errors are based on authentication and authorization semantics derived from processing the SET.

You should also indicate that a receive may also issue similar error messages at the HTTP level based on the authorization header.

For invalid request, again, this can be at the HTTP level and within SET
   | invalid_request       | The request body cannot be parsed as a    |
   |                       | SET, or the event payload within the SET  |
   |                       | does not conform to the event's           |
   |                       | definition.                               |

How does the receiver know whether something like an improper parameter is the issue or whether it is a SET event rejection issue?

I believe Section 4 does not strike the correct balance. Many scenarios that have been discussed seem asymmetric and there is a strong likelihood of a receiver being overwhelmed.

* Identity Provider dominant: Major IDP issuing events to smaller relying party web sites.
* Web site dominant: Major web sites overwhelming small IDPs (e.g. corporate based IDPs) with events.
  
Event streaming has been maturing. A more nuanced view from the Kafka documentation may be useful for the WG to consider (http://kafka.apache.org/documentation.html#design_pull ):
> […]There are pros and cons to both [push and pull] approaches. However, a push-based system has difficulty dealing with diverse consumers as the broker controls the rate at which data is transferred. The goal is generally for the consumer to be able to consume at the maximum possible rate; unfortunately, in a push system this means the consumer tends to be overwhelmed when its rate of consumption falls below the rate of production (a denial of service attack, in essence). A pull-based system has the nicer property that the consumer simply falls behind and catches up when it can. This can be mitigated with some kind of backoff protocol by which the consumer can indicate it is overwhelmed, but getting the rate of transfer to fully utilize (but never over-utilize) the consumer is trickier than it seems. Previous attempts at building systems in this fashion led us to go with a more traditional pull model.
> Another advantage of a pull-based system is that it lends itself to aggressive batching of data sent to the consumer. A push-based system must choose to either send a request immediately or accumulate more data and then send it later without knowledge of whether the downstream consumer will be able to immediately process it. If tuned for low latency, this will result in sending a single message at a time only for the transfer to end up being buffered anyway, which is wasteful. A pull-based design fixes this as the consumer always pulls all available messages after its current position in the log (or up to some configurable max size). So one gets optimal batching without introducing unnecessary latency.
> The deficiency of a naive pull-based system is that if the broker has no data the consumer may end up polling in a tight loop, effectively busy-waiting for data to arrive. To avoid this we have parameters in our pull request that allow the consumer request to block in a "long poll" waiting until data arrives (and optionally waiting until a given number of bytes is available to ensure large transfer sizes).
> 

My concerns about pursuing HTTP-Push without pull remain unresolved.

My long-term preference is a single protocol with the following capabilities:
* ability to deliver multiple events in a single message body
* clear ability to do re-delivery at the transmitters discretion (e.g. overwhelmed or unavailable partner). The current spec seems to waffle here.
* ability to support some amount of recovery where network/security relationships warrant the cost
* a bi-directional delivery that can be deployed as POST or GET initiated. This in part because it reduces the number of security credentials and complications needed to maintain multiple streams in opposite directions over many different security and network relationships.
* ability to support a diverse number of client relationships (asymmetric and symmetric in subject population or event frequency)
* strong confirmation of message transfer via acknowledgment (e.g. to enable publisher to drop events from its recovery buffers)

Phil

> On Feb 4, 2019, at 10:42 AM, Marius Scurtescu <[e-mail]> wrote:
> 
> I support the publication of this document, with the changes suggested by Yaron Shefer and Annabelle Richard Backman.
> 
> On Fri, Jan 25, 2019 at 1:04 PM Yaron Sheffer <[e-mail] <mailto:[e-mail]>> wrote:
> Thanks for the quick response! A few comments inline.
> 
> On 25/01/2019 22:18, Richard Backman, Annabelle wrote:
> > Thanks for the feedback! Responses inline.
> > 
> > -- 
> > 
> > Annabelle Richard Backman
> > 
> > AWS Identity
> > 
> > On 1/25/19, 9:59 AM, "Id-event on behalf of Yaron Sheffer" 
> > <[e-mail] <mailto:[e-mail]> on behalf of [e-mail] <mailto:[e-mail]>> wrote:
> > 
> >      The document looks good. Here's a bunch o' comments.
> > 
> >      * Intro: receipt->recipient
> > 
> > [richanna]👍[/richanna]
> > 
> >      * "The transmitter and recipient are known to one another." But 
> > then we
> > 
> >      require the *issuer* to be authenticated by the Recipient. We need to
> > 
> >      clarify the relationship between Issuer (defined in the base spec) and
> > 
> >      Transmitter. BTW, Issuer should be capitalized.
> > 
> > [richanna]
> > 
> > The intent of that sentence is to indicate that the spec assumes the 
> > Transmitter already knows who it needs to transmit the SET to, and the 
> > Recipient knows who it expects to receive SETs from, and that discovery 
> > of either is out of scope.
> > 
> > You’re absolutely right. Section 2 is missing a validation step: “The 
> > SET Recipient is willing to accept the SET, when transmitted by the SET 
> > Transmitter.” Also, we can add text to the definition of SET Transmitter 
> > to clarify that it need not be the Issuer of the SET.
> > 
> > [/richanna]
> > 
> >      * "a decryption failure that may be due to misconfigured keys" - 
> > this is
> > 
> >      a strange reason for retransmission, because it would normally 
> > require a
> > 
> >      very slow manual operation, which could easily overwhelm the 
> > Transmitter.
> > 
> > [richanna]
> > 
> > I had in mind issues or delays with key propagation after a rotation, 
> > premature rejection of an outgoing key due to clock skew, intermittent 
> > failure when retrieving a Transmitter’s public key, etc. These are 
> > issues that may (depending on root cause) work themselves out after a 
> > few minutes, though I suppose they are not properly “misconfigured 
> > keys.” This example is probably just muddying the waters without 
> > contributing much to the text.
> > 
> > [/richanna]
> > 
> >      * "Transmitting a SET": if the normal response has an empty body, 
> > why do
> > 
> >      we send "Accept: application/json"? The Accept header is not 
> > mandatory,
> > 
> >      https://tools.ietf.org/html/rfc7231#section-5.3.2 <https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_rfc7231-23section-2D5.3.2&d=DwMFaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=na5FVzBTWmanqWNy4DpctyXPpuYqPkAI1aLcLN4KZNA&m=BvF8V4tpvmen0k4B-cqiDJM_jl4O5JF6_al7mA3UZJA&s=f7gsNGdhVUlodq21NuzNhQDQTnJB9CZOG6_CCWRVt5w&e=>
> > 
> > [richanna]
> > 
> > Error responses will contain a JSON-formatted body, which the 
> > Transmitter MUST be prepared to accept.
> > 
> > [/richanna]
> 
> YS: I still think the Accept header is redundant (in particular because 
> if you cannot understand JSON, you cannot use this protocol at all), but 
> I can live with it.
> 
> > 
> >      * "Failure Response": why is responding with the right HTTP status 
> > code
> > 
> >      a MAY? What else do we expect them to do?
> > 
> > [richanna]
> > 
> > As I understand it, HTTP Servers have some discretion with regards to 
> > which status codes they respond with in certain situations. The “right 
> > HTTP status code” can be subjective, and may change over time as new 
> > status codes are defined for specific cases. SET Transmitters need to be 
> > prepared to receive standard HTTP status codes, but I don’t see a reason 
> > to impose more requirements on the SET Recipient in this area than HTTP 
> > itself does.
> > 
> > [/richanna]
> > 
> 
> YS: I agree with everything you say, but still, this would be more 
> correct: "the SET Recipient SHOULD respond with an appropriate HTTP 
> Status Code", even if the "appropriate" code is very fuzzy. Or else, 
> let's remove the RFC 2119 word: "the Recipient responds".
> 
> >      * "the SET Transmitter is not permitted to transmit SETs issued by the
> > 
> >      issuer specified in the SET" - I suggest to change the text to 
> > indicate
> > 
> >      this is from the point of view of the Recipient: "the Recipient is not
> > 
> >      willing to accept SETs issued by the specified issuer from this
> > 
> >      particular SET Transmitter".
> > 
> > [richanna]👍[/richanna]
> > 
> >      * Error codes: what if the Recipient expects the SET to be protected
> > 
> >      (e.g. signed) but it is not? Is this "authentication_failed"?
> > 
> > [richanna]
> > 
> > SET signatures authenticate the Issuer, not the Transmitter, so 
> > `authentication_failed` would be inappropriate. The Recipient should 
> > return `access_denied` as it is unwilling to accept this SET from this 
> > Transmitter. Now I’m actually wondering if there is a difference between 
> > `invalid_request` and `access_denied` that is meaningful at runtime. 
> > I’ll start a thread on this.
> > 
> > [/richanna]
> > 
> >      * "MUST support TLS 1.2" - we should allow for a future where TLS
> > 
> >      1.3-only is acceptable. So, "MUST support TLS 1.2 or later".
> > 
> > [richanna]👍[/richanna]
> > 
> >      * IANA: the Expert Review policy requires a designated expert,
> > 
> >      obviously. Since this is the only SecEvent registry that needs it so
> > 
> >      far, I'd rather go for the simpler FCFS policy.
> > 
> > [richanna]
> > 
> > 👍, though I think we should add text to the effect of “Error Codes are 
> > intended to be interpreted by automated systems, and therefore SHOULD 
> > identify classes of errors in response to which an automated system 
> > could take some distinct action.”
> > 
> > [/richanna]
> 
> YS: guidance is always good.
> 
> > 
> >      Thanks,
> > 
> >                  Yaron
> > 
> >      On 24/01/2019 20:11, Dick Hardt wrote:
> > 
> >      > Dear SecEvent participants,
> > 
> >      >
> > 
> >      > This is to start a 2-week working group last call
> > 
> >      > on draft-ietf-secevent-http-push-04
> > 
> >      >
> > 
> >      > https://datatracker.ietf.org/doc/draft-ietf-secevent-http-push/ <https://urldefense.proofpoint.com/v2/url?u=https-3A__datatracker.ietf.org_doc_draft-2Dietf-2Dsecevent-2Dhttp-2Dpush_&d=DwMFaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=na5FVzBTWmanqWNy4DpctyXPpuYqPkAI1aLcLN4KZNA&m=BvF8V4tpvmen0k4B-cqiDJM_jl4O5JF6_al7mA3UZJA&s=PkEZygG0Fgx50rP5SgjLIgZQc5smVzozBjz9iBge4Qw&e=>
> > 
> >      >
> > 
> >      > until Feb 8, 2019.
> > 
> >      >
> > 
> >      > Please send any comments to the list. In addition, if you have no
> > 
> >      > comments but support publication of this document as-is, please
> > 
> >      > make your voice heard.
> > 
> >      >
> > 
> >      > Thanks,
> > 
> >      >
> > 
> >      >       Dick and Yaron
> > 
> >      _______________________________________________
> > 
> >      Id-event mailing list
> > 
> >      [e-mail] <mailto:[e-mail]>
> > 
> >      https://www.ietf.org/mailman/listinfo/id-event <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_id-2Devent&d=DwMFaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=na5FVzBTWmanqWNy4DpctyXPpuYqPkAI1aLcLN4KZNA&m=BvF8V4tpvmen0k4B-cqiDJM_jl4O5JF6_al7mA3UZJA&s=QGLx9ymWKPAhM3Co6sSgtYxNF4af-cmPZSSFZFDQ--c&e=>
> > 
> 
> _______________________________________________
> Id-event mailing list
> [e-mail] <mailto:[e-mail]>
> https://www.ietf.org/mailman/listinfo/id-event <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_id-2Devent&d=DwMFaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=na5FVzBTWmanqWNy4DpctyXPpuYqPkAI1aLcLN4KZNA&m=BvF8V4tpvmen0k4B-cqiDJM_jl4O5JF6_al7mA3UZJA&s=QGLx9ymWKPAhM3Co6sSgtYxNF4af-cmPZSSFZFDQ--c&e=>