Re: [Ecrit] planned-changes: two questions

[Randall Gellens <[e-mail]>]

On 8 Sep 2021, at 16:31, Brian Rosen wrote:

> Inline.  I think we need other opinions.  Certainly, we don’t have 
> rough consensus.
>
>
>> On Sep 8, 2021, at 5:45 PM, Randall Gellens 
>> <[e-mail]> wrote:
>>
>> Why is a subsequent transaction better than a parallel one? The 
>> subsequent transaction model delays knowledge of the state.
>>
>> If the client doesn't see an immediate POST, it doesn't know if 
>> there's been an error or not.
>>
> Yes it does.  If it doesn’t see an immediate POST, then there was a 
> problem and it has to try again.

Meanwhile, the server may be retrying.  There could have been a temp DNS 
issue or other temp failure

>> If the client sees a POST and sends an ID, it doesn't know if the 
>> server received it and stored the URI.
>>
> Technically true, but vanishingly small issue.  You had a complete 
> HTTPS transaction, but somehow the client isn’t sure the server got 
> the ID.
>> If the server tries a POST but gets an error, it may retry but 
>> meanwhile the client may retry the query, since it didn't get a POST.
>>
> That's always going to be something the server has to deal with: 
> re-enrollment of the URI.  The client gets a normal response, and that 
> tells it the URI was accepted.
>> Should a client retain IDs if it hasn't seen a POST?
>>
> No, lack of POST means URI is not enrolled

How long does the client retain the pending IDs waiting for a 
verification POST?

>> Also, requiring that the client send an ID seems like it may 
>> constrain client architectures since any part of the client system 
>> that receives a POST must know which IDs are pending.
>>
> I don’t agree.  This is a pretty normal “I send you a magic number 
> and you have to return it to me” thing.  There are no “pending 
> IDs”.  This is the first ID, so you need to return it, and only it, 
> but it’s the only ID the client has at this point.

So it's not a pending ID.  It's not used for notification, just 
verification.  That wasn't clear.

>> What is the objection to a parallel POST?
>>
> I was taught that you don’t embed an HTTP transaction in another 
> with the same partner because things go wrong with timeouts and logic. 
>  It’s not a parallel POST, it’s an HTTPS transaction one way that 
> encloses another HTTPS transaction the other way.  You are dependent 
> on the inner one completing before the outer one times out or has some 
> other issue.  I just think that’s dangerous.  I will admit that I 
> was taught this quite a long time ago (before HTTPS for sure) and 
> things could have changed, but I can’t recall an API that does that.

It seems simpler than completing a transaction that isn't actually 
complete.  It seems simpler that both sides know the state, without 
ambiguity.

> Also note that keep alive works exactly the same way as the initial 
> POST.  In your proposal, those are different.  That’s not much, but 
> it’s some code.  The “test” transaction tells you that the 
> mechanism works and your URI is being retained, whether that happens 
> the first time, or a year later.

My proposal used the same 'test' in a POST  to the URI for both initial 
verification and subsequent keep-alives.

>> --Randall
>>
>> On 8 Sep 2021, at 14:22, Brian Rosen wrote:
>>
>> I don’t think there is any fragility in the URI setup.  For a new 
>> URI, the client requests the server to keep it in a findService.  
>> That transaction concludes by sending an ID.  The server immediately 
>> sends the test notification, to which the client responds with the 
>> ID.  At that point both sides know the URI was accepted and the 
>> client is valid.  That may be repeated periodically so both sides 
>> know the other is happy with the arrangements.  If the client 
>> doesn’t get the test notification, it knows the URI isn’t 
>> accepted.  If the server doesn’t see the ID, it’s knows that was 
>> a bogus request, or there is some other problem, and it shouldn’t 
>> save the URI.
>>
>> I proposed a null ID as the test transaction flag.  That would be a 
>> piece of XML with the right namespace, the right element name, but no 
>> value.  I think that is adequate and meets Guy’s minimal mechanism 
>> criteria.  The same XML is always sent.  It either has a set of IDs 
>> or it’s empty.
>>
>> Brian
>>
>>> On Sep 8, 2021, at 5:11 PM, Randall Gellens 
>>> <[e-mail] <mailto:[e-mail]>> 
>>> wrote:
>>>
>>> The proposed mechanism seems fragile. A client has no way to know if 
>>> its request to store a URI was accepted. That makes it hard to 
>>> debug. If there are errors of any kind when the server initially 
>>> verifies a URI, the client has no idea. Even if the client repeats 
>>> the transaction, the server will silently discard the URI. To me, 
>>> that's asking for problems.
>>>
>>> In my proposal, when the client requests to be notified and provides 
>>> a URI, the server immediately does a parallel POST to that URI. If 
>>> it gets a 202 Accepted response to the POST (or we can choose a 
>>> different value), it stores the URI and returns the query result. If 
>>> it gets a different response, it does not store the URI and returns 
>>> the query result with a uriNotStored warning. If a client is trying 
>>> to get the server to launch an attack on a third party entity, that 
>>> entity will likely not support the URI in the first place, or will 
>>> likely return an error response. If you want additional 
>>> verification, have the server include the queried location in its 
>>> test POST. That way, if the client maliciously sent a URI of a third 
>>> party LIS, that LIS will know it does not have a query with that 
>>> location outstanding.
>>>
>>> Doing a parallel POST from the server to the client is one small 
>>> transaction; this mechanism provides immediate feedback to both 
>>> client and server. If there are DNS failures or network congestion 
>>> or whatever, the server returns uriNotStored and the client can try 
>>> again later.
>>>
>>> As for multiple IDs in a single notification POST, having the client 
>>> specify in the initial request the maximum it is prepared to accept 
>>> seems reasonable. We can require that clients support a minimum 
>>> number. If a server has more than the maximum, it sends them in 
>>> multiple POSTs, with whatever separation in time it chooses. A 
>>> server can include fewer IDs in a POST than the client's maximum.
>>>
>>> We can have a 'test' notification value that is used both for the 
>>> initial verification test and for periodic keep-alive tests.
>>>
>>> --Randall
>>>
>>> On 8 Sep 2021, at 12:17, Brian Rosen wrote:
>>>
>>> Okay, I think the three of us are converging,  Here is a restatement 
>>> of your description:
>>> 1) In a validation query, a Client can request to be notified when 
>>> the proffered LI should be revalidated, and provides a URI to send 
>>> the notifications to;
>>> 2) In the validation response, the Server provides an ID that the 
>>> Client associates with the LI it just validated.   The server may 
>>> silently ignore repeated requests to store a URI where the test in 4 
>>> below fails.
>>> 3) Immediately thereafter, if the URI is new to the Server, the 
>>> Server sends a ‘test’ notification to the URI, with an empty ID.
>>> 4) The recipient at the URI is expected to respond with the ID 
>>> provided in step 2. If it does, the Server stores the URI for future 
>>> notifications.  If it does not, the server ignores the request to 
>>> store the URI.
>>> 5) Some time after, the Server notifies the Client of an upcoming 
>>> planned change by sending a notification to the successfully tested 
>>> URI with the location ID;
>>> 6) The client revalidates each LI in its database that matches the 
>>> ID as of the date of the planned change. If no ID matches, it is a 
>>> no-op at the Client. Revalidations may also result in no-op at the 
>>> Client.
>>> 7) LIs at the Client that are invalidated by the planned change are 
>>> modified in its database to be valid (which probably mean another 
>>> revalidation cycle) with an effective date set to <revalidateAsoF> 
>>> value.
>>> 8) The Server may send ’test’ notifications to the URI without 
>>> any ID as a form of “keep-alive”.  Any ID provided by the Server 
>>> to the client may be used as the response to the test transaction
>>>
>>>
>>> There has been a discussion of sending more than one ID in a 
>>> transaction.  I think that is a decent idea, but I worry about how 
>>> big that could be.  Either we put a hard limit in the text or have 
>>> something in the response to the test transaction that specifies a 
>>> size limit for that client.
>>>
>>> Brian
>>>
>>>> On Sep 7, 2021, at 8:31 PM, Caron, Guy <[e-mail] 
>>>> <mailto:[e-mail]>> wrote:
>>>>
>>>> 1) In a validation query, a Client can request to be notified when 
>>>> the proffered LI should be revalidated, and provides a URI to send 
>>>> the notifications to;
>>>> 2) In the validation response, the Server provides an ID that the 
>>>> Client associates with the LI it just validated;
>>>> 3) Immediately thereafter, the Server sends a ‘test’ 
>>>> notification to the URI, without any ID;\
>>>> [br[For every new ID?  Or just once?  I wanted this to be a one 
>>>> time registration.
>>>> [GC] Just once per offered URI.
>>>>
>>>>
>>>> 4) The recipient at the URI is expected to respond with the ID 
>>>> provided in step 2. If it does, the Server stores the URI for 
>>>> future notifications. If it does not, the Server [let’s pick one: 
>>>> reject silently the URI and block the Client permanently/provides a 
>>>> ‘uriNotStored’ warning response to the URI {not compatible with 
>>>> the current proposal to test outside of LoST}/reject silently the 
>>>> URI and block the Client temporarily/other?];
>>>> [br]I don’t think an explicit failure is a problem.  The LoST 
>>>> server can limit retries if it needs to.
>>>> [GC] Ok for HTTP failures but what I’m talking about is when it 
>>>> fails to return the ID, like in your DoS example.
>>>>
>>>>
>>>>
>>>> 5) Some time after, the Server notifies the Client of an upcoming 
>>>> planned change by sending a notification to the successfully tested 
>>>> URI with the location ID;
>>>> 6) The client revalidates each LI in its database that matches the 
>>>> ID as of the date of the planned change. If no ID matches, it is a 
>>>> no-op at the Client. Revalidations may also result in no-op at the 
>>>> Client.
>>>> 7) LIs at the Client that are invalidated by the planned change are 
>>>> modified in its database to be valid (which probably mean another 
>>>> revalidation cycle) with an effective date set to <revalidateAsoF> 
>>>> value.
>>>> [br]I wanted periodic keep alives.  How would that work?
>>>> [GC] You mean at step 3? As I mentioned below, I was wondering 
>>>> about the necessity for periodic tests. If the group is convinced 
>>>> it is needed, the Server can simply redo step 3 and the Client can 
>>>> respond with one or many IDs it has from that Server.
>>>
>>