Re: [Ecrit] planned-changes: two questions

[Randall Gellens <[e-mail]>]

I think a simpler approach is that when a query includes a URI, before 
returning the response, the server POSTS to the URI.  If it receives a 
success response, it stores the URI and returns the response.  If it 
receives a non-success response, it informs the client that the URI was 
not stored.  We can use uriNotStored or have a new warning such as 
uriTestFailure.  This way, the server does not need to maintain a 
pending URI and retry the test, and the client knows immediately if 
there's a problem.  Presumably, if the URI is bogus, the URI host will 
not return a success result.  I think that should be good enough, but if 
we want stronger protection, we could tie in the TLS authentication when 
the server does the POST.

--Randall

On 2 Sep 2021, at 14:29, Brian Rosen wrote:

> > On Sep 2, 2021, at 3:48 PM, Caron, Guy <[e-mail]> wrote:
>>
>> Inline under [GC].
>>
>> If you agree with my comments inline below, here is how I see the 
>> process (building on what Randall initially provided on this thread):
>>
>> 1) In a validation query, a Client can request to be notified when 
>> the proffered LI should be revalidated, and provides a URI to send 
>> the notifications to;
>> 2) In the validation response, the Server provides an ID that the 
>> Client associates with the LI it just validated;
>> 3) Immediately thereafter, the Server sends a ‘test’ notification 
>> to the URI, without any ID;\
> [br[For every new ID?  Or just once?  I wanted this to be a one time 
> registration.
>> 4) The recipient at the URI is expected to respond with the ID 
>> provided in step 2. If it does, the Server stores the URI for future 
>> notifications. If it does not, the Server [let’s pick one: reject 
>> silently the URI and block the Client permanently/provides a 
>> ‘uriNotStored’ warning response to the URI {not compatible with 
>> the current proposal to test outside of LoST}/reject silently the URI 
>> and block the Client temporarily/other?];
> [br]I don’t think an explicit failure is a problem.  The LoST server 
> can limit retries if it needs to.
>
>
>> 5) Some time after, the Server notifies the Client of an upcoming 
>> planned change by sending a notification to the successfully tested 
>> URI with the location ID;
>> 6) The client revalidates each LI in its database that matches the ID 
>> as of the date of the planned change. If no ID matches, it is a no-op 
>> at the Client. Revalidations may also result in no-op at the Client.
>> 7) LIs at the Client that are invalidated by the planned change are 
>> modified in its database to be valid (which probably mean another 
>> revalidation cycle) with an effective date set to <revalidateAsoF> 
>> value.
> [br]I wanted periodic keep alives.  How would that work?
>>
>> Guy
>>
>> De : Brian Rosen <[e-mail] <mailto:[e-mail]>>
>> Envoyé : 1 septembre 2021 19:17
>> À : Caron, Guy <[e-mail] <mailto:[e-mail]>>
>> Cc : Randall Gellens <[e-mail] 
>> <mailto:[e-mail]>>; ECRIT <[e-mail] 
>> <mailto:[e-mail]>>
>> Objet : [EXT]Re: [Ecrit] planned-changes: two questions
>>
>> Inline
>>
>>
>> On Sep 1, 2021, at 3:28 PM, Caron, Guy <[e-mail] 
>> <mailto:[e-mail]>> wrote:
>>
>> Ok. You have convinced me of the necessity of validating the URI 
>> before the Server stores it and that the security considerations in 
>> RFC5222 do not cover this threat.
>>
>> I observe that this vulnerability was already mentioned in the 
>> Security section of the planned-changes draft:
>>
>> The server is subject to abuse by clients because it is being asked 
>> to store something and may need to send data to an uncontrolled URI.
>>
>> However, the original threat vector and associated response is 
>> probably insufficient to cover the DoS you raised, hence this new 
>> proposal to test the URI.
>>
>> So, I’m conceptually favorable to adding a mechanism to test the 
>> URI before being stored by the Server.
>> Good
>>
>>
>>
>> I would like this mechanism to be as simple as possible and I think 
>> the goals can be achieved by testing the URI and receiving a response 
>> with at least one of the IDs previously supplied by the Server. I 
>> would leave it to policy at the Server to determine when and how 
>> often the URIs would be tested, with a recommendation (MUST? SHOULD?) 
>> to test it upon reception before storing it.
>> I think you are suggesting we add text that says the LoST server can 
>> send a notification to the LoST client that has no changes, and just 
>> serves as a keep alive.  I’m sure we can make that work but 
>> generally, I find this kind of implicit test to be less satisfactory 
>> than an explicit keep alive transaction.   I get that an actual 
>> change may not be seen as a change to the client, and thus it works 
>> with no real code, but it doesn’t seem as clean as an explicit keep 
>> alive.  I’d be interested in herring from others on that
>> [GC] Could we not, for example, declare a value/element/attribute 
>> 'test' to be passed in the notification with no IDs? Would that meet 
>> your idea of an explicit test mechanism?
> [br] I suppose.  Doesn’t seem much different than a parameter to the 
> findService, but I’m okay with it
>>
>>
>>
>> We probably need to say something about what to do if the URI test 
>> fails. Do we inform the Client that provided the URI that it may have 
>> been compromised? Do we simply drop the URI silently? Do we block the 
>> Client that provided the URI entirely?
>> At minimum wait and try again, the client could be down for a while.  
>> I would not allow a lot of location records to be marked with the URI 
>> that didn’t test valid.
>>
>> I am somewhat leery of creating circumstances where the client has to 
>> go through and revalidate it’s entire record set because the 
>> mechanism failed in some way.  There clearly would be circumstances 
>> where that happened, but missing a keep alive shouldn’t be one of 
>> them.  The initial one however should be a hard block.  If you 
>> can’t get the first ID returned, then we have to not accept other 
>> registrations.
>> [GC] On a second thought, what's the value of testing the URI after 
>> the first test? If it passed, it's valid and the Server stores it for 
>> use when a planned change occurs. If after that and for some reason, 
>> the Client is not responsive to the notification, does the Server 
>> care?
> Generally when we have things that don’t do anything for months and 
> then we need it, I would like to see periodic keep alives.  Things 
> change on both sides and both sides have a vested interest in making 
> sure the mechanism works when they need it.
>>
>>
>>
>> I’m not sure if the <command> thing is necessary since it requires 
>> the Server to keep state. I would think that if the URI has been 
>> tested and stored, it means it is active and valid until it is 
>> deleted through the reception of an empty <plannedChange> for 
>> example, or replaced through the reception of a new URI in 
>> <plannedChange>.
>> I don’t think there is any server state other than the URI is 
>> valid.  Each transaction stands alone.
>>
>> Without the delete, you can’t remove one location record from the 
>> set that gets notified of planned changes.  You can change the URI, 
>> but that changes it for all notifications.
>> [GC] Why? A Client that no longer wants to be notified for a given ID 
>> sends an empty <plannedChange> for the LI it no longer has. If that 
>> ID at the Server is associated with only one location (say one 
>> specific civic number), it deletes the Client's URI. If it is 
>> associated with more than one record (say a range of civic numbers), 
>> it accepts the transaction but keeps the URI. A notification for that 
>> ID sent to the Client that only had the civic number would result in 
>> a no-op.
> That means that the client has to be identified by something like the 
> TLS cert, because the URI isn’t there for the delete.  I was using 
> the domain of the URI as the ID of the client.
>
> But we have another problem: I think the server does not track the LI. 
>  On any findService with the new option, it looks for a matching 
> record and returns the ID it has for that record.  It’s the client 
> that tracks possible multiple LIs for one ID, not the server.  So in 
> your example, the client would notice that any address in the range 
> had the same ID.  It would get one notification if the range record 
> changed, and it would update all its records in the range.  But if it 
> deleted one of a set of records that have the same ID, it wouldn’t 
> request delete.  Only if it got down to one record with a given ID, 
> and it needed to delete that record, would it request delete for that 
> ID at the server.
>>
>>
>>
>> Lastly, I’m not sure we still need the warning ‘uriNotStored’ 
>> given that we’re going with one generic URI per Client. The only 
>> argument I can think of for keeping it would be the proliferation of 
>> Clients within the coverage aria of the Server that would be so large 
>> that the Server couldn’t store them all. A counter argument to that 
>> would be that doing so could be seen as giving privileges to certain 
>> Clients over others.
>>
>> We’re creating circumstances where the client won’t get a 
>> notification.  I think we need to tell them that.
>> [GC] Only if the URI fails the validity test.
> [br]I wanted to allow the server to limit the number of IDs it was 
> tracking for a given client.  I wanted to limit the number of queries 
> the server is responding to (although it might well silently discard a 
> DOS attack.
>>
>>
>> One limit I might put on a client is how many locations they can ask 
>> to be notified.  If they try to set up a situation where no matter 
>> what changed, they get a notification, I think that may be too much 
>> to ask, and the server can have a limit.  It would be okay if it’s 
>> one URI per valid record at the LIS, but the LoST server doesn’t 
>> have that data.
>> [GC] It's one generic URI per Client with one or more IDs stored by 
>> the Client against its LIs, right? As you suggesting we go back to 
>> one URI per Client record in its database?
> [br]  One generic URI per client.  So if the client asks for its URI 
> to be stored against too many IDs, it can be told no.
>>
>>
>>
>> Thanks,
>>
>> Guy
>>
>>
>> De : Brian Rosen <[e-mail] <mailto:[e-mail]>>
>> Envoyé : 31 août 2021 13:54
>> À : Randall Gellens <[e-mail] 
>> <mailto:[e-mail]>>
>> Cc : Caron, Guy <[e-mail] <mailto:[e-mail]>>; ECRIT 
>> <[e-mail] <mailto:[e-mail]>>
>> Objet : [EXT]Re: [Ecrit] planned-changes: two questions
>>
>> Of course an attacker would conceal itself and register multiple URIs 
>> under different identities.  But the bigger  problem is that if a 
>> large planned change occurred, the victim could receive a large 
>> number of notifications it didn’t expect.
>>
>> With a test transition, we know that the client expects to see 
>> notifications, we only need one per URI (we could specify one per 
>> domain — after the scheme and before the first slash).  If we 
>> don’t get the ID, we know not to allow that URI to be provisioned.  
>> So it’s one test transaction vs a large number of planned change 
>> transactions.
>>
>> Brian
>>
>>
>>
>>
>> On Aug 31, 2021, at 1:35 PM, Randall Gellens 
>> <[e-mail] <mailto:[e-mail]>> wrote:
>>
>> If a malicious client registers a URI that is designed to attack a 
>> third site, the test transaction causes the LoST server to connect to 
>> it and POST a command. Without a test transaction, the LoST server 
>> stores the URI and at a future time connects to it and sends a 
>> command. Either way, a LoST client can register potentially one URI 
>> per queried location, and a LoST server will connect to that URI and 
>> POST a message. A queried location could be tied to many other 
>> locations, so there could be many locations for which a change would 
>> trigger the LoST server to use the URI, but is that worse?
>> --Randall
>> On 31 Aug 2021, at 10:27, Brian Rosen wrote:
>> If the work group wants to have the LoST server keep the URI until 
>> expressly deleted, that’s okay with me.
>>
>> Authenticating the client to the server doesn’t mean the URI is 
>> authenticated.  We can’t restrict the URI to be the same entity as 
>> the client running the LoST transaction.  And the clients are wide 
>> ranging.  Could be an enterprise running its own LIS for example.  We 
>> can’t assume the North American PKI is workable everywhere, and 
>> even that doesn’t extend to an enterprise LIS.  ISTM we have to run 
>> a test transaction with the notification service to make sure it’s 
>> what we think it is.
>>
>> Brian
>>
>>
>>
>>
>>
>> On Aug 31, 2021, at 9:01 AM, Caron, Guy <[e-mail] 
>> <mailto:[e-mail]>> wrote:
>>
>> Inline.
>>
>> Guy
>>
>> -----Message d'origine-----
>> De : Brian Rosen <[e-mail] <mailto:[e-mail]>>
>> Envoyé : 31 août 2021 08:11
>> À : Caron, Guy <[e-mail] <mailto:[e-mail]>>
>> Cc : Randall Gellens <[e-mail] 
>> <mailto:[e-mail]>>; ECRIT <[e-mail] 
>> <mailto:[e-mail]>>
>> Objet : [EXT]Re: [Ecrit] planned-changes: two questions
>>
>> You delete the URI when you delete the record in the LIS.
>> [GC] That's fine but that was not the question Randall posed. He 
>> asked whether the URI is deleted after a notification. I agree that 
>> if the Client does not host the location anymore that the URI 
>> associated with that location in the Server should be deleted. This 
>> could be achieved with an empty <plannedChange>.
>>
>> I don’t think this is covered in 5222.  The mechanism causes the 
>> LoST server to send notifications to the client, but the client is 
>> allowed to put any URI in the record, and it can add it to as many 
>> records as it wants.
>> [GC] I thought we agreed on using one generic URI per Client. Clients 
>> should be authenticated by the Servers.
>>  An evil implementation could record URIs against multiple targets 
>> that were unaware that the evil implementation did it, until they got 
>> a large number of PUSH transactions they didn’t expect or 
>> understand as a result of a large planned change.
>> [GC] Only authenticated Clients should be allowed to provide URIs to 
>> be stored by the Servers.
>>
>> The proposed mechanism qualifies the client URI before its used in a 
>> planned change.
>>
>> Brian
>>
>>
>>
>> On Aug 31, 2021, at 7:37 AM, Caron, Guy <[e-mail] 
>> <mailto:[e-mail]>> wrote:
>>
>> Well, this is not going in the direction I thought.
>>
>> What is the purpose of deleting the URIs at the Server 
>> post-validation?
>>
>> Regarding opening a new DoS, I guess I'm not following. Wouldn't this 
>> case be covered by the security considerations in RFC 5222?
>>
>> What you're proposing puts back significant load on the Servers (a 
>> key consideration for creating planned-changes in the first place) 
>> and complicates the mechanism.
>>
>> Guy
>>
>>
>> -----Message d'origine-----
>> De : Ecrit <[e-mail] <mailto:[e-mail]>> 
>> De la part de Brian Rosen Envoyé :
>> 30 août 2021 11:22 À : Randall Gellens <[e-mail] 
>> <mailto:[e-mail]>> Cc
>> : ECRIT <[e-mail] <mailto:[e-mail]>> Objet : [EXT]Re: 
>> [Ecrit] planned-changes: two
>> questions
>>
>> Answer 1: yes.  Since there is going to be a revalidation, just 
>> deleting the setting seems right to me.
>> Answer 2: Up to server.  If I were implementing, I would hash the 
>> real ID with the URI and some kind of predictable nonce.
>>
>> We probably have to say more about how the server identifies the 
>> client, so that replacement of the URI works.  Could we say we use 
>> the domain of the URI (the entire domain with all the dots) to 
>> identify the client, and anything can occur after it (meaning a slash 
>> and whatever)?  If we do that, then how would delete the 
>> notification?  Force there to be something other than the domain 
>> (ugly).  Explicit delete request?
>>
>> Hmmm, we’ve opened a DoS attack: a rogue client stores a bunch of 
>> URIs for servers it wants to victimize.  In North America we have a 
>> real simple solution for that, because we have a PKI, so we know, for 
>> sure, who the client is, and could restrict who we allow to store 
>> URIs, but that wouldn’t be true in general.  Also, it would be nice 
>> for the client to have confidence the mechanism worked before it 
>> needed it.
>>
>> So
>> Let’s add a “command” to plannedChange in the findService 
>> request.
>> And, have the client have a response to the notification which is the
>> ID (json with the 200)
>>
>>
>> The client starts by sending a command of “initialize”.  The 
>> domain is the identity of the client.  The response is an immediate 
>> notification to the with whatever LI was in the request and an ID.  
>> The  response by the client (which is the notification web server) is 
>> a piece of json containing the ID.  We can say that the LI in this 
>> initialize command could be something simple like the Country Code 
>> that wouldn’t get a planned change.
>>
>> Thereafter, the LoST server (notification client) periodically 
>> repeats this keepalive notification every day or week with the 
>> initialize LI.  The client has to respond with the ID.
>>
>> The regular notification request is a command of “notify”.  The 
>> server ignores a request for notification from an uninitialized 
>> client.
>> The notification can be deleted with a command of “delete”.  If 
>> you delete the initialize LI, then the server won’t send any more 
>> notifications to that client and deletes all URIs it was saving for 
>> that client.  The client would have to re-initialize to reset.
>>
>> Brian
>>
>>
>>
>> On Aug 27, 2021, at 5:41 PM, Randall Gellens 
>> <[e-mail] <mailto:[e-mail]>> wrote:
>>
>> I think we're moving to a model where:
>> - In a query, a client can request to be notified when the location
>> should be revalidated;
>> - In the response, the server provides an ID which the client
>> associates with the location it just validated;
>> - The server sends a notification to the URI, containing the ID;
>> - The client revalidates each location with which that ID is 
>> associated.
>>
>> Question 1: Does the server delete/inactivate the URI once it has 
>> sent the notification?
>>
>> Question 2: Presumably, when the client revalidates the location(s), 
>> it will again request notification.  Does the server return the same 
>> ID as before, or a different ID?  A different ID could perhaps be 
>> useful in edge cases where the server didn't send or the client 
>> didn't get the notification, but any utility seems small.  If it's 
>> the same ID, then the answer to question 1 can be that the URI 
>> remains active until the client asks to no longer be notified (by 
>> sending an empty URI?).
>>
>> --Randall
>>
>> _______________________________________________
>> Ecrit mailing list
>> [e-mail] <mailto:[e-mail]>
>> https://www.ietf.org/mailman/listinfo/ecrit 
>> <https://www.ietf.org/mailman/listinfo/ecrit>
>>
>> _______________________________________________
>> Ecrit mailing list
>> [e-mail] <mailto:[e-mail]>
>> https://www.ietf.org/mailman/listinfo/ecrit 
>> <https://www.ietf.org/mailman/listinfo/ecrit>
>> ----------------------------------------------------------------------
>> -------- External Email: Please use caution when opening links and
>> attachments / Courriel externe: Soyez prudent avec les liens et
>> documents joints
>>
>> ------------------------------------------------------------------------------
>> External Email: Please use caution when opening links and attachments 
>> / Courriel externe: Soyez prudent avec les liens et documents joints
>>
>>
>> External Email: Please use caution when opening links and attachments 
>> / Courriel externe: Soyez prudent avec les liens et documents joints
>>
>> External Email: Please use caution when opening links and attachments 
>> / Courriel externe: Soyez prudent avec les liens et documents joints