Re: [Ecrit] planned-changes: two questions

[Randall Gellens <[e-mail]>]

I see the concern, but I think the alternative is uglier and more 
fragile.  It also seems more complex.  If there's a timeout during the 
query, the client tries again, probably after waiting a bit, which seems 
cleaner than noticing that it didn't get the confirmation POST.  Absent 
congestion or overload, a single TCP/TLS setup and POST round-trip 
should be very fast.

--Randall

On 3 Sep 2021, at 9:40, Brian Rosen wrote:

> Dunno, seems pretty ugly to me.  Suppose the notify transaction ran 
> close to some kind of timeout for some reason.  You might push the 
> findService transaction over its timeout.  I’m probably reaching a 
> bit there.
>
> Brian
>
>
>> On Sep 2, 2021, at 6:04 PM, Randall Gellens 
>> <[e-mail]> wrote:
>>
>> We need to do it once per new URI. If the URI already is stored, 
>> there's no reason to validate it again.
>>
>> The overhead of a new TCP/TLS connection and one round-trip isn't 
>> much, and is only once per new URI.
>>
>> Also, for URI deletion, we should have an explicit way to do this, 
>> perhaps a "delete" parameter along with the URI to be deleted.
>>
>> --Randall
>>
>> On 2 Sep 2021, at 14:51, Brian Rosen wrote:
>>
>> You are proposing that you set up a TLS connection, POST on that 
>> connection, the server sets up a reverse connection, POSTs on that, 
>> the client responds to that POST, and then the server responds to 
>> that post?  Not a great plan I think.  If second TLS connection 
>> already exists, so you are asking to do a complete transaction B->A 
>> before the response to A->B I might be less worried.  Doesn’t sound 
>> like a great plan
>>
>>
>> Also, we need only do this once per client.  Not once per validation.
>>
>> Brian
>>
>>> On Sep 2, 2021, at 5:45 PM, Randall Gellens 
>>> <[e-mail] <mailto:[e-mail]>> 
>>> wrote:
>>>
>>> I think a simpler approach is that when a query includes a URI, 
>>> before returning the response, the server POSTS to the URI. If it 
>>> receives a success response, it stores the URI and returns the 
>>> response. If it receives a non-success response, it informs the 
>>> client that the URI was not stored. We can use uriNotStored or have 
>>> a new warning such as uriTestFailure. This way, the server does not 
>>> need to maintain a pending URI and retry the test, and the client 
>>> knows immediately if there's a problem. Presumably, if the URI is 
>>> bogus, the URI host will not return a success result. I think that 
>>> should be good enough, but if we want stronger protection, we could 
>>> tie in the TLS authentication when the server does the POST.
>>>
>>> --Randall
>>>
>>> On 2 Sep 2021, at 14:29, Brian Rosen wrote:
>>>
>>>
>>>
>>>> On Sep 2, 2021, at 3:48 PM, Caron, Guy <[e-mail] 
>>>> <mailto:[e-mail]>> wrote:
>>>>
>>>> Inline under [GC].
>>>>
>>>> If you agree with my comments inline below, here is how I see the 
>>>> process (building on what Randall initially provided on this 
>>>> thread):
>>>>
>>>> 1) In a validation query, a Client can request to be notified when 
>>>> the proffered LI should be revalidated, and provides a URI to send 
>>>> the notifications to;
>>>> 2) In the validation response, the Server provides an ID that the 
>>>> Client associates with the LI it just validated;
>>>> 3) Immediately thereafter, the Server sends a ‘test’ 
>>>> notification to the URI, without any ID;\
>>> [br[For every new ID?  Or just once?  I wanted this to be a one time 
>>> registration.
>>>> 4) The recipient at the URI is expected to respond with the ID 
>>>> provided in step 2. If it does, the Server stores the URI for 
>>>> future notifications. If it does not, the Server [let’s pick one: 
>>>> reject silently the URI and block the Client permanently/provides a 
>>>> ‘uriNotStored’ warning response to the URI {not compatible with 
>>>> the current proposal to test outside of LoST}/reject silently the 
>>>> URI and block the Client temporarily/other?];
>>> [br]I don’t think an explicit failure is a problem.  The LoST 
>>> server can limit retries if it needs to.
>>>
>>>
>>>> 5) Some time after, the Server notifies the Client of an upcoming 
>>>> planned change by sending a notification to the successfully tested 
>>>> URI with the location ID;
>>>> 6) The client revalidates each LI in its database that matches the 
>>>> ID as of the date of the planned change. If no ID matches, it is a 
>>>> no-op at the Client. Revalidations may also result in no-op at the 
>>>> Client.
>>>> 7) LIs at the Client that are invalidated by the planned change are 
>>>> modified in its database to be valid (which probably mean another 
>>>> revalidation cycle) with an effective date set to <revalidateAsoF> 
>>>> value.
>>> [br]I wanted periodic keep alives.  How would that work?
>>>>
>>>> Guy
>>>>
>>>> De : Brian Rosen <[e-mail] <mailto:[e-mail]>>
>>>> Envoyé : 1 septembre 2021 19:17
>>>> À : Caron, Guy <[e-mail] <mailto:[e-mail]>>
>>>> Cc : Randall Gellens <[e-mail] 
>>>> <mailto:[e-mail]>>; ECRIT <[e-mail] 
>>>> <mailto:[e-mail]>>
>>>> Objet : [EXT]Re: [Ecrit] planned-changes: two questions
>>>>
>>>> Inline
>>>>
>>>>
>>>> On Sep 1, 2021, at 3:28 PM, Caron, Guy <[e-mail] 
>>>> <mailto:[e-mail]>> wrote:
>>>>
>>>> Ok. You have convinced me of the necessity of validating the URI 
>>>> before the Server stores it and that the security considerations in 
>>>> RFC5222 do not cover this threat.
>>>>
>>>> I observe that this vulnerability was already mentioned in the 
>>>> Security section of the planned-changes draft:
>>>>
>>>> The server is subject to abuse by clients because it is being asked 
>>>> to store something and may need to send data to an uncontrolled 
>>>> URI.
>>>>
>>>> However, the original threat vector and associated response is 
>>>> probably insufficient to cover the DoS you raised, hence this new 
>>>> proposal to test the URI.
>>>>
>>>> So, I’m conceptually favorable to adding a mechanism to test the 
>>>> URI before being stored by the Server.
>>>> Good
>>>>
>>>>
>>>>
>>>> I would like this mechanism to be as simple as possible and I think 
>>>> the goals can be achieved by testing the URI and receiving a 
>>>> response with at least one of the IDs previously supplied by the 
>>>> Server. I would leave it to policy at the Server to determine when 
>>>> and how often the URIs would be tested, with a recommendation 
>>>> (MUST? SHOULD?) to test it upon reception before storing it.
>>>> I think you are suggesting we add text that says the LoST server 
>>>> can send a notification to the LoST client that has no changes, and 
>>>> just serves as a keep alive.  I’m sure we can make that work but 
>>>> generally, I find this kind of implicit test to be less 
>>>> satisfactory than an explicit keep alive transaction.   I get that 
>>>> an actual change may not be seen as a change to the client, and 
>>>> thus it works with no real code, but it doesn’t seem as clean as 
>>>> an explicit keep alive.  I’d be interested in herring from others 
>>>> on that
>>>> [GC] Could we not, for example, declare a value/element/attribute 
>>>> 'test' to be passed in the notification with no IDs? Would that 
>>>> meet your idea of an explicit test mechanism?
>>> [br] I suppose.  Doesn’t seem much different than a parameter to 
>>> the findService, but I’m okay with it
>>>>
>>>>
>>>>
>>>> We probably need to say something about what to do if the URI test 
>>>> fails. Do we inform the Client that provided the URI that it may 
>>>> have been compromised? Do we simply drop the URI silently? Do we 
>>>> block the Client that provided the URI entirely?
>>>> At minimum wait and try again, the client could be down for a 
>>>> while.  I would not allow a lot of location records to be marked 
>>>> with the URI that didn’t test valid.
>>>>
>>>> I am somewhat leery of creating circumstances where the client has 
>>>> to go through and revalidate it’s entire record set because the 
>>>> mechanism failed in some way.  There clearly would be circumstances 
>>>> where that happened, but missing a keep alive shouldn’t be one of 
>>>> them.  The initial one however should be a hard block.  If you 
>>>> can’t get the first ID returned, then we have to not accept other 
>>>> registrations.
>>>> [GC] On a second thought, what's the value of testing the URI after 
>>>> the first test? If it passed, it's valid and the Server stores it 
>>>> for use when a planned change occurs. If after that and for some 
>>>> reason, the Client is not responsive to the notification, does the 
>>>> Server care?
>>> Generally when we have things that don’t do anything for months 
>>> and then we need it, I would like to see periodic keep alives.  
>>> Things change on both sides and both sides have a vested interest in 
>>> making sure the mechanism works when they need it.
>>>>
>>>>
>>>>
>>>> I’m not sure if the <command> thing is necessary since it 
>>>> requires the Server to keep state. I would think that if the URI 
>>>> has been tested and stored, it means it is active and valid until 
>>>> it is deleted through the reception of an empty <plannedChange> for 
>>>> example, or replaced through the reception of a new URI in 
>>>> <plannedChange>.
>>>> I don’t think there is any server state other than the URI is 
>>>> valid.  Each transaction stands alone.
>>>>
>>>> Without the delete, you can’t remove one location record from the 
>>>> set that gets notified of planned changes.  You can change the URI, 
>>>> but that changes it for all notifications.
>>>> [GC] Why? A Client that no longer wants to be notified for a given 
>>>> ID sends an empty <plannedChange> for the LI it no longer has. If 
>>>> that ID at the Server is associated with only one location (say one 
>>>> specific civic number), it deletes the Client's URI. If it is 
>>>> associated with more than one record (say a range of civic 
>>>> numbers), it accepts the transaction but keeps the URI. A 
>>>> notification for that ID sent to the Client that only had the civic 
>>>> number would result in a no-op.
>>> That means that the client has to be identified by something like 
>>> the TLS cert, because the URI isn’t there for the delete.  I was 
>>> using the domain of the URI as the ID of the client.
>>>
>>> But we have another problem: I think the server does not track the 
>>> LI.  On any findService with the new option, it looks for a matching 
>>> record and returns the ID it has for that record.  It’s the client 
>>> that tracks possible multiple LIs for one ID, not the server.  So in 
>>> your example, the client would notice that any address in the range 
>>> had the same ID.  It would get one notification if the range record 
>>> changed, and it would update all its records in the range.  But if 
>>> it deleted one of a set of records that have the same ID, it 
>>> wouldn’t request delete.  Only if it got down to one record with a 
>>> given ID, and it needed to delete that record, would it request 
>>> delete for that ID at the server.
>>>>
>>>>
>>>>
>>>> Lastly, I’m not sure we still need the warning ‘uriNotStored’ 
>>>> given that we’re going with one generic URI per Client. The only 
>>>> argument I can think of for keeping it would be the proliferation 
>>>> of Clients within the coverage aria of the Server that would be so 
>>>> large that the Server couldn’t store them all. A counter argument 
>>>> to that would be that doing so could be seen as giving privileges 
>>>> to certain Clients over others.
>>>>
>>>> We’re creating circumstances where the client won’t get a 
>>>> notification.  I think we need to tell them that.
>>>> [GC] Only if the URI fails the validity test.
>>> [br]I wanted to allow the server to limit the number of IDs it was 
>>> tracking for a given client.  I wanted to limit the number of 
>>> queries the server is responding to (although it might well silently 
>>> discard a DOS attack.
>>>>
>>>>
>>>> One limit I might put on a client is how many locations they can 
>>>> ask to be notified.  If they try to set up a situation where no 
>>>> matter what changed, they get a notification, I think that may be 
>>>> too much to ask, and the server can have a limit.  It would be okay 
>>>> if it’s one URI per valid record at the LIS, but the LoST server 
>>>> doesn’t have that data.
>>>> [GC] It's one generic URI per Client with one or more IDs stored by 
>>>> the Client against its LIs, right? As you suggesting we go back to 
>>>> one URI per Client record in its database?
>>> [br]  One generic URI per client.  So if the client asks for its URI 
>>> to be stored against too many IDs, it can be told no.
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Guy
>>>>
>>>>
>>>> De : Brian Rosen <[e-mail] <mailto:[e-mail]>>
>>>> Envoyé : 31 août 2021 13:54
>>>> À : Randall Gellens <[e-mail] 
>>>> <mailto:[e-mail]>>
>>>> Cc : Caron, Guy <[e-mail] <mailto:[e-mail]>>; ECRIT 
>>>> <[e-mail] <mailto:[e-mail]>>
>>>> Objet : [EXT]Re: [Ecrit] planned-changes: two questions
>>>>
>>>> Of course an attacker would conceal itself and register multiple 
>>>> URIs under different identities.  But the bigger  problem is that 
>>>> if a large planned change occurred, the victim could receive a 
>>>> large number of notifications it didn’t expect.
>>>>
>>>> With a test transition, we know that the client expects to see 
>>>> notifications, we only need one per URI (we could specify one per 
>>>> domain — after the scheme and before the first slash).  If we 
>>>> don’t get the ID, we know not to allow that URI to be 
>>>> provisioned.  So it’s one test transaction vs a large number of 
>>>> planned change transactions.
>>>>
>>>> Brian
>>>>
>>>>
>>>>
>>>>
>>>> On Aug 31, 2021, at 1:35 PM, Randall Gellens 
>>>> <[e-mail] <mailto:[e-mail]>> 
>>>> wrote:
>>>>
>>>> If a malicious client registers a URI that is designed to attack a 
>>>> third site, the test transaction causes the LoST server to connect 
>>>> to it and POST a command. Without a test transaction, the LoST 
>>>> server stores the URI and at a future time connects to it and sends 
>>>> a command. Either way, a LoST client can register potentially one 
>>>> URI per queried location, and a LoST server will connect to that 
>>>> URI and POST a message. A queried location could be tied to many 
>>>> other locations, so there could be many locations for which a 
>>>> change would trigger the LoST server to use the URI, but is that 
>>>> worse?
>>>> --Randall
>>>> On 31 Aug 2021, at 10:27, Brian Rosen wrote:
>>>> If the work group wants to have the LoST server keep the URI until 
>>>> expressly deleted, that’s okay with me.
>>>>
>>>> Authenticating the client to the server doesn’t mean the URI is 
>>>> authenticated.  We can’t restrict the URI to be the same entity 
>>>> as the client running the LoST transaction.  And the clients are 
>>>> wide ranging.  Could be an enterprise running its own LIS for 
>>>> example.  We can’t assume the North American PKI is workable 
>>>> everywhere, and even that doesn’t extend to an enterprise LIS.  
>>>> ISTM we have to run a test transaction with the notification 
>>>> service to make sure it’s what we think it is.
>>>>
>>>> Brian
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Aug 31, 2021, at 9:01 AM, Caron, Guy <[e-mail] 
>>>> <mailto:[e-mail]>> wrote:
>>>>
>>>> Inline.
>>>>
>>>> Guy
>>>>
>>>> -----Message d'origine-----
>>>> De : Brian Rosen <[e-mail] <mailto:[e-mail]>>
>>>> Envoyé : 31 août 2021 08:11
>>>> À : Caron, Guy <[e-mail] <mailto:[e-mail]>>
>>>> Cc : Randall Gellens <[e-mail] 
>>>> <mailto:[e-mail]>>; ECRIT <[e-mail] 
>>>> <mailto:[e-mail]>>
>>>> Objet : [EXT]Re: [Ecrit] planned-changes: two questions
>>>>
>>>> You delete the URI when you delete the record in the LIS.
>>>> [GC] That's fine but that was not the question Randall posed. He 
>>>> asked whether the URI is deleted after a notification. I agree that 
>>>> if the Client does not host the location anymore that the URI 
>>>> associated with that location in the Server should be deleted. This 
>>>> could be achieved with an empty <plannedChange>.
>>>>
>>>> I don’t think this is covered in 5222.  The mechanism causes the 
>>>> LoST server to send notifications to the client, but the client is 
>>>> allowed to put any URI in the record, and it can add it to as many 
>>>> records as it wants.
>>>> [GC] I thought we agreed on using one generic URI per Client. 
>>>> Clients should be authenticated by the Servers.
>>>>  An evil implementation could record URIs against multiple targets 
>>>> that were unaware that the evil implementation did it, until they 
>>>> got a large number of PUSH transactions they didn’t expect or 
>>>> understand as a result of a large planned change.
>>>> [GC] Only authenticated Clients should be allowed to provide URIs 
>>>> to be stored by the Servers.
>>>>
>>>> The proposed mechanism qualifies the client URI before its used in 
>>>> a planned change.
>>>>
>>>> Brian
>>>>
>>>>
>>>>
>>>> On Aug 31, 2021, at 7:37 AM, Caron, Guy <[e-mail] 
>>>> <mailto:[e-mail]>> wrote:
>>>>
>>>> Well, this is not going in the direction I thought.
>>>>
>>>> What is the purpose of deleting the URIs at the Server 
>>>> post-validation?
>>>>
>>>> Regarding opening a new DoS, I guess I'm not following. Wouldn't 
>>>> this case be covered by the security considerations in RFC 5222?
>>>>
>>>> What you're proposing puts back significant load on the Servers (a 
>>>> key consideration for creating planned-changes in the first place) 
>>>> and complicates the mechanism.
>>>>
>>>> Guy
>>>>
>>>>
>>>> -----Message d'origine-----
>>>> De : Ecrit <[e-mail] <mailto:[e-mail]>> 
>>>> De la part de Brian Rosen Envoyé :
>>>> 30 août 2021 11:22 À : Randall Gellens <[e-mail] 
>>>> <mailto:[e-mail]>> Cc
>>>> : ECRIT <[e-mail] <mailto:[e-mail]>> Objet : [EXT]Re: 
>>>> [Ecrit] planned-changes: two
>>>> questions
>>>>
>>>> Answer 1: yes.  Since there is going to be a revalidation, just 
>>>> deleting the setting seems right to me.
>>>> Answer 2: Up to server.  If I were implementing, I would hash the 
>>>> real ID with the URI and some kind of predictable nonce.
>>>>
>>>> We probably have to say more about how the server identifies the 
>>>> client, so that replacement of the URI works.  Could we say we use 
>>>> the domain of the URI (the entire domain with all the dots) to 
>>>> identify the client, and anything can occur after it (meaning a 
>>>> slash and whatever)?  If we do that, then how would delete the 
>>>> notification?  Force there to be something other than the domain 
>>>> (ugly).  Explicit delete request?
>>>>
>>>> Hmmm, we’ve opened a DoS attack: a rogue client stores a bunch of 
>>>> URIs for servers it wants to victimize.  In North America we have a 
>>>> real simple solution for that, because we have a PKI, so we know, 
>>>> for sure, who the client is, and could restrict who we allow to 
>>>> store URIs, but that wouldn’t be true in general.  Also, it would 
>>>> be nice for the client to have confidence the mechanism worked 
>>>> before it needed it.
>>>>
>>>> So
>>>> Let’s add a “command” to plannedChange in the findService 
>>>> request.
>>>> And, have the client have a response to the notification which is 
>>>> the
>>>> ID (json with the 200)
>>>>
>>>>
>>>> The client starts by sending a command of “initialize”.  The 
>>>> domain is the identity of the client.  The response is an immediate 
>>>> notification to the with whatever LI was in the request and an ID.  
>>>> The  response by the client (which is the notification web server) 
>>>> is a piece of json containing the ID.  We can say that the LI in 
>>>> this initialize command could be something simple like the Country 
>>>> Code that wouldn’t get a planned change.
>>>>
>>>> Thereafter, the LoST server (notification client) periodically 
>>>> repeats this keepalive notification every day or week with the 
>>>> initialize LI.  The client has to respond with the ID.
>>>>
>>>> The regular notification request is a command of “notify”.  The 
>>>> server ignores a request for notification from an uninitialized 
>>>> client.
>>>> The notification can be deleted with a command of “delete”.  If 
>>>> you delete the initialize LI, then the server won’t send any more 
>>>> notifications to that client and deletes all URIs it was saving for 
>>>> that client.  The client would have to re-initialize to reset.
>>>>
>>>> Brian
>>>>
>>>>
>>>>
>>>> On Aug 27, 2021, at 5:41 PM, Randall Gellens 
>>>> <[e-mail] <mailto:[e-mail]>> 
>>>> wrote:
>>>>
>>>> I think we're moving to a model where:
>>>> - In a query, a client can request to be notified when the location
>>>> should be revalidated;
>>>> - In the response, the server provides an ID which the client
>>>> associates with the location it just validated;
>>>> - The server sends a notification to the URI, containing the ID;
>>>> - The client revalidates each location with which that ID is 
>>>> associated.
>>>>
>>>> Question 1: Does the server delete/inactivate the URI once it has 
>>>> sent the notification?
>>>>
>>>> Question 2: Presumably, when the client revalidates the 
>>>> location(s), it will again request notification.  Does the server 
>>>> return the same ID as before, or a different ID?  A different ID 
>>>> could perhaps be useful in edge cases where the server didn't send 
>>>> or the client didn't get the notification, but any utility seems 
>>>> small.  If it's the same ID, then the answer to question 1 can be 
>>>> that the URI remains active until the client asks to no longer be 
>>>> notified (by sending an empty URI?).
>>>>
>>>> --Randall
>>>>
>>>> _______________________________________________
>>>> Ecrit mailing list
>>>> [e-mail] <mailto:[e-mail]>
>>>> https://www.ietf.org/mailman/listinfo/ecrit 
>>>> <https://www.ietf.org/mailman/listinfo/ecrit>
>>>>
>>>> _______________________________________________
>>>> Ecrit mailing list
>>>> [e-mail] <mailto:[e-mail]>
>>>> https://www.ietf.org/mailman/listinfo/ecrit 
>>>> <https://www.ietf.org/mailman/listinfo/ecrit>
>>>> ----------------------------------------------------------------------
>>>> -------- External Email: Please use caution when opening links and
>>>> attachments / Courriel externe: Soyez prudent avec les liens et
>>>> documents joints
>>>>
>>>> ------------------------------------------------------------------------------
>>>> External Email: Please use caution when opening links and 
>>>> attachments / Courriel externe: Soyez prudent avec les liens et 
>>>> documents joints
>>>>
>>>>
>>>> External Email: Please use caution when opening links and 
>>>> attachments / Courriel externe: Soyez prudent avec les liens et 
>>>> documents joints
>>>>
>>>> External Email: Please use caution when opening links and 
>>>> attachments / Courriel externe: Soyez prudent avec les liens et 
>>>> documents joints
>>>
>>