Re: [regext] Poll messages with unhandled namespaces (was Re: I-D Action: draft-ietf-regext-change-poll-07.txt)

["Gould, James" <[e-mail]>]

Patrick,

Thank you for the detailed description of your reasoning.  I will leave it that we disagree on the purpose of the login services, the need for the server to honor the login services for poll messaging, and the impacts in returning unsupported responses or response extensions to clients.  On the impact, there are two elements to consider including first the XML parser in validating a response against the XML schemas, and second is the unmarshalling of the XML to a representative object.  A client could disable the XML parser validation and move along to the unmarshalling step.  In the unmarshalling step the client would need to deal with receipt of unsupported content.  The client could throw an exception because the XML (e.g., DOM tree) could not be unmarshalled, ignore the unsupported content, or come up with some other representation of the unsupported content (e.g., convert to list of XML blobs or DOM objects).  A client would need to proactively deal with the unexpected if the server does not honor the login services of the client.  With the inclusion of the greeting and login services, it seems clear to me that the intention of the protocol is not to have the clients deal with the unexpected.  

Since this is not a problem unique to draft-ietf-regext-change-poll, I agree that it's best suited to be addressed in a separate Internet Draft that addresses service-aware EPP poll messaging.  Is there interest in such a draft by the working group?  

Thanks,
  
—
 
JG



James Gould
Distinguished Engineer
[e-mail]

703-948-3271
12061 Bluemont Way
Reston, VA 20190

Verisign.com <http://verisigninc.com/> 
On 4/20/18, 1:58 AM, "regext on behalf of Patrick Mevzek" <[e-mail] on behalf of [e-mail]> wrote:

    On Wed, Apr 18, 2018, at 15:43, Gould, James wrote:
    > This particular thread was started Martin Casanova, who brings up an 
    > excellent point that needs to be considered.  Martin, you may want to 
    > weigh on this.
    
    You can probably admit that, based on the amount of time the problem exists (basically since registries started to use EPP), and the number since then on mailing-lists about this problem, that it is not an hot topic.
    That does not mean it can not be addressed, but it may mean that registries/registrars already found out solutions to this.
    
    > I have the following questions and related thoughts:
    > 
    > 
    >   1.  Do you believe that it is protocol compliant for the server to 
    > return something that the client did not explicitly include in the login 
    > services?
    
    It is certainly not ideal, but not a big problem either (in this specific case for the specific reasons I gave and will give below), and certainly to my eyes less a problem than putting format into a <msg> element that is defined for human consumption (even more so when you mix both text and XML elements in the same <msg> element, which is a peril waiting to happen)
    
    >   2.  What is the purpose of the server’s greeting services and the 
    > client’s login services?
    >      *   I believe that the server and client services are used to 
    > negotiate the common set of supporting services.  The client should not 
    > pass services that the server does not support and vice-versa.
    
    It is not a full negotiation since there is only one exchange and only a binary result possible (accepting the login or not).
    
    Text says:
    "A <svcs> element that contains one or more <objURI> elements that
          contain namespace URIs representing the objects to be managed
          during the session."
    
    Note that it says objects, not messages or commands.
    For me it is more tailored to let the client advertise which objects he wants to manipulate at the registry, that is which namespaces will appear in its commands.
    Imagine a registry handling both domain names and let's say AS number or IP blocks, within the same EPP server.
    Upon login, some clients may have rights to only manage domains or only manage AS numbers. So they advertise what they would like to do, based on server greeting. And then the server can act on that.
    
    For me the <svcs> definition in RFC5730 does not restrict the notifications messages namespaces. So I think from this disagreement, we go then on separate ways.
    
    >   3.  What do you do with validating clients?
    >      *   The Verisign EPP SDK is one implementation of a validating 
    > client, but there can be others.  A validating client would either need 
    > to be updated to support the extension or turn off validation and be 
    > capable of processing a message that it doesn’t support.  If the 
    > protocol is followed, there should be no issue with validating the XML 
    > per the supported XML schemas in either the server or the client based 
    > on #2.
    
    Again, the validating client can recognize it can not validate the notification message (if it does it synchronously which you suppose and for which I already said that there are ways to do it differently), and just dump it aside for further/later processing and before being updated if you insist on in-session immediate validation.
    
    I think you are missing some use cases. Let me share things I have seen in various places.
    
    First, from my experience, many registries (I can not give precise numbers I would say in the ballpark of 1 among 2) define their own namespace extension for notification messages, with more or less format.
    I doubt that a validating client such as Verisign EPP SDK would work great in this case if it wants to be used for many ccTLDs especially and validate all notifications messages. It just does not fit what is happening.
    Note that you do not have this problem in gTLDs where far less extensions are used.
    
    Of course it would be so much simpler if they were not so many extensions doing the same things, but the situation is like it is for now and I doubt this will change soon, or anytime in fact.
    
    Another important point: I know multiple EPP clients (mine included for part of the following, but I know others too) that exhibit, on purpose, one or more of the following traits:
    
    1) not validating incoming EPP messages and even more often not validating outgoing ones.
    Why? Because when you try to support many registries you tend to observe that not all registries are strictly conforming to their own EPP schemas; so it happens that you get as client some EPP messages that do not validate at all, even if given  by the registry. Of course you will say that this is the server fault and that it should be fixed (or the schema amended) and it is of course the true technical explanation but then when you have to conduct your business you can not wait on the registry to be fixed, your client should not be halted by some bogus format, it should find workarounds.
    
    Note that it also happened for core schema. I know one registry particularly that re-used the <domain:reason> element in a <domain:check> command for more that it was designed to (which also illustrates the peril of shoehorning a formatted structure inside a free text element), and just forgetting that its schema forces it to be less 32 characters or less... and some messages did have a reason larger than that. Some registrars were not happy about that when they started to receive such messages...
    
    So how a validating client is supposed to handle that? Just refuse the message and make it impossible to do any domain:check?
    Again, certainly possible technically, but not realistic business-wise.
    
    One could also paste here the Postel law, or its half: be liberal in what you accept
    (which breaks horribly when applied in security contexts but that is another story).
    
    2) not processing notifications at all in-session, just acking all messages immediately, after having stored them locally in a DB or something which will get consumed by some asynchronous processes.
    To me this looks very efficient and solve multiple problems at once. Including the validation problem or even for message 100% validated the problem of parsing them.
    Take for example the transfer case and registries sending notifications about transfers being started, refused, accepted, etc.
    Not all of them, again specifically in ccTLDs, are using the standard panData element. Many of them are putting all the information (domain name, status of transfer, other registrar ID, etc...) as free text in the <msg> part of the notification. You have to parse that. You can not build a client that will have this knowledge even before connecting to the EPP server for the simple reason that often there is not even a comprehensive documentation from the registry with all the messages. So you will have to pick them up during your life. You either try to parse them immediately, or you store them for later. If you want to validate and parse them immediately what will you do for all unknown cases? Just throwing a fatal exception will again not help the registrar to do its business.
    
    So I applaud the Verisign SDK to be strict and to stick to the technical rules and do validation always, I just do not think it could be used as is to connect to many ccTLDs registries because you will then be hit by many new namespaces, specific to one registry, that are not necessarily fully documented nor always respected by the registry itself.
    
    To put things in perspective, extracted from my client, here are some namespaces related to registry notification messages a client can receive:
    
    http://www.afnic.fr/xml/epp/frnic-1.4
    http://rxsd.domain-registry.nl/sidn-ext-epp-1.0
    http://tld-box.at/xmlns/resdata-1.1
    http://www.nic.at/xsd/at-ext-message-1.0
    http://www.nominet.org.uk/epp/xml/std-notifications-1.2
    urn:afilias:params:xml:ns:oxrs-1.1
    urn:ietf:params:xml:ns:changePoll-1.0
    urn:ietf:params:xml:ns:cira-1.0
    urn:ietf:params:xml:ns:extreport-1.0
    urn:ietf:params:xml:ns:poll-1.0
    
    and this is certainly not exhaustive...
    
    How many open or closed EPP clients do you know that are capable of both parsing all these extensions and validating them (putting aside the problem of receiving an EPP frame not even respecting the schema associated to the namespace)? I bet you can count them on one hand...
    
    > You are correct that there are a set of existing EPP extensions that do 
    > include poll messages (e.g., RGP Poll Mapping, Low Balance Mapping, 
    > Launch Phase Extension, and object mappings that support transfer poll 
    > messages like Email Forwarding and Defensive Registration).  This is 
    > based on the extensions registered in the EPP Extension Registry 
    > (https://www.iana.org/assignments/epp-extensions/epp-extensions.xhtml).  
    
    There are unfortunately a very small percentage of all extensions that exist...
    Current version of my client knows about 145 different EPP namespaces...
    And I am willing to bet that there are in fact more than 200 in the wild.
    
    As written above, I have no idea how to do an EPP client capable of
    validating all of them.
    
    > With the exception of the RGP Poll Mapping and the Low Balance Mapping, 
    > the poll messages are associated with actions that the client previously 
    > took, so it’s reasonable to assume that the client supports it.
    
    This assumptions fail completely for all the cases outlined above, specially
    in ccTLDs.
    There are many other exceptions in the wild besides the two you list.
    
    One example: the registry notifing the registrar that it just deleted one of its contact or host because it became unlinked and obsolete after some policy delay...
    
    > Since it looks like Patrick and I are most likely at an impasse, I ask 
    > the list to weigh in on the two options that include:
    
    I disagree. We are not at an impasse. You see a problem and wish to resolve it by some solution. If you are motivated to defend it, write a draft and let the WG consensus build itself on it, if it reaches other interests.
    I am certainly not here to say not to do it.
    
    I have no counter proposal to yours since my solution is basically "do nothing", as the status quo, while not perfect, seems good enough to me or at least best than the solution we speak here about. But of course it would help if registrars specially come and share their experiences and "wishlist".
    
    So, feel free to go forward with a draft if you like. 
    
    >   2.  Server considers the login services and filters non-supported 
    > extensions (object and command / response) from the poll message 
    > response with an encoded <msgQ><msg> indicating the extension namepaces 
    > not supported.  
    
    This also raises an another important problem, partially depending on policy.
    Let us imagine the registry starts some new notification messages under a new namespace (or, less radical: just changes some namespace version).
    At the exact moment this happens all currently logged in clients, as well as all future clients not having been updated will know nothing about this namespace hence, per your proposal, the server will filter out these messages.
    
    Where will these messages go? If they are lost for good it is bad as the registrar will have no way to know that (except parsing in a new way the msg element, which involves updates), at all. Even maybe no way to know it needs to be updated (whereas if it started to receive new messages it would just see the new namespace and could act or error on it, but at least decide itself what it wants to do). Maybe the client will just get updated hours or days later, but even in any tiny timeframe some messages may come like that and be lost forever... (the registrar can not necessarily wait on the first of such message by not acknowledging it until it gets updated, because this will block it from retrieveing the following ones, that may contain even more important information for its business). I think this is taxing registrars too much.
    
    You suppose also that the registrars will get updated to parse the new <msg> format: this is an assumption that can break.
    
    Filtering messages at the server side seems creating more problems than solutions to me.
    This is tied to the problem of sitting messages not being acknowledged by the registrar. Some registries are then sending them by email for example after a certain delay to be able to purge the queues.
    I know others where there is no purge and you can find messages almost 10 years old...
    (the usefulness of such messages now is seriously debattable...)
    But at least not just filtering some out and making them disappear definitively.
    
    -- 
      Patrick Mevzek
      [e-mail]
    
    _______________________________________________
    regext mailing list
    [e-mail]
    https://www.ietf.org/mailman/listinfo/regext