Re: [lisp] H3 steering committee review on draft-ietf-lisp-nexagon-36

[Sharon Barkai <[e-mail]>]

Dear Isaac and H3 Steering Committee, 

Thank you very much again for the detailed thoughtful review of drat-ietf-lisp-nexagon.
Will try to incorporate most of your comments already in the next draft.
Bellow please find authors comments. Once again thank you for the work!




H3 Steering Review IETF-LISP-NEXAGON Response



>> We are happy to see standards using H3 as the spatial indexing system being proposed. We believe H3 is well suited to use in the mobility space, and in general it seems like an excellent choice for a mobility edge network using the LISP protocol.

Likewise, the synergies between H3 spatial indexing and logical IPv6 (EID) addressing are indeed powerful and lend themselves to spatial realtime information sharing. The authors believe this is evident in the lisp-nexagon draft and can point the way for next generation AR applications. Addressing application routing challenges using existing well defined virtualized layer3 specifications such as LISP-RFCs can help application developers avoid redefining the same mechanisms again and again at layer7 per application where the core application routing functionality is essentially the same: source routed to shards, portable elasticity of shards, publish-subscribe and privacy of client applications. 

Addressing application routing challenges using layer3 network virtualization can also reduce latency, queueing overheads, and DNS back and forth interruptions, especially in Edge IoT applications. The synergies between H3 spatial indexing and LISP IPv6 EIDs and the resulting LISP based application routing support simpler interoperability in multi-vendor eco-system such as active spatial mapping involving automotive OEMs, Mobility Service Providers, and mobile applications.

Some of the excellent proposals brought forth by the H3 steering committee and referenced below have to do with implementation behavior of the addressable entities represented by the MobiliyClientEIDs and H3ServiceEID. Namely, proposals regarding intelligent scoping by the clients, improved clustering-consolidation by the services, and optimal sharding by the backend as a whole are good and useful. We will try to incorporate these proposals into the draft without deviating from its layer3 LISP interface and logical-addressing / service-routing focus. 

>>On page 11 it is suggested that multicast subscription is to a singular coarse-resolution cell (h3.rB) based on the parent-child relationship with the high-resolution cell (h3.rS) the client is located within.

The multicast subscriptions are indeed for coarse resolution cells but most likely to more than one. The subscriptions use standard lisp multicast RFC 8378 per client logic determining areas of interest. Each client will likely subscribe to areas of interest in its route and adjacent areas to those as (simply!) identified using H3 indexing. Non vehicle clients such as muni surge charging for optimal street use per conditions, as a real life example, will subscribe per CRM requirements. The next draft will clarify this point better.

>> First, the parent-child relationship is a simple tree based on approximate containment, which works decently well between neighboring resolutions, but becomes less-and-less contained as the delta in the resolutions increases, as can be seen here <https://observablehq.com/@nrabinowitz/h3-hierarchical-non-containment?collection=@nrabinowitz/h3>. Second, there will be an “edge effect” as the client in question moves towards the edge of the coarse-resolution cell as the majority of the data in the feed will be “behind” it. 

That is correct the client logic should account for for these factors when applying subscription logic, also these “rough” estimate of areas of interest are a factor in determining the size of the coarse resolution used for shards, too large will mean more irrelevant information filtered in the client, and too small will mean more edge effects. A reasonable choice during testing was resolution 9 or 10 but it is not mandated in the draft.

>> It would be more useful to keep the client close to the center of its data feed by instead subscribing to a kRing of hexagon IDs <https://h3geo.org/docs/api/traversal#kring> centered on its location rather than a singular hexagon. Even just a kRing of 1 would reduce this problem significantly, and there would be a net-zero impact on data volume if the coarse-resolution hexagon ID was reduced in conjunction with the kRing to produce an equal area coverage (h3.rB == kRing(h3.r(B-1), 1) == kRing(h3.r(B-2), 3), etc).

This is a very good recommendation for the client logic when using lisp-nexagon interface for subscription.

>>On page 4 you can find the only explanation of how the backend for this protocol should be developed, with a reference to a “density-factor” fudge factor and apparently explicit allocation of the coarse-resolution hexagons (h3.rB) to specific servers based on data volume. This could work for data ingest, but there is no explanation on how the subscribers will get all messages when they are sharded between servers, nor what would happen if the subscribers cannot ingest the total data volume in question. We believe this is a suboptimal architecture, having prior experience at Uber dealing with something similar, and are willing to provide an alternative, but given the small amount of space in the IETF dedicated to the backend implementation, we aren’t sure if this is the point of the RFC.

That is correct, the standard interface can not mandate an application backend implementation. It is assumed that the backend shards are made portable between servers virtue of the fact that the upstream queues and the downstream channels are EID based. Thus given LISP portability of queues and channels, any functional design will most likely be portable as functions are pervasive and state regenerates. Additional optimizations can be made by exlerated bootstrapping of a moved shard based on recent packet capture of its latest channel feed. This is however not specified.

The shard size recommendation is that channel feeds will fit within “few” UDP MTUs. Tests on urban areas show resolution 9 or 10 are generally in that scope. For low density areas such resolutions will be sub-optimal and much more coarse or larger tiles will be more appropriate. However for simplicity only two resolution sizes are specified H3.rS for detections and H3.rB for aggregation. This simplicity consideration which is suboptimal for rural can be mitigated using LISP longest match mapping quality, by which a de-facto larger shard can be the route destination for containing multiple logical H3.rB tiles. This optimization is not specified but if applied will remain interoperable. 

>> On page 9 the unicast detection upload logic is described and the precise message payload outlined. We noticed that location accuracy estimation (from the combined uncertainties of the GPS device and the offset estimation) are not included, only the high resolution H3 cell ID (h3.rS). Due to later sections dedicated to the importance of clients being able to filter out poor-quality detections from their subscriptions, we believe the estimated accuracy radius should be included in some form. In the appendix at the end of this review we have a set of proposals of varying levels of overhead that could add to or modify the byte layout of the header format.

That is correct, the H3.rS localization by the producing client will have varying errors. The consolidation data-clustering logic associated with the EIDs of H3.rB should account for that. There are visual methods to further correct the localization errors of the producing clients. These will not be specified in the RFC but can be augmented by vendor implementations while maintaining multi-vendor interoperability. The proposal in the appendix seems to be very suited to the aggregation consolidation logic, which combined with the accuracy score encoded in the EID, its vector projectile as indicated by previous uploads, additional detections, and snap to road logic for the consolidated state sent using the shard EID channel. Estimated error can also be given by the producing clients, but will be finally determined by the H3EIDService consolidation shard.


>> On page 15 and page 16, there are two contradictory statements surrounding the MobilityClientEIDs. On page 15 it is said that clients could learn to trust or distrust data sources by MobilityClientEID and on page 16 it is said that clients would remain anonymous because their MobilityClientEIDs would shift frequently. It would not be possible to satisfy both – if the trustworthiness of the data source is resolved, then it would be possible to make a solid guess as to its current location and the path it has taken, while if the MobilityClientEIDs shift frequently enough to prevent this, it isn’t possible to determine if the data in question is valid or spoofed in some way or another

This is indeed a sensitive point. The MobilityClientEID is changed every T (15) min to prevent tracking and spoofing. These EIDs are allocated by AAA which on one hand has the real identity and the aggregated accuracy score of the client. The AAA receives aggregated score from all shards regarding accuracy of MobilityClientEID detections so can not know from which shards this score was obtained making tracking hard. This is an additional shard size (H3.rB) consideration, the bigger the shard size the more track-able the client but the less meaningful the tracking. The smaller the shards the more meaningful the tracking but less likely given a larger pool of scoring elements.

>>We don’t consider this a solved problem so this is not a hard criticism, but these two sections should be reworded to be more explicit about the trade-off here. For particular, tightly-regulated implementations of this protocol, however, the closest thing to a solution would be the TPM system Microsoft has developed <https://support.microsoft.com/en-us/topic/what-is-tpm-705f241d-025d-4470-80c5-4feeb24fa1ee>, which would allow a central authority to verify the GPS and mainboard firmware are signed builds known to the central authority, and then configure the clients to trust all data streamed to them as all clients are verified during the login process described on page 5, and a TPM-based handshake could be employed.
 
This is the general idea of separating AAA from the geolocation shards the clients interact with. The aggregate rolling “creditScore” is reported back to the AAA but without shard distinction. Will further explore the MS solution as well  as geo-privacy, in addition to the multi-vendor ecosystem, is one of the considerations on using a standards based application routing rather than any vendor specific. 

>>On page 1 (and several other pages) the terminology “nexagon” is used without explanation. What does this mean?

True will correct that in the next draft. 


> On Jul 27, 2022, at 21:03, Isaac Brodsky <[e-mail]> wrote:
> 
> Hello,
> 
> Please find the review by the H3 steering committee on https://datatracker.ietf.org/doc/html/draft-ietf-lisp-nexagon-36 <https://datatracker.ietf.org/doc/html/draft-ietf-lisp-nexagon-36> at this Google Doc link:
> 
> https://docs.google.com/document/d/1tjImJGJrKsalmHJQ7hKpidPyZNLNfSKupX6-dr4vT9w/edit# <https://docs.google.com/document/d/1tjImJGJrKsalmHJQ7hKpidPyZNLNfSKupX6-dr4vT9w/edit#>
> 
> Thanks,
> Isaac