Re: [scim] [EXTERNAL] Re: IETF 114 preliminary Agenda and call for other agenda requests

[Danny Zollner <[e-mail]>]

The answer to that is that it isn’t always feasible. There are apps out there with single customer tenants with 500K-1M+ user accounts, and sometimes tens or hundreds of thousands of groups. In some cases you can filter results to help – this is sometimes applicable when the client is pushing data to the server, for example. If the client is pulling data, however, there are use cases (i.e.: HR system data needing to be 100% retrieved by a client on a regular basis) where both change detection(delta query) and pagination are integral to accomplishing this efficiently.

-Danny

From: Phillip Hunt <[e-mail]>
Sent: Wednesday, July 13, 2022 5:10 PM
To: Danny Zollner <[e-mail]>
Cc: Matt Peterson <[e-mail]>; Nancy Cam-Winget (ncamwing) <ncamwing=[e-mail]>; [e-mail]; Pamela Dingle <[e-mail]>; Janelle Allen (janelall) <[e-mail]>
Subject: Re: [EXTERNAL] Re: IETF 114 preliminary Agenda and call for other agenda requests

Danny

The question remains. Why not download the entire result set in one GET?

Phil


On Jul 13, 2022, at 2:30 PM, Danny Zollner <[e-mail]<mailto:[e-mail]>> wrote:

Hi Phil,

On the topic of cursor-based pagination, I’m not sure how it would conflict with the SCIM Events draft. I think this is a case of different participants in the working group coming at the spec from different use cases. In the case that I’m trying to represent, it’s a client/server model where a client is trying to retrieve large amounts of data at once. In the same way that some SCIM implementations may not wish to implement the SCIM Events draft, cursor-based pagination as an optional extension to provide a method to break large amounts of data into smaller, more efficiently consumable chunks should be fine. For those who cannot implement it due to their own architecture limitations, they can potentially look at other options for traversal of large amounts of data.

On the topic of multi-valued attribute pagination and filtering, I (and Microsoft) are very interested in this, and from discussions with some service provider implementers, I believe there is adequate interest. Specifically on the filtering piece, I recall that there was some previous feedback about the value of it. I do not have a strong opinion on the filtering component, but given the past feedback I was suggesting that it may be worth discussing. I’d like the working group to discuss your draft, draft-hunt-scim-mv-filtering. If you’d like to present on it I think that would be great, otherwise we can attempt to have someone else speak to it if that’s OK.

On the topic of change detection, this again may be a case of different use cases and lack of precision in the terminology that I used in my last email – sorry for that. While ETAGs are helpful in some circumstances such as version controlling when multiple actors may be interacting with the same resource, the problem I’m hoping to see solved is different. Similar to the above profile of a client/server model where a large number of resources are being retrieved, I’m proposing some form of change tracking that allows a client to request from a server “Return all objects that have changed since some prior point in time”. This of course could also have filters applied, so you could structure requests like “Return all users where department = Sales that have changed since some prior point in time”. The problem is then – how do you represent that point in time? You mentioned detecting changes either via ETAGs or via “has/has not changed since a specified date”. Having discussed this topic with some of my more senior coworkers, the feedback I’ve heard is that dates of any format (lastModified, etc..) are a bad choice as they are not monotonically increasing, and services backed by multiple physical devices (so.. almost anything operating at large scale nowadays?) won’t have consistent dates/times between them. The approach I’ve got in mind right now would be a state cookie that is managed by the server and is likely opaque to the client.

Finally, on the topic of version incrementing/changes to the existing core schema and protocol docs, I am not advocating for incrementing the version or making any specific changes at this point, but rather that we defer on this until later, as there is ample work in front of us with the dozen+ topics that can easily be implemented as extensions. We should start compiling a list of proposed changes that have to occur in the core schema and protocol RFCs rather than in an extension, and once we believe we’ve exhausted things there can discuss how best to implement those changes at that time (the version increment/reissue 2.0/other option discussion).

Thanks,

Danny



From: Phillip Hunt <[e-mail]<mailto:[e-mail]>>
Sent: Wednesday, July 13, 2022 11:42 AM
To: Danny Zollner <[e-mail]<mailto:[e-mail]>>
Cc: Matt Peterson <[e-mail]<mailto:[e-mail]>>; Nancy Cam-Winget (ncamwing) <ncamwing=[e-mail]<mailto:ncamwing=[e-mail]>>; [e-mail]<mailto:[e-mail]>; Pamela Dingle <[e-mail]<mailto:[e-mail]>>; Janelle Allen (janelall) <[e-mail]<mailto:[e-mail]>>
Subject: [EXTERNAL] Re: IETF 114 preliminary Agenda and call for other agenda requests

I’ve put some comments below. These are discussion items that presenters might want to include in their presentation.

Phillip Hunt
@independentid
[e-mail]<mailto:[e-mail]>





On Jul 12, 2022, at 9:12 PM, Danny Zollner <[e-mail]<mailto:[e-mail]>> wrote:

I’d like to cover the following items related to protocol and schema work:


  *   Cursor-based pagination – via Matt’s draft (that I somehow didn’t realize existed until last month!)
<PH> Much of the use case for this was to support replication and co-ordination of events across domains. How does/doesnot this specification conflict with the SCIM Events WG draft?

I remain ocncerned about the resource costs for service providers and the denial of service attack vector that results.  I have heard that many clients can’t hold a result set in memory. Fair enough. But in this case, would it be better to use a streaming parser OR simply store results direct to local disk for processing?

Why cursors are hard:  If a client has limited resources to store one result set, the cost to a service provider with thousands of clients is multiplied and may not be possible.  In todays systems, causing a SCIM service provider to hold temporary state over data impacts a cluster in many ways.  Today’s server is simply making calls to a database and doing some translation. The server itself is largely stateless. With cursors, the results sets have to be maintained and could effectively mean storing a copy of the entire database. In clusters enviornments (most of them) access to that result set likely needs to be shared.  If not the load-balancer has to maintain “sticky” connections so that each client request goes to the same cluster node.  This could still lead to problems if the client is too slow and the connection is reset. As written now, SCIM servers don’t need to hold “state” across requests.  Implementing cursors would mandate a major re-write for most implementations.

There is also a mistaken assumption that we need paging because SCIM has a result size limitation. That’s not part of SCIM Protocol. The spec simply permits service providers to set limits for things like Javascript UI clients. Presumably for a priviledged provisioning client, they ahve access to the enitre data set. It makes no sense to enforce a 1000 result set limit in these scenarios.

—> The problem is that many implementations make max results a server setting rather than an access control entitlement.  This is not so much something we need to deal with. We may just need to alert industry product managers of why they need to support provisioning clients properly.



  *
  *   Multi-valued attribute pagination – Phil’s draft is a good place to start there. Based on feedback on the mailing list a few months ago, I think there are questions on whether multi-valued attribute filtering is needed, which is also included in Phil’s draft. Splitting multi-valued attribute pagination away from multi-valued attribute filtering if/when it is adopted by the working group may be the best choice.
<PH> The current draft re-uses aspects of SCIM that already supports CMVA filtering. For this reaon, pulling the functionality apart doesn’t make sense. AFAIK there are only a couple of requestors (e.g. Gregg Wilson/Oracle) for this draft.  Without wider support, should we drop the spec?

I wasn’t planning on presenting this. Please let me know if I should.



  *
  *   Discussion on change detection/delta query approaches – I’ll prepare some slides on this and may try to send out an email this week or next week as well, but won’t have a draft for this by IETF 114.
<PH> The current SCIM Protocol spec indicates support for ETAGs specifically in Sec 3.14 of RFC7644 (versioning resources) and can be used in queries. In SCIM’s case an ETAG is a hash of the resource. ETAGs more broadly in HTTP allow a requestor to make queries or changes using HTTP Preconditions (RFC7232).  For example, you can do a GET conditional only on the resource having changed. You can also condity a modify (PUT, PATCH, DELETE) on a resource being unchanged (has a specific etag) or has/has not changed since a specified date.  I suspect pre-conditions are not widely supported in many SCIM implementations.  I have implemented it in i2scim.io<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fi2scim.io%2F&data=05%7C01%7CDanny.Zollner%40microsoft.com%7C67dd70a985724464e8c108da651c6ecb%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637933470070640532%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=l1kG4a%2B%2FUeYHF9rs2H3kX38HJp1UAbGrFBNX%2Bhbnc1U%3D&reserved=0>.

I am not sure if this is what you are really after, but you may want to distinguish your requirement from the HTTP standard if different.  :-)

Also, SCIM Events intent is to notify subscribers of changes in real time.  In the simplest message, the new etag is shared (an etag is a hash of the resource).



  *
  *   Soft deletion
  *   Expansion of account status context beyond active true/false – Janelle is working on a draft for this, although I do not know if this will make it in time for IETF 114
  *   My roles/entitlements draft – it has expired but I’ll renew it and would like to put it up for a call for adoption. I don’t know if we need to figure out the contains/containedBy topic that I wrote to the mailing list about last month prior to adoption or if that can be left for the working group to figure out later
  *   HR Schema action plan/next steps
  *   Resetting the milestones that we set last year? (Do we need to worry about this?)

     *   Current plan is that we work on a bunch of the new features in the form of separate extension drafts to SCIM 2.0, and leave RFC 7643 and 7644 alone for now. Once we’re further along in the new features (i.e.: the above + others from the charter) we can determine the final approach of how to make changes to the protocol and schema – that is, do we revise SCIM 2.0 schema and protocol docs while remaining 2.0 but issuing new RFCs (apparently possible), or increment version.
<PH> I haven’t really seen a need to go to SCIM 2.x or 3. Almost all the enhancements can be positioned as extensions to the current SCIM 2 RFCs.

Some of the discussion about extending groups etc suggest much more complex CMVA objects (e.g. verified claims etc). This is interesting. But I would prefer to simply deifne new objects rather than version the protocol first.  There is also nothing saying you can’t have a group represented “virtually” where extended schema is available at one resource path “/<extended>Groups/“ while “/Groups” holds the old schema version of the same groups.

The more complex extension would be to add meta data to attributes like roles.   In some systems, a role binding is actually a policy that has thins like conditions and expiry dates.
—> I guess what I am saying here is this topic deserves a lot of dinner time and hallway discussion over many glasses of wine and beers.

As for editorial clean ups, I am not sure myself what the IETF permits moving from “proposed standard” to “standard” which is another possible route.   In particular, I think many of the examples need to be improved as people depend too much on figures and have gotten mis-leading impressions.  Some of the figures need to better reflect the normative text intent.  I have a concern that many don’t realize that SCIM is technicly a profile of HTTP and that the HTTP RFC do apply. I wonder if in the introduction, we can make this more obvious/clear.

There is one issue with PATCH that we should clarify.  I believe it was wether replacing a value for an attribute value that does not exist MAY be converted to an ADD.  In Postel’s ROBUST design, I would say go for it. There are others that say the transaction that must fail.  The spec is ambiguous on this point and any clarfication would result in a normative change for some.


I think these topics could easily take an hour, if not more. Whatever time is not used by use cases or SCIM Events can be allocated here.

Thanks,

Danny Zollner

From: Matt Peterson (mpeterso) <[e-mail]<mailto:[e-mail]>>
Sent: Monday, July 11, 2022 11:35 PM
To: Nancy Cam-Winget (ncamwing) <ncamwing=[e-mail]<mailto:ncamwing=[e-mail]>>; [e-mail]<mailto:[e-mail]>; Pamela Dingle <[e-mail]<mailto:[e-mail]>>; Janelle Allen (janelall) <[e-mail]<mailto:[e-mail]>>; Danny Zollner <[e-mail]<mailto:[e-mail]>>; Phillip Hunt <[e-mail]<mailto:[e-mail]>>
Subject: [EXTERNAL] RE: IETF 114 preliminary Agenda and call for other agenda requests

Nancy,

There has been recent activity on the mailing list (and also discussion in the SCIM interest group before our chart) that shows continued interest in cursor pagination (draft-peterson-scim-cursor-pagination).

If appropriate for our charter, I would like one more opportunity to check if cursor pagination is interesting to enough people to warrant effort on my part to create a new revision of the draft to address 2 interoperability issues that were raised on the mailing list last month.

--
Matt

From: scim <[e-mail]<mailto:[e-mail]>> On Behalf Of Nancy Cam-Winget (ncamwing)
Sent: Monday, July 11, 2022 6:51 PM
To: [e-mail]<mailto:[e-mail]>; Pamela Dingle <[e-mail]<mailto:[e-mail]>>; Janelle Allen (janelall) <[e-mail]<mailto:[e-mail]>>; Danny Zollner <[e-mail]<mailto:[e-mail]>>; Phillip Hunt <[e-mail]<mailto:[e-mail]>>
Subject: [scim] IETF 114 preliminary Agenda and call for other agenda requests

CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.

Hello SCIMers,

Here is the current proposed agenda and a call for other requests:

5 min: Agenda Bash & Logistics
                SCIM Chairs: Nancy Cam-Winget, Aaron Parecki

TDB min: Use Cases
                Pamela Dingle

TBD min: Protocol & Schema
                Janelle Allen and Danny Zollner

TBD min: SCIM Events
                Phil Hunt
AOB

@Pamela Dingle<mailto:[e-mail]>, @Janelle Allen (janelall)<mailto:[e-mail]>, @Danny Zollner<mailto:[e-mail]> and @Phillip Hunt<mailto:[e-mail]>: can you provide requested times to provide updates on the work above?

If there are other requests for agenda slots, please reply to this email or send a request to [e-mail]<mailto:[e-mail]>

Thanks, Nancy