Re: [scim] IETF 114 preliminary Agenda and call for other agenda requests

["Nancy Cam-Winget (ncamwing)" <[e-mail]>]

Hi Phil,

Speaking as a participant (not chair), I believe the SCIM events draft provide other capabilities beyond the filtering being discussed so
I would not discount its usefulness based on the pagination discussion.

That said (now as chair), we do need to discuss how the use cases that require filtering whether thru indexing, pagination or other proposals.
But we should describe the use cases as a formulation of requirements before we can address whether one proposal is better than another and
I’ve not been able to get a clear understanding of such requirements yet.

The proposed discussion topics are good but we’re still not congealing on the use cases that will drive requirements.  From the thread, I’m distilling
The need to discuss the following topics all of which need use cases to substantiate their requirements:

  *   Soft delete
  *   Account status
  *   HR schema action
  *   Roles and entitlements
  *   Pagination: there are several themes on this, but I’m trying to understand the breadth of use cases, as I’m distilling it as a form of either course or fine grain filtering.
  *   Change detection/delta query: what requirements are needed beyond the current capabilities in RFC7644 and the SCIM events draft?
We can tackle each of them individually as we have a 2 hr slot on Friday July 29 12:30-14:30ET  so I propose the following updated agenda:

5min   - agenda bash
5min.  – use case document draft update
10 min – soft delete
10 min  - account status
10min  - HR schema action
10min. – roles and entitlements
10 min.  – SCIM events draft update
10 min.  – change detection/delta query
45 min.  – Pagination

It would be good if Matt and Phil could help lead the discussion on Pagination with a goal of getting clear understanding of the use cases and requirements
To assess a path forward.  As it stands the draft submission deadline has passed.  Matt and Danny (and Janelle) you can submit your drafts during the IETF 114
Sessions but the group will likely not have had time to review the drafts so having you speak to the salient points is good.

We can also do some prep work in the side meetings and adjust agenda accordingly.  I’m leaving 5min as a transition buffer.  We do need to talk about milestones
But without having a clear understanding on what drafts/work we want to undertake, it will be hard to refine it so I’m prioritizing the discussions per this thread.

Thoughts/comments are of course welcomed.

Best, Nancy


From: Phillip Hunt <[e-mail]>
Date: Thursday, July 14, 2022 at 5:21 PM
To: "Matt Peterson (mpeterso)" <[e-mail]>
Cc: Danny Zollner <[e-mail]>, "Nancy (ncamwing)" <[e-mail]>, "[e-mail]" <[e-mail]>, Pamela Dingle <[e-mail]>, "Janelle Allen (janelall)" <[e-mail]>
Subject: Re: IETF 114 preliminary Agenda and call for other agenda requests

Matt

I think we are going to have to agree to disagree on the performance and scalability issues of cursors. :)

The bigger issue is that having two solutions to the same problem is not useful from an interop perspective. Normally when multiple IDs are submitted for the same problem the group adopts one only. I thought that is what we did.

Maybe the group made an error by adopting scim events?  I encourage revisiting that decision one more time if it means one interoperable and implementable standard for all.

Phil


On Jul 13, 2022, at 7:27 PM, Matt Peterson (mpeterso) <[e-mail]> wrote:
Phil,

The cursor pagination draft was written for the common use case where SCIM service is written on top of an underlying database or protocol that uses cursors for result pagination.   This is a very common scenario.  We have implemented SCIM services on top of more than 30 different protocols and databases and nearly half of these use cursor pagination.  With this experience as evidence, I do not agree with your statement that cursor pagination requires the SCIM service to maintain copy of result sets.  It is trivial for SCIM service to store the back-end cursor, or even pass the cursor to the SCIM caller.   On the other hand, attempting to implement index/offset pagination on top of a database/protocol that uses cursors *does* require the SCIM service to maintain result sets (possibly the entire result set).

Also, there is no evidence that use of cursors for pagination in a web API opens a meaningful “denial of service attack vector”.    Many public (and extensively used) Web APIs use cursor pagination (including Facebook, and Twitter).   I can think of almost as many attacks with on index/offset as cursor.  All are easy to mitigate with trivial safeguards that server-side developers will be familiar with.

I do agree with you, that the primary use case for a SCIM client retrieving large (paginated) result sets is for the SCIM client to sync a “local cache” of results (especially for detecting deletes) – and that the SCIM events draft is a better approach for this use case.  However, I remember us testing the assertion in one discussion with the interest group (pre charter) with the suggestion that pagination (index or cursor) is not needed at all.   Not everyone agreed with us 😊

If we can convince the community that pagination of results is not needed then… great.  However, if we can’t, because people need it (or think they need it), then we should provide guidance for implementing *both* index/offset *and* (optionally) cursor pagination.

--
Matt

From: Phillip Hunt <[e-mail]>
Sent: Wednesday, July 13, 2022 11:42 AM
To: Danny Zollner <[e-mail]>
Cc: Matt Peterson (mpeterso) <[e-mail]>; Nancy Cam-Winget (ncamwing) <ncamwing=[e-mail]>; [e-mail]; Pamela Dingle <[e-mail]>; Janelle Allen (janelall) <[e-mail]>
Subject: Re: IETF 114 preliminary Agenda and call for other agenda requests

CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.

I’ve put some comments below. These are discussion items that presenters might want to include in their presentation.

Phillip Hunt
@independentid
[e-mail]<mailto:[e-mail]>





On Jul 12, 2022, at 9:12 PM, Danny Zollner <[e-mail]<mailto:[e-mail]>> wrote:

I’d like to cover the following items related to protocol and schema work:


  *   Cursor-based pagination – via Matt’s draft (that I somehow didn’t realize existed until last month!)
<PH> Much of the use case for this was to support replication and co-ordination of events across domains. How does/doesnot this specification conflict with the SCIM Events WG draft?

I remain ocncerned about the resource costs for service providers and the denial of service attack vector that results.  I have heard that many clients can’t hold a result set in memory. Fair enough. But in this case, would it be better to use a streaming parser OR simply store results direct to local disk for processing?

Why cursors are hard:  If a client has limited resources to store one result set, the cost to a service provider with thousands of clients is multiplied and may not be possible.  In todays systems, causing a SCIM service provider to hold temporary state over data impacts a cluster in many ways.  Today’s server is simply making calls to a database and doing some translation. The server itself is largely stateless. With cursors, the results sets have to be maintained and could effectively mean storing a copy of the entire database. In clusters enviornments (most of them) access to that result set likely needs to be shared.  If not the load-balancer has to maintain “sticky” connections so that each client request goes to the same cluster node.  This could still lead to problems if the client is too slow and the connection is reset. As written now, SCIM servers don’t need to hold “state” across requests.  Implementing cursors would mandate a major re-write for most implementations.

There is also a mistaken assumption that we need paging because SCIM has a result size limitation. That’s not part of SCIM Protocol. The spec simply permits service providers to set limits for things like Javascript UI clients. Presumably for a priviledged provisioning client, they ahve access to the enitre data set. It makes no sense to enforce a 1000 result set limit in these scenarios.

—> The problem is that many implementations make max results a server setting rather than an access control entitlement.  This is not so much something we need to deal with. We may just need to alert industry product managers of why they need to support provisioning clients properly.



  *
  *   Multi-valued attribute pagination – Phil’s draft is a good place to start there. Based on feedback on the mailing list a few months ago, I think there are questions on whether multi-valued attribute filtering is needed, which is also included in Phil’s draft. Splitting multi-valued attribute pagination away from multi-valued attribute filtering if/when it is adopted by the working group may be the best choice.
<PH> The current draft re-uses aspects of SCIM that already supports CMVA filtering. For this reaon, pulling the functionality apart doesn’t make sense. AFAIK there are only a couple of requestors (e.g. Gregg Wilson/Oracle) for this draft.  Without wider support, should we drop the spec?

I wasn’t planning on presenting this. Please let me know if I should.



  *
  *   Discussion on change detection/delta query approaches – I’ll prepare some slides on this and may try to send out an email this week or next week as well, but won’t have a draft for this by IETF 114.
<PH> The current SCIM Protocol spec indicates support for ETAGs specifically in Sec 3.14 of RFC7644 (versioning resources) and can be used in queries. In SCIM’s case an ETAG is a hash of the resource. ETAGs more broadly in HTTP allow a requestor to make queries or changes using HTTP Preconditions (RFC7232).  For example, you can do a GET conditional only on the resource having changed. You can also condity a modify (PUT, PATCH, DELETE) on a resource being unchanged (has a specific etag) or has/has not changed since a specified date.  I suspect pre-conditions are not widely supported in many SCIM implementations.  I have implemented it in i2scim.io<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fi2scim.io%2F&data=05%7C01%7CMatt.Peterson%40oneidentity.com%7C3f2c805b3b684d17db1108da64eeac0c%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637933273500225059%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=PK%2FZtvyWpuE%2FWaZLIMoel4OGAS5m8MeK8WIamkeUA6I%3D&reserved=0>.

I am not sure if this is what you are really after, but you may want to distinguish your requirement from the HTTP standard if different.  :-)

Also, SCIM Events intent is to notify subscribers of changes in real time.  In the simplest message, the new etag is shared (an etag is a hash of the resource).



  *
  *   Soft deletion
  *   Expansion of account status context beyond active true/false – Janelle is working on a draft for this, although I do not know if this will make it in time for IETF 114
  *   My roles/entitlements draft – it has expired but I’ll renew it and would like to put it up for a call for adoption. I don’t know if we need to figure out the contains/containedBy topic that I wrote to the mailing list about last month prior to adoption or if that can be left for the working group to figure out later
  *   HR Schema action plan/next steps
  *   Resetting the milestones that we set last year? (Do we need to worry about this?)

     *   Current plan is that we work on a bunch of the new features in the form of separate extension drafts to SCIM 2.0, and leave RFC 7643 and 7644 alone for now. Once we’re further along in the new features (i.e.: the above + others from the charter) we can determine the final approach of how to make changes to the protocol and schema – that is, do we revise SCIM 2.0 schema and protocol docs while remaining 2.0 but issuing new RFCs (apparently possible), or increment version.
<PH> I haven’t really seen a need to go to SCIM 2.x or 3. Almost all the enhancements can be positioned as extensions to the current SCIM 2 RFCs.

Some of the discussion about extending groups etc suggest much more complex CMVA objects (e.g. verified claims etc). This is interesting. But I would prefer to simply deifne new objects rather than version the protocol first.  There is also nothing saying you can’t have a group represented “virtually” where extended schema is available at one resource path “/<extended>Groups/“ while “/Groups” holds the old schema version of the same groups.

The more complex extension would be to add meta data to attributes like roles.   In some systems, a role binding is actually a policy that has thins like conditions and expiry dates.
—> I guess what I am saying here is this topic deserves a lot of dinner time and hallway discussion over many glasses of wine and beers.

As for editorial clean ups, I am not sure myself what the IETF permits moving from “proposed standard” to “standard” which is another possible route.   In particular, I think many of the examples need to be improved as people depend too much on figures and have gotten mis-leading impressions.  Some of the figures need to better reflect the normative text intent.  I have a concern that many don’t realize that SCIM is technicly a profile of HTTP and that the HTTP RFC do apply. I wonder if in the introduction, we can make this more obvious/clear.

There is one issue with PATCH that we should clarify.  I believe it was wether replacing a value for an attribute value that does not exist MAY be converted to an ADD.  In Postel’s ROBUST design, I would say go for it. There are others that say the transaction that must fail.  The spec is ambiguous on this point and any clarfication would result in a normative change for some.


I think these topics could easily take an hour, if not more. Whatever time is not used by use cases or SCIM Events can be allocated here.

Thanks,

Danny Zollner

From: Matt Peterson (mpeterso) <[e-mail]<mailto:[e-mail]>>
Sent: Monday, July 11, 2022 11:35 PM
To: Nancy Cam-Winget (ncamwing) <ncamwing=[e-mail]<mailto:ncamwing=[e-mail]>>; [e-mail]<mailto:[e-mail]>; Pamela Dingle <[e-mail]<mailto:[e-mail]>>; Janelle Allen (janelall) <[e-mail]<mailto:[e-mail]>>; Danny Zollner <[e-mail]<mailto:[e-mail]>>; Phillip Hunt <[e-mail]<mailto:[e-mail]>>
Subject: [EXTERNAL] RE: IETF 114 preliminary Agenda and call for other agenda requests

Nancy,

There has been recent activity on the mailing list (and also discussion in the SCIM interest group before our chart) that shows continued interest in cursor pagination (draft-peterson-scim-cursor-pagination).

If appropriate for our charter, I would like one more opportunity to check if cursor pagination is interesting to enough people to warrant effort on my part to create a new revision of the draft to address 2 interoperability issues that were raised on the mailing list last month.

--
Matt

From: scim <[e-mail]<mailto:[e-mail]>> On Behalf Of Nancy Cam-Winget (ncamwing)
Sent: Monday, July 11, 2022 6:51 PM
To: [e-mail]<mailto:[e-mail]>; Pamela Dingle <[e-mail]<mailto:[e-mail]>>; Janelle Allen (janelall) <[e-mail]<mailto:[e-mail]>>; Danny Zollner <[e-mail]<mailto:[e-mail]>>; Phillip Hunt <[e-mail]<mailto:[e-mail]>>
Subject: [scim] IETF 114 preliminary Agenda and call for other agenda requests

CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.

Hello SCIMers,

Here is the current proposed agenda and a call for other requests:

5 min: Agenda Bash & Logistics
                SCIM Chairs: Nancy Cam-Winget, Aaron Parecki

TDB min: Use Cases
                Pamela Dingle

TBD min: Protocol & Schema
                Janelle Allen and Danny Zollner

TBD min: SCIM Events
                Phil Hunt
AOB

@Pamela Dingle<mailto:[e-mail]>, @Janelle Allen (janelall)<mailto:[e-mail]>, @Danny Zollner<mailto:[e-mail]> and @Phillip Hunt<mailto:[e-mail]>: can you provide requested times to provide updates on the work above?

If there are other requests for agenda slots, please reply to this email or send a request to [e-mail]<mailto:[e-mail]>

Thanks, Nancy