Re: [scim] Call for adoption of draft-peterson-scim-cursor-pagination/

I do not plan to implement the specification at this time due to limits in underlying NoSQL data stores. 

I have expressed most of these before, but I still have concerns:

* Amount of data transferred which each cycle could incur high data transfer/exfiltration costs and data exposure risks while transferring entire data sets on a repeating basis
* Currency - how often can an entire data set be downloaded using paging and keep systems close enough in sync.  In the past I had customers paging LDAP lasting over 48 hours to accomplish a sync cycle.  
* DoS - Despite some imporovement avoiding indexing, a number of server times will be unable to keep state of an entire data set for extended periods of time (e.g. minutes to hours) unless paging allows unreturned entties to keep changing.  This raises other challenges when using for synchronization.  In my experience this requires storying “copies” of information in some form. This means a relatively few number of clients could exhaust service provider resources very quickly leading to denial of service.  Does paging impose “locking” preventing a service provider from accepting changes while paging in progress?
* Last IETF meeting presenters indicated this spec requires additional “delta” processing capabilioties
* History:  This was tried before with LDAP VLV where replcation remained out of scope.  Many meta-directory implementations focused on bullk transfer sync tech.
* Utility:   There is no way to express information about how information changes or how many changes took place in an interval. 
* Mis-use:  I heard a number of cases where paging was to be used to bootstrap new or failed server nodes.  Is this really a good use?
* Databse mis-use:  Almost all examples from database vendors refer to paging a “small” set of rows for situations like user interface design
* Simplify:  For the planned use, I would expect it would be much simpler for SCIM Service Providers to allow clients to GET large data sets in exceess of the normal server result set limit.  Clients can download 100s of GB of data to disk very quickly these data.  That data then can hold state while the client processes it.
* Mis-read of RFC7644 size limit:  I have heard many say paging is needed to get around result set limits.  The RFCs currently state that paging cannot be used to override result set limits.  The results set is the result set whether you get it in pages or not.   The RFC does NOT require size limits. It only permits a service provider to limit.

Phillip Hunt
phil.hunt@independentid.com

> On Jan 1, 2023, at 4:08 PM, Pamela Dingle <Pamela.Dingle=40microsoft.com@dmarc.ietf.org> wrote:
> 
> Hi Nancy,
> 
> I support the adoption of the draft and am willing to be a reviewer.  
> From: scim <scim-bounces@ietf.org <mailto:scim-bounces@ietf.org>> on behalf of Mike Kiser <mike.kiser=40sailpoint.com@dmarc.ietf.org <mailto:mike.kiser=40sailpoint.com@dmarc.ietf.org>>
> Sent: Saturday, December 31, 2022 1:36 AM
> To: Nancy Cam-Winget (ncamwing) <ncamwing=40cisco.com@dmarc.ietf.org <mailto:ncamwing=40cisco.com@dmarc.ietf.org>>; SCIM WG <scim@ietf.org <mailto:scim@ietf.org>>
> Subject: [EXTERNAL] Re: [scim] Call for adoption of draft-peterson-scim-cursor-pagination/
>  
> Some people who received this message don't often get email from mike.kiser=40sailpoint.com@dmarc.ietf.org <mailto:mike.kiser=40sailpoint.com@dmarc.ietf.org>. Learn why this is important <https://aka.ms/LearnAboutSenderIdentification>	
> Nancy —
>  
> I support the draft and plan to implement it.
>  
> -Mike
>  
> Mike Kiser
> Director of Strategy and Standards
> SailPoint
>  
> From: scim <scim-bounces@ietf.org> on behalf of Nancy Cam-Winget (ncamwing) <ncamwing=40cisco.com@dmarc.ietf.org>
> Date: Wednesday, December 21, 2022 at 11:14
> To: SCIM WG <scim@ietf.org>
> Subject: [scim] Call for adoption of draft-peterson-scim-cursor-pagination/
> 
> Hello SCIMers, We have had discussions both over the email list and in our plenary and virtual sessions on the use of https: //datatracker. ietf. org/doc/draft-peterson-scim-cursor-pagination/ . The group has also adopted https: //datatracker. ietf. org/doc/draft-ietf-scim-events/
> ZjQcmQRYFpfptBannerStart
> This Message Is From an External Sender
> This message came from outside your organization.
> ZjQcmQRYFpfptBannerEnd
> Hello SCIMers,
>  
> We have had discussions both over the email list and in our plenary and virtual sessions on the use of
> https://datatracker.ietf.org/doc/draft-peterson-scim-cursor-pagination/ <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-peterson-scim-cursor-pagination%2F__%3B!!MsNKLpFGsw!K8MPsjQxR3mfU34ULpWq9r_41ejdJhgzYY7VBb53zg2TP4qaGHLEfSSjCLY3_enw_aJ4H9j0tP2Yi2iiL9N_syG8WvxEQr3KOg%24&data=05%7C01%7Cpamela.dingle%40microsoft.com%7Cfb724dd3f6d04d061b3408daea7b9580%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638080113926048537%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=uvS8yvPpf2NJy5yJUY%2BPba4NfdMXAAWfLjk0piaNdXw%3D&reserved=0>   .  The group has also adopted
> https://datatracker.ietf.org/doc/draft-ietf-scim-events/ <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-ietf-scim-events%2F__%3B!!MsNKLpFGsw!K8MPsjQxR3mfU34ULpWq9r_41ejdJhgzYY7VBb53zg2TP4qaGHLEfSSjCLY3_enw_aJ4H9j0tP2Yi2iiL9N_syG8WvwBkegq8A%24&data=05%7C01%7Cpamela.dingle%40microsoft.com%7Cfb724dd3f6d04d061b3408daea7b9580%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638080113926048537%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C&sdata=dJVXrW4zBB68KEU7yxkv52tJa6I1Hd1GP7xMr4ikNM4%3D&reserved=0>  and there was broad acceptance that both may
> be needed to address different use cases.  These scenarios and use cases should be captured at least in the
> use cases draft.
>  
> To that extent, we are doing an adoption call for draft-peterson-scim-cursor-pagination/
>  
> Please respond to this thread on the following:
>  
>   1.  You have read the draft and believe it is ready to be adopted by the working group. Any other feedback on the content of the draft is welcomed too.
>   2.  You are willing to be a reviewer of the document
>   3.  You support the draft and plan to implement
>   4.  You support the draft but have no time or plans to implement now, but can provide feedback
>   5.  You do not support the working group adopt the draft and provide rationale and feedback for why
>  
> Please provide your feedback by January 16 (extending given the holiday break).
>  
> Happy Holidays!
>  
> Best, Nancy 
> (on behalf of both SCIM chairs)
>  
>  
> _______________________________________________
> scim mailing list
> scim@ietf.org
> https://www.ietf.org/mailman/listinfo/scim