Re: [scim] Queries on SCIM Cursor-based Pagination Draft

[Anuradha Karunarathna <[e-mail]>]

Appreciate all inputs.

The use case for  “previousCursor” is:
The UI client which lists out users/groups uses cursor-based pagination as
its pagination mechanism. If the UI client supports infinite scrolling
there should be a way to navigate to the previous and next set of results.

Regards,
Anuradha

On Wed, Feb 9, 2022 at 11:51 PM Matt Peterson (mpeterso) <
[e-mail]> wrote:

> Danny,
>
>
>
> I also can not think of a good use case for “previousCursor”, but because
> others had interest, I included it in the draft as OPTIONAL
>
>
>
> --
>
> Matt
>
>
>
> *From:* Danny Mayer <[e-mail]>
> *Sent:* Wednesday, February 9, 2022 10:31 AM
> *To:* Matt Peterson (mpeterso) <[e-mail]>; Anuradha
> Karunarathna <[e-mail]>; [e-mail]
> *Subject:* Re: [scim] Queries on SCIM Cursor-based Pagination Draft
>
>
>
> *CAUTION:* This email originated from outside of the organization. Do not
> follow guidance, click links, or open attachments unless you recognize the
> sender and know the content is safe.
>
>
>
> I'm not sure I understand this. Why would you have a previous cursor? In
> order to get to a point where you might have a previous cursor you would
> already have traversed that data set. I expect that it would be up to the
> client to have absorbed that data set and not expect the server to have
> remembered that. I can't imagine how you would be able to jump into the
> middle of a set of data. Maybe I'm not understanding how you got to this
> point. Don't forget too that the data may have changed between requests so
> whatever was on that previous data set may have changed.
>
> Danny
>
> On 2/8/22 11:37 AM, Matt Peterson (mpeterso) wrote:
>
> Anurada,
>
>
>
> Thank you for the feedback on SCIM Cursor-based Pagination Draft.
>
>
>
> You asked:  “No explanation on backward traversal (traverse to the
> previous page result set)”
>
>
>
> I agree.  The draft could describe more clearly the use of previousCursor
> (when supported).   Would the following rewording of the cursor query
> parameter be clearer?
>
>
>
> cursor – To request the next page, pass the nextCursor value from
>
> the current result page. For the previous page, pass the
>
> previousCursor from the current result page. The cursor parameter
>
> SHOULD be omitted for the first request of a paginated query.
>
>
>
>
>
> You asked: “As per the following paragraph in the introduction, is the
> cursor-based pagination supported only when all resources are fetched from
> the server?”
>
>
>
> Not only is filtering an OPTIONAL capability,for service providers,
>
> it is also a very common for SCIM clients to not to use a query
>
> filter in order to intentionally to retrieve all resources.
>
> Therefore, pagination of results (more so than filtering) is a
>
> primary scalability mechanism for SCIM service providers.
>
>
>
> No, cursor-based pagination should be supported for filtered and
> unfiltered requests.  This paragraph in the introduction was intended to
> counter the opinion that high performance pagination would not be needed if
> clients would constrain queries with filter.   In a future version of this
> draft, I would omit this paragraph as there is less controversy on this
> point than I originally thought.
>
>
>
> --
>
> Matt
>
>
>
> *From:* scim <[e-mail]> <[e-mail]> *On Behalf
> Of *Anuradha Karunarathna
> *Sent:* Monday, February 7, 2022 9:24 AM
> *To:* [e-mail]
> *Subject:* [scim] Queries on SCIM Cursor-based Pagination Draft
>
>
>
> *CAUTION:* This email originated from outside of the organization. Do not
> follow guidance, click links, or open attachments unless you recognize the
> sender and know the content is safe.
>
>
>
> Hi all,
>
>
>
> WSO2 Identity Server(IS) is a SCIM service provider which supports SCIM
> 2.0, and SCIM is the main protocol used in identity management in WSO2-IS.
>
> After seeing performance and functionality improvements that can be
> achieved with the cursor-based pagination we are planning to implement the
> cursor-based pagination draft [1] in WSO2 IS. While reading the draft spec
> we encountered the following concerns. Appreciate your clarification and
> input on these points.
>
>
>
>    1.
>    2.
>    3. No explanation on backward traversal (traverse to the previous page
>    result set)
>    4.
>
>
>
> 1.
>
> 2.
>
> 3.      As per the draft, count and cursor are the URL pagination
> parameters for cursor-based pagination.
>
> 4.      (eg: GET /Users?cursor=VZUTiyhEQJ94I&count=10).
>
> 5.
>
> 6.
>
> 7.
>
> 8.      Also, the definition of the “cursor” attribute is
>
> 9.      *“**The*
>
> 10.   *string*
>
> 11.   * value from the nextCursor **attribute*
>
> 12.   * from the previous result page. A  cursor parameter SHOULD be
> omitted for the first request of the paginated query**”*
>
> 13.
>
> 14.
>
> 15.
>
> 16.   If the SCIM service provider supports previousCursor, what should
> be the API request format to
>
> 17.    traverse back to the previous page?
>
> 18.
>
> 1.
>
> 2.
>
> 3.      Suggestions:
>
> 4.
>
> 1.
>
> 2.
>
> 3.      Instead of “cursor” query param, support two query params as
> “before” and “after” as similar to
>
> 4.       Facebook API[2]
>
> 5.
>
> 6.
>
> 7.
>
> 8.      Introduce new query param as “cursorDirection”  to indicate
> whether the client needs the before/next
>
> 9.       set of results from the cursor, and change cursor attribute
> definition
>
> 10.
>
> 11.
>
> 12.
>
> 13.   Only if “count” and “cursor” params are supported, add a prefix to
> the cursor such as prev_ and
>
> 14.    next_ that can be parsed before sending to the DB layer[3].
>
> 15.
>
> *Get next page results*: GET
> /Users?cursor=base64encode<next_cursor1>&count=10
>
> *Get previous page results*: GET
> /Users?cursor=base64encode<pre_cursor2>&count=10
>
>
>
>    1.
>    2.
>    3. Supporting filtering params and cursor-based pagination params
>    together in the API request
>    4.
>
>
>
> As per the following paragraph in the introduction, is the cursor-based
> pagination supported only when all resources are fetched from the server?
>
>
>
> “  Not only is filtering an OPTIONAL capability,for service providers,
>
>    it is also a very common for SCIM clients to not to use a query
>
>    filter in order to intentionally to retrieve all resources.
>
>    Therefore, pagination of results (more so than filtering) is a
>
>    primary scalability mechanism for SCIM service providers.
>
> “
>
>
>
> [1]
> https://datatracker.ietf.org/doc/html/draft-peterson-scim-cursor-pagination-00
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-peterson-scim-cursor-pagination-00&data=04%7C01%7CMatt.Peterson%40oneidentity.com%7Cc9c495422ae544d06a2908d9ebf2083e%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637800247033454244%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=J6ZmPYq2NI630ISNdfiwP7svVH0atbiuxNXN2XMQ%2FlI%3D&reserved=0>
>
>
> [2] https://developers.facebook.com/docs/graph-api/results
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdevelopers.facebook.com%2Fdocs%2Fgraph-api%2Fresults&data=04%7C01%7CMatt.Peterson%40oneidentity.com%7Cc9c495422ae544d06a2908d9ebf2083e%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637800247033454244%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ko1N0kM%2BnYOVJVBybULOOGc6PMJDBaEvfoXJRxqaapQ%3D&reserved=0>
>
> [3]
> https://medium.com/swlh/how-to-implement-cursor-pagination-like-a-pro-513140b65f32
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmedium.com%2Fswlh%2Fhow-to-implement-cursor-pagination-like-a-pro-513140b65f32&data=04%7C01%7CMatt.Peterson%40oneidentity.com%7Cc9c495422ae544d06a2908d9ebf2083e%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637800247033454244%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2F%2BL19mJQL3286dbfC0GDaCRQCn%2F5Jx3fm%2FemBgECVhg%3D&reserved=0>
>
>
>
>
> Regards,
>
> Anuradha
>
>
>
> _______________________________________________
>
> scim mailing list
>
> [e-mail]
>
> https://www.ietf.org/mailman/listinfo/scim <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fscim&data=04%7C01%7CMatt.Peterson%40oneidentity.com%7Cc9c495422ae544d06a2908d9ebf2083e%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637800247033454244%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=eCMSMqFE%2Bti6S42XlnCYfYhWX2Dyy%2FCOZ3nDzKIIP60%3D&reserved=0>
>
>