Re: [scim] Queries on SCIM Cursor-based Pagination Draft

[Jeremy Palenchar <[e-mail]>]

I agree with Danny here.

Previous Cursor is interesting in the case of, say, a Facebook
history/timeline where a user might find an article and then traverse the
history / graph in a forward or reverse direction looking for future OR
past events.

When interacting with a SCIM server, I can't think of a reason to access
the previous page in the cursor.

Does anyone have a use case for this that I am missing?


-jeremy

On Wed, Feb 9, 2022 at 9:31 AM Danny Mayer <[e-mail]> wrote:

> I'm not sure I understand this. Why would you have a previous cursor? In
> order to get to a point where you might have a previous cursor you would
> already have traversed that data set. I expect that it would be up to the
> client to have absorbed that data set and not expect the server to have
> remembered that. I can't imagine how you would be able to jump into the
> middle of a set of data. Maybe I'm not understanding how you got to this
> point. Don't forget too that the data may have changed between requests so
> whatever was on that previous data set may have changed.
>
> Danny
> On 2/8/22 11:37 AM, Matt Peterson (mpeterso) wrote:
>
> Anurada,
>
>
>
> Thank you for the feedback on SCIM Cursor-based Pagination Draft.
>
>
>
> You asked:  “No explanation on backward traversal (traverse to the
> previous page result set)”
>
>
>
> I agree.  The draft could describe more clearly the use of previousCursor
> (when supported).   Would the following rewording of the cursor query
> parameter be clearer?
>
>
>
> cursor – To request the next page, pass the nextCursor value from
>
> the current result page. For the previous page, pass the
>
> previousCursor from the current result page. The cursor parameter
>
> SHOULD be omitted for the first request of a paginated query.
>
>
>
>
>
> You asked: “As per the following paragraph in the introduction, is the
> cursor-based pagination supported only when all resources are fetched from
> the server?”
>
>
>
> Not only is filtering an OPTIONAL capability,for service providers,
>
> it is also a very common for SCIM clients to not to use a query
>
> filter in order to intentionally to retrieve all resources.
>
> Therefore, pagination of results (more so than filtering) is a
>
> primary scalability mechanism for SCIM service providers.
>
>
>
> No, cursor-based pagination should be supported for filtered and
> unfiltered requests.  This paragraph in the introduction was intended to
> counter the opinion that high performance pagination would not be needed if
> clients would constrain queries with filter.   In a future version of this
> draft, I would omit this paragraph as there is less controversy on this
> point than I originally thought.
>
>
>
> --
>
> Matt
>
>
>
> *From:* scim <[e-mail]> <[e-mail]> *On Behalf
> Of * Anuradha Karunarathna
> *Sent:* Monday, February 7, 2022 9:24 AM
> *To:* [e-mail]
> *Subject:* [scim] Queries on SCIM Cursor-based Pagination Draft
>
>
>
> *CAUTION:* This email originated from outside of the organization. Do not
> follow guidance, click links, or open attachments unless you recognize the
> sender and know the content is safe.
>
>
>
> Hi all,
>
>
>
> WSO2 Identity Server(IS) is a SCIM service provider which supports SCIM
> 2.0, and SCIM is the main protocol used in identity management in WSO2-IS.
>
> After seeing performance and functionality improvements that can be
> achieved with the cursor-based pagination we are planning to implement the
> cursor-based pagination draft [1] in WSO2 IS. While reading the draft spec
> we encountered the following concerns. Appreciate your clarification and
> input on these points.
>
>
>
>    1.
>    2.
>    3. No explanation on backward traversal (traverse to the previous page
>    result set)
>    4.
>
>
>
> ·
>
> ·
>
> ·        As per the draft, count and cursor are the URL pagination
> parameters for cursor-based pagination.
>
> ·        (eg: GET /Users?cursor=VZUTiyhEQJ94I&count=10).
>
> ·
>
> ·
>
> ·
>
> ·        Also, the definition of the “cursor” attribute is
>
> ·        *“**The*
>
> ·        *string*
>
> ·        * value from the nextCursor **attribute*
>
> ·        * from the previous result page. A  cursor parameter SHOULD be
> omitted for the first request of the paginated query**”*
>
> ·
>
> ·
>
> ·
>
> ·        If the SCIM service provider supports previousCursor, what
> should be the API request format to
>
> ·         traverse back to the previous page?
>
> ·
>
> o
>
> o
>
> o   Suggestions:
>
> o
>
> 1.
>
> 2.
>
> 3.      Instead of “cursor” query param, support two query params as
> “before” and “after” as similar to
>
> 4.       Facebook API[2]
>
> 5.
>
> 6.
>
> 7.
>
> 8.      Introduce new query param as “cursorDirection”  to indicate
> whether the client needs the before/next
>
> 9.       set of results from the cursor, and change cursor attribute
> definition
>
> 10.
>
> 11.
>
> 12.
>
> 13.   Only if “count” and “cursor” params are supported, add a prefix to
> the cursor such as prev_ and
>
> 14.    next_ that can be parsed before sending to the DB layer[3].
>
> 15.
>
> *Get next page results*: GET
> /Users?cursor=base64encode<next_cursor1>&count=10
>
> *Get previous page results*: GET
> /Users?cursor=base64encode<pre_cursor2>&count=10
>
>
>
> 2.
>
> 3.
>
> 4.      Supporting filtering params and cursor-based pagination params
> together in the API request
>
> 5.
>
>
>
> As per the following paragraph in the introduction, is the cursor-based
> pagination supported only when all resources are fetched from the server?
>
>
>
> “  Not only is filtering an OPTIONAL capability,for service providers,
>
>    it is also a very common for SCIM clients to not to use a query
>
>    filter in order to intentionally to retrieve all resources.
>
>    Therefore, pagination of results (more so than filtering) is a
>
>    primary scalability mechanism for SCIM service providers.
>
> “
>
>
>
> [1]
> https://datatracker.ietf.org/doc/html/draft-peterson-scim-cursor-pagination-00
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-peterson-scim-cursor-pagination-00&data=04%7C01%7Cmatt.peterson%40quest.com%7Cb71a260276744b4a0b2908d9ea564b70%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637798478629490490%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=UkfdM6WJPc5kO%2FYXW8XEAOMK%2BJo7Yu6Tz84JLWYbTKM%3D&reserved=0>
>
>
> [2] https://developers.facebook.com/docs/graph-api/results
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdevelopers.facebook.com%2Fdocs%2Fgraph-api%2Fresults&data=04%7C01%7Cmatt.peterson%40quest.com%7Cb71a260276744b4a0b2908d9ea564b70%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637798478629490490%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=w8VD1YVMi3ZY9ugyo2OwqTILNfcYnR7GPc39oQj5ogo%3D&reserved=0>
>
> [3]
> https://medium.com/swlh/how-to-implement-cursor-pagination-like-a-pro-513140b65f32
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmedium.com%2Fswlh%2Fhow-to-implement-cursor-pagination-like-a-pro-513140b65f32&data=04%7C01%7Cmatt.peterson%40quest.com%7Cb71a260276744b4a0b2908d9ea564b70%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637798478629490490%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=P%2B8mICK6yoZpnWCcgIB%2FwtSN1ntI%2B8%2Ba64YRi4baVYQ%3D&reserved=0>
>
>
>
> Regards,
>
> Anuradha
>
> _______________________________________________
> scim mailing [e-mail]://www.ietf.org/mailman/listinfo/scim
>
> _______________________________________________
> scim mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/scim
>