Re: [scim] Queries on SCIM Cursor-based Pagination Draft

[Danny Mayer <[e-mail]>]

I'm not sure I understand this. Why would you have a previous cursor? In 
order to get to a point where you might have a previous cursor you would 
already have traversed that data set. I expect that it would be up to 
the client to have absorbed that data set and not expect the server to 
have remembered that. I can't imagine how you would be able to jump into 
the middle of a set of data. Maybe I'm not understanding how you got to 
this point. Don't forget too that the data may have changed between 
requests so whatever was on that previous data set may have changed.

Danny

On 2/8/22 11:37 AM, Matt Peterson (mpeterso) wrote:
>
> Anurada,
>
> Thank you for the feedback on SCIM Cursor-based Pagination Draft.
>
> You asked:  “No explanation on backward traversal (traverse to the 
> previous page result set)”
>
> I agree.  The draft could describe more clearly the use of 
> previousCursor (when supported).   Would the following rewording of 
> the cursor query parameter be clearer?
>
> cursor – To request the next page, pass the nextCursor value from
>
> the current result page. For the previous page, pass the
>
> previousCursor from the current result page. The cursor parameter
>
> SHOULD be omitted for the first request of a paginated query.
>
> You asked: “As per the following paragraph in the introduction, is the 
> cursor-based pagination supported only when all resources are fetched 
> from the server?”
>
> Not only is filtering an OPTIONAL capability,for service providers,
>
> it is also a very common for SCIM clients to not to use a query
>
> filter in order to intentionally to retrieve all resources.
>
> Therefore, pagination of results (more so than filtering) is a
>
> primary scalability mechanism for SCIM service providers.
>
> No, cursor-based pagination should be supported for filtered and 
> unfiltered requests.  This paragraph in the introduction was intended 
> to counter the opinion that high performance pagination would not be 
> needed if clients would constrain queries with filter.   In a future 
> version of this draft, I would omit this paragraph as there is less 
> controversy on this point than I originally thought.
>
> --
>
> Matt
>
> *From:* scim <[e-mail]> *On Behalf Of * Anuradha Karunarathna
> *Sent:* Monday, February 7, 2022 9:24 AM
> *To:* [e-mail]
> *Subject:* [scim] Queries on SCIM Cursor-based Pagination Draft
>
> *CAUTION:*This email originated from outside of the organization. Do 
> not follow guidance, click links, or open attachments unless you 
> recognize the sender and know the content is safe.
>
> Hi all,
>
> WSO2 Identity Server(IS) is a SCIM service provider which supports 
> SCIM 2.0, and SCIM is the main protocol used in identity management in 
> WSO2-IS.
>
> After seeing performance and functionality improvements that can be 
> achieved with the cursor-based pagination we are planning to implement 
> the cursor-based pagination draft [1] in WSO2 IS. While reading the 
> draft spec we encountered the following concerns. Appreciate your 
> clarification and input on these points.
>
> 1.
> 2.
>  3. No explanation on backward traversal (traverse to the previous
>     page result set)
> 4.
>
> ·
>
> ·
>
> ·As per the draft, count and cursor are the URL pagination parameters 
> for cursor-based pagination.
>
> ·(eg: GET /Users?cursor=VZUTiyhEQJ94I&count=10).
>
> ·
>
> ·
>
> ·
>
> ·Also, the definition of the “cursor” attribute is
>
> ·/“//The/
>
> ·*/string/*
>
> ·*/ value from the nextCursor /*/attribute/
>
> ·/ from the previous result page. A  cursor parameter SHOULD be  
> omitted for the first request of the paginated query//”/
>
> ·
>
> ·
>
> ·
>
> ·If the SCIM service provider supports previousCursor, what should be 
> the API request format to
>
> · traverse back to the previous page?
>
> ·
>
> o
>
> o
>
> oSuggestions:
>
> o
>
> 1.
>
> 2.
>
> 3.Instead of “cursor” query param, support two query params as 
> “before” and “after” as similar to
>
> 4. Facebook API[2]
>
> 5.
>
> 6.
>
> 7.
>
> 8.Introduce new query param as “cursorDirection”  to indicate whether 
> the client needs the before/next
>
> 9. set of results from the cursor, and change cursor attribute definition
>
> 10.
>
> 11.
>
> 12.
>
> 13.Only if “count” and “cursor” params are supported, add a prefix to 
> the cursor such as prev_ and
>
> 14. next_ that can be parsed before sending to the DB layer[3].
>
> 15.
>
> *Get next page results*: GET 
> /Users?cursor=base64encode<next_cursor1>&count=10
>
> *Get previous page results*: GET 
> /Users?cursor=base64encode<pre_cursor2>&count=10
>
> 2.
>
> 3.
>
> 4.Supporting filtering params and cursor-based pagination params 
> together in the API request
>
> 5.
>
> As per the following paragraph in the introduction, is the 
> cursor-based pagination supported only when all resources are fetched 
> from the server?
>
> “Not only is filtering an OPTIONAL capability,for service providers,
>
>    it is also a very common for SCIM clients to not to use a query
>
>    filter in order to intentionally to retrieve all resources.
>
>    Therefore, pagination of results (more so than filtering) is a
>
>    primary scalability mechanism for SCIM service providers.
>
> “
>
> [1] 
> https://datatracker.ietf.org/doc/html/draft-peterson-scim-cursor-pagination-00 
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-peterson-scim-cursor-pagination-00&data=04%7C01%7Cmatt.peterson%40quest.com%7Cb71a260276744b4a0b2908d9ea564b70%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637798478629490490%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=UkfdM6WJPc5kO%2FYXW8XEAOMK%2BJo7Yu6Tz84JLWYbTKM%3D&reserved=0>
>
> [2] https://developers.facebook.com/docs/graph-api/results 
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdevelopers.facebook.com%2Fdocs%2Fgraph-api%2Fresults&data=04%7C01%7Cmatt.peterson%40quest.com%7Cb71a260276744b4a0b2908d9ea564b70%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637798478629490490%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=w8VD1YVMi3ZY9ugyo2OwqTILNfcYnR7GPc39oQj5ogo%3D&reserved=0>
>
> [3] 
> https://medium.com/swlh/how-to-implement-cursor-pagination-like-a-pro-513140b65f32 
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmedium.com%2Fswlh%2Fhow-to-implement-cursor-pagination-like-a-pro-513140b65f32&data=04%7C01%7Cmatt.peterson%40quest.com%7Cb71a260276744b4a0b2908d9ea564b70%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637798478629490490%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=P%2B8mICK6yoZpnWCcgIB%2FwtSN1ntI%2B8%2Ba64YRi4baVYQ%3D&reserved=0>
>
>
>
> Regards,
>
> Anuradha
>
>
> _______________________________________________
> scim mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/scim