[scim] Queries on SCIM Cursor-based Pagination Draft

[Anuradha Karunarathna <[e-mail]>]

Hi all,

WSO2 Identity Server(IS) is a SCIM service provider which supports SCIM
2.0, and SCIM is the main protocol used in identity management in WSO2-IS.

After seeing performance and functionality improvements that can be
achieved with the cursor-based pagination we are planning to implement the
cursor-based pagination draft [1] in WSO2 IS. While reading the draft spec
we encountered the following concerns. Appreciate your clarification and
input on these points.


   1.

   No explanation on backward traversal (traverse to the previous page
   result set)



   -

   As per the draft, count and cursor are the URL pagination parameters for
   cursor-based pagination. (eg: GET /Users?cursor=VZUTiyhEQJ94I&count=10).
   -

   Also, the definition of the “cursor” attribute is “The string value from
   the nextCursor attribute from the previous result page. A  cursor
   parameter SHOULD be  omitted for the first request of the paginated query
   ”
   -

   If the SCIM service provider supports previousCursor, what should be the
   API request format to traverse back to the previous page?
   -

      Suggestions:


   1.

   Instead of “cursor” query param, support two query params as “before”
   and “after” as similar to Facebook API[2]
   2.

   Introduce new query param as “cursorDirection”  to indicate whether the
   client needs the before/next set of results from the cursor, and change
   cursor attribute definition
   3.

   Only if “count” and “cursor” params are supported, add a prefix to the
   cursor such as prev_ and next_ that can be parsed before sending to the DB
   layer[3].

Get next page results: GET /Users?cursor=base64encode<next_cursor1>&count=10

Get previous page results: GET
/Users?cursor=base64encode<pre_cursor2>&count=10



   1.

   Supporting filtering params and cursor-based pagination params together
   in the API request


As per the following paragraph in the introduction, is the cursor-based
pagination supported only when all resources are fetched from the server?

“  Not only is filtering an OPTIONAL capability,for service providers,

   it is also a very common for SCIM clients to not to use a query

   filter in order to intentionally to retrieve all resources.

   Therefore, pagination of results (more so than filtering) is a

   primary scalability mechanism for SCIM service providers.

“

[1]
https://datatracker.ietf.org/doc/html/draft-peterson-scim-cursor-pagination-00


[2] https://developers.facebook.com/docs/graph-api/results
[3]
https://medium.com/swlh/how-to-implement-cursor-pagination-like-a-pro-513140b65f32

Regards,
Anuradha