Re: [scim] Concerns Regarding Cursor-Based Pagination

"Bojitha Piyathilake (intern)" <bojitha@wso2.com> Thu, 09 June 2022 10:28 UTC

Return-Path: <bojitha@wso2.com>
X-Original-To: scim@ietfa.amsl.com
Delivered-To: scim@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0F6A7C15AADA for <scim@ietfa.amsl.com>; Thu, 9 Jun 2022 03:28:54 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.076
X-Spam-Level:
X-Spam-Status: No, score=-2.076 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_BLOCKED=0.001, SPF_HELO_NONE=0.001, T_KAM_HTML_FONT_INVALID=0.01, T_REMOTE_IMAGE=0.01, T_SCC_BODY_TEXT_LINE=-0.01, T_SPF_PERMERROR=0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=wso2.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 01KNtynsEJvH for <scim@ietfa.amsl.com>; Thu, 9 Jun 2022 03:28:50 -0700 (PDT)
Received: from mail-vs1-xe31.google.com (mail-vs1-xe31.google.com [IPv6:2607:f8b0:4864:20::e31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 19F77C15AAD7 for <scim@ietf.org>; Thu, 9 Jun 2022 03:28:50 -0700 (PDT)
Received: by mail-vs1-xe31.google.com with SMTP id k4so22199480vsp.3 for <scim@ietf.org>; Thu, 09 Jun 2022 03:28:50 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wso2.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=4EhH3+1Aru2oTJFVzvzdesoDCj1Lpt+/JR4hhWOA9EI=; b=bxPDHqg9NL/int2u9y3UrKzLZneJfXOHwEaBOhlm5qUIHC2C2idSiSG/qo9hCv5C6R EhZH8LlaqsHnf0bQn0xVI8qajtu68m4lSfi+I6xo/sIbykn0W9rwREIsNGMN0M6wcmQR k8gl/3uBr3h1zph1bz6/QI1mGgryZyuWOM+bo=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=4EhH3+1Aru2oTJFVzvzdesoDCj1Lpt+/JR4hhWOA9EI=; b=ZEIZyF1X0IpLdJaFXHLdy7X14aHNLIOtMVihtCsgYAVN4EoZcurneOi3skjsGi9XB/ 1e8IDeVdIlmhhV3h5d8m5Am1c8nSvFgT8p9Ac+8+/rSEt+k3DdZRMV3ZrtIvL18hmR35 JX4uQOcYrxpYWRPYqpxPb5IF8ySev4hA1cZQch2uzNBzYCYCAUbCs9ebWtABbY9WK/Nn /GOoqG8ZTod+eDjTtdD3OgiixoiTTcml5x250KYCKOmCrj2i5znUhHewyJ8G1VlOOQZF XLi2fkBhRy0QTY0hvW7MYalvivadmnYAVzq0Oo3w9Q9BDJ1YmkTW4rRlDjK7GBAZXOsZ xz/Q==
X-Gm-Message-State: AOAM531MiGa1h+jO0ubQZPz7Jck0wJRoP1S3L0Wh91FObih/nB186c8F mPOo4aNw6h8WQJnqq2N2FEXbJHc99p/48E64ZTmKsL1KoSgfNg==
X-Google-Smtp-Source: ABdhPJxK49yma+IMNZYySpj1EkwbnqOYVLQnrzLUa7uXuqr/NwueZ96n/T4IlMhkPIxy6MuXCUGN1siKznlhNf5NcqM=
X-Received: by 2002:a67:af0b:0:b0:34b:a841:dce9 with SMTP id v11-20020a67af0b000000b0034ba841dce9mr11103835vsl.26.1654770528557; Thu, 09 Jun 2022 03:28:48 -0700 (PDT)
MIME-Version: 1.0
References: <CAKphzcCDyGjXuqjuQY-4KtTjDGL+nYxN6zDnZW0zSM0Y+qQGmA@mail.gmail.com> <SJ1PR19MB6138AE6E460D530FC279688BE1DF9@SJ1PR19MB6138.namprd19.prod.outlook.com>
In-Reply-To: <SJ1PR19MB6138AE6E460D530FC279688BE1DF9@SJ1PR19MB6138.namprd19.prod.outlook.com>
From: "Bojitha Piyathilake (intern)" <bojitha@wso2.com>
Date: Thu, 9 Jun 2022 15:59:17 +0530
Message-ID: <CAKphzcC6AB1kkOgzNhOU1GpM09UHh9Ewq6KqsC60SVY5GAq-3w@mail.gmail.com>
To: "Matt Peterson (mpeterso)" <Matt.Peterson=40oneidentity.com@dmarc.ietf.org>, scim@ietf.org
Content-Type: multipart/alternative; boundary="0000000000008a38fe05e101498a"
Archived-At: <https://mailarchive.ietf.org/arch/msg/scim/RRYYES7WRcDvWwy0bls7WrRFkks>
Subject: Re: [scim] Concerns Regarding Cursor-Based Pagination
X-BeenThere: scim@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: Simple Cloud Identity Management BOF <scim.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/scim>, <mailto:scim-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/scim/>
List-Post: <mailto:scim@ietf.org>
List-Help: <mailto:scim-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/scim>, <mailto:scim-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 09 Jun 2022 10:28:54 -0000

Hi Matt,

Thanks for the clarifications and sorry for the delay. As for question 2,
there are some points I would like to share;


   1.

   The service provider configuration could be updated to reflect which
   pagination type is provided by the SCIM service provider by default. For
   example;


{

      "schemas": [

         "urn:ietf:params:scim:schemas:core:2.0:ServiceProviderConfig"],

      ...

      "pagination": {

         "cursor":true,

         “default”: “cursor”

      },

      ...

   }

If the default type is “cursor”, then we explicitly check for the existence
of startIndex, else pagination will be done with the cursor, and vice versa.


   1.

   The RFC detailing offset pagination is already published and states that
   the offset value is optional. Requiring the existence of startIndex to
   perform offset pagination would mean the RFC needs to be updated. Whereas
   updating the draft to state that the cursor parameter is mandatory to
   perform cursor pagination, would not require a new version of the RFC -
   7644 to be released. However, when performing cursor pagination, there is
   no cursor value for the initial request. Our approach was to pass an empty
   string as the initial cursor.


Best regards,


On Wed, Jun 1, 2022 at 6:27 AM Matt Peterson (mpeterso) <Matt.Peterson=
40oneidentity.com@dmarc.ietf.org> wrote:

> Hi Bojitha,
>
>
>
> It’s important to note that the cursor-based pagination draft is… a draft,
> which means it hasn’t received the full rigor of peer review.  All 3 of
> your questions are reasons enough for a new version of the draft to be
> submitted.
>
>
>
> Question 1
>
> In the Cursor-based Pagination of SCIM Resources draft
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-peterson-scim-cursor-pagination-00%23section-2&data=05%7C01%7Cmatt.peterson%40oneidentity.com%7C15ec43f5057c4cf25ece08da41fdbd2b%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637894855326559034%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=yOfNhvjHZM%2Bc8i3S1TMJGt%2F7m9eEWuzqsosop2EkIOM%3D&reserved=0>
> [1], it hasn’t been mentioned how to handle an unspecified count. Am I
> right to assume this can follow the same, as per RFC - 7644
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Frfc7644%23section-3.4.2.4&data=05%7C01%7Cmatt.peterson%40oneidentity.com%7C15ec43f5057c4cf25ece08da41fdbd2b%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637894855326559034%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=GRrNcrT1wAUUbnSninzHGh5qXb76icnr7l8W9KRKeMg%3D&reserved=0>
> [2] which states, “If unspecified, the maximum number of results is set by
> the service provider”?
>
>
>
> Correct, the draft assumes that if count is omitted, the service provider
> will set the count as specified in RFC 7644.  It would be better if the
> draft explicitly mentioned this.
>
>
>
> Question 2
>
> In a situation where we wish to provide both cursor based pagination and
> index based pagination in the same SCIM service provider, how would you
> suggest the user decide the pagination approach? RFC - 7644
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Frfc7644%23section-3.4.2.4&data=05%7C01%7Cmatt.peterson%40oneidentity.com%7C15ec43f5057c4cf25ece08da41fdbd2b%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637894855326559034%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=GRrNcrT1wAUUbnSninzHGh5qXb76icnr7l8W9KRKeMg%3D&reserved=0>
> [2], states that both the startIndex value and the count are optional
> parameters when making the first offset based request. startIndex will be
> defaulted to 1 and the count will be defaulted to the value specified by
> the service provider. Furthermore, Cursor-based Pagination of SCIM
> Resources draft
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-peterson-scim-cursor-pagination-00%23section-2&data=05%7C01%7Cmatt.peterson%40oneidentity.com%7C15ec43f5057c4cf25ece08da41fdbd2b%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637894855326559034%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=yOfNhvjHZM%2Bc8i3S1TMJGt%2F7m9eEWuzqsosop2EkIOM%3D&reserved=0>
> [1], states that the cursor value of the initial request should be omitted.
>
> eg:
>
> Possible first request for offset pagination: /scim2/Users?count=20 ->
> Offset will be defaulted to 1.
>
> Possible first request for cursor pagination: /scim2/Users?count=20 ->
> Initial cursor is omitted.
>
> The SCIM service provider fails to distinguish the pagination mechanism
> that the user wants for these requests.
>
>
>
> This is an ambiguity that I did not think of when writing the draft.   I
> assumed the ServiceProvider would detect index-based pagination by the
> existence of startIndex in the query.  You are correct, however, that
> startIndex is OPTIONAL.  The draft should be modified to address this
> ambiguity.  Do you have a recommendation?
>
>
>
> Question 3
>
> When querying for resources using a POST request combined with the
> "/.search" path extension, the query params will be contained in the
> request body. How should the cursor attribute be passed? This point is not
> mentioned in the Cursor-based Pagination of SCIM Resources draft [1]
>
>
>
> {
>
>   "schemas": [
>
>     "urn:ietf:params:scim:api:messages:2.0:SearchRequest"
>
>   ],
>
>   "filter": "userName sw A",
>
>   "domain": "PRIMARY",
>
>   "count": 10,
>
>   "cursor": "eyJ2YWx1ZSI6IlNDSU0yQ1"
>
> }
>
>
>
> Correct, this is how the cursor and count should be passed.  Again, the
> draft should be modified to use of cursor pagination with "/.search"
>
>
>
> Once there is feedback on your question #2, I will resubmit a new revision
> of the draft.
>
>
>
> --
>
> Matt Peterson
>


-- 
*Bojitha Piyathilake* | Software Engineering Intern | WSO2 Inc.
(m) +94771715600 | (e) bojitha@wso2.com

<http://wso2.com/signature>
[image: http://wso2.com/signature]
<http://wso2.com/signature>