Re: [scim] Concerns Regarding Cursor-Based Pagination

"Matt Peterson (mpeterso)" <Matt.Peterson@oneidentity.com> Wed, 01 June 2022 00:57 UTC

Return-Path: <Matt.Peterson@oneidentity.com>
X-Original-To: scim@ietfa.amsl.com
Delivered-To: scim@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B8823C15C016 for <scim@ietfa.amsl.com>; Tue, 31 May 2022 17:57:49 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -7.11
X-Spam-Level:
X-Spam-Status: No, score=-7.11 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=oneidentity.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mVoYzsApWgKV for <scim@ietfa.amsl.com>; Tue, 31 May 2022 17:57:45 -0700 (PDT)
Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2098.outbound.protection.outlook.com [40.107.237.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BDE10C15790C for <scim@ietf.org>; Tue, 31 May 2022 17:57:45 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=I/WjflGbIIDbEn52CZOOY6zvAIZg8agZYaD3LHJVn3aJ39Wgku0+cjC0Mt5KGQEw16SbW1coIuDlcXA3iLNRlJeEGwayCXsm5ap/sWOCGdGsjA/bn4JNviz38+x4cUCi9t41O6FJpncp6YmA167OsskIINxacfSTucjOXNWdBLo0zsA3+q/c4HCfQ9y0QXy4jfanjTpGMBiWsnLU6xL5ZdGMY5XVt0+/YfLDhC1jrBiFQeWTD0X9JFczUSovup5HCY9/jjDl8xNkssrV1++yP3lPIxWjtYIFUHcDUj7IWQApFLjk5RIy0J1Tar6PWAOeDq1lCzewQhCJ2248XO936g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rWPDZtWViEXjbKN/jCnYRhpkxlUJ8f4e3NLRL2HEBPo=; b=Pk8ujgzsd4nLscl6pnm3EisUXfRtX1e3bth1eHdpsqYoy+d5Lb+447IfwXgKzoYoaX69qg08e5FGToMfE5X9FFUdEH7uuSAN6wn+ShQATvJs9NI5cvIXBRHKNWCDDJJFaHRqGvWB2x+alCoqCTV7pQBpoNjTJxvcZXETKt0h15ZkWJabQjZbn4DLTajeJUttmoKZ0Lbjtqi1ktXW0xTzUyjQbfRdDLF1iUC+oh2QKaDWTDb8K2EChfNIrQ7ooiIGYPcbaWXUikUdFbqeD5LM9eVZx7oeTtuy9x+DwARET7ZmumfSnbjV3sTV+bZRZ+qnXAkeaQK7Zf9m+1dOzHq+fg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oneidentity.com; dmarc=pass action=none header.from=oneidentity.com; dkim=pass header.d=oneidentity.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oneidentity.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rWPDZtWViEXjbKN/jCnYRhpkxlUJ8f4e3NLRL2HEBPo=; b=KlPamVus671CjTvts1bYh1cC5dSqgvvIJOvLEQyCl8iKOO2RN+Mevi+erdskNbFThdTqAQu+oekR1Rcyz0kHluJO8Kes4w6mtcmYtPZV6hgZ02hFawBdIhmOTIGLYlsrzn5wKiyoIT6te0Fr3GU+/0BJlEx8CK4LoOIuF8VYOTGymxPVJHEGfCEJeMWslKJoEnd1bmUsnqFAEZX/Qu6fhxpKdbhFxhcbDVlHhbjtVBVaXFkDzPEWvUUhgGwDpU0+HXDOBVfYxj4dmMOpWUhA2pZDAhNFMK6N9SbcPNKDAAnzgDRG4BUGjVh2Do2zrnbxPeby8gri+WWU1EhnPEDYNw==
Received: from SJ1PR19MB6138.namprd19.prod.outlook.com (2603:10b6:a03:48b::14) by DS7PR19MB4407.namprd19.prod.outlook.com (2603:10b6:5:2c0::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5314.12; Wed, 1 Jun 2022 00:57:41 +0000
Received: from SJ1PR19MB6138.namprd19.prod.outlook.com ([fe80::7c1f:11f7:8821:d1d2]) by SJ1PR19MB6138.namprd19.prod.outlook.com ([fe80::7c1f:11f7:8821:d1d2%9]) with mapi id 15.20.5293.019; Wed, 1 Jun 2022 00:57:41 +0000
From: "Matt Peterson (mpeterso)" <Matt.Peterson@oneidentity.com>
To: "Bojitha Piyathilake (intern)" <bojitha=40wso2.com@dmarc.ietf.org>, "scim@ietf.org" <scim@ietf.org>
Thread-Topic: [scim] Concerns Regarding Cursor-Based Pagination
Thread-Index: AQHYc+adWfIUrtgBUkO8gojz6pthkK05rNXw
Date: Wed, 1 Jun 2022 00:57:41 +0000
Message-ID: <SJ1PR19MB6138AE6E460D530FC279688BE1DF9@SJ1PR19MB6138.namprd19.prod.outlook.com>
References: <CAKphzcCDyGjXuqjuQY-4KtTjDGL+nYxN6zDnZW0zSM0Y+qQGmA@mail.gmail.com>
In-Reply-To: <CAKphzcCDyGjXuqjuQY-4KtTjDGL+nYxN6zDnZW0zSM0Y+qQGmA@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=oneidentity.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: cefff8c1-f5cc-436a-2431-08da4369bafd
x-ms-traffictypediagnostic: DS7PR19MB4407:EE_
x-microsoft-antispam-prvs: <DS7PR19MB4407FFCCE7873BCD321AF555E1DF9@DS7PR19MB4407.namprd19.prod.outlook.com>
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: nTCOhvS+pkVPG6P4yNmj/QF6AwnOj5hNvXdZd3iAKSRlkuY0Qx+CDO4fVcvEKJU8K3/jULa9iUQhXrQuhT+c5dmc0sjZInn027bVoP2sMJbw4yTGAh6uYreNUhWwHVX6TP9WFXBTM20KtpRZYdU39pRY52Wc0c2a5ChXYtl17/Ree/chnUhy3fz9pKd2WzLFvRBuQg3AU1tHfPmbvN+dOuqcvzf3oAOG2wUW1vXBA6VZ5nLzQvPx51x4oLVdfh3gbHZ8d3D5cIDy45nX6kgQufMEveAQxT43lV7BvCrjgxTwIEsSclEw45a/37ImsDnrmy6WXCRUSZQa3PhA/eUONASIh5DWM1pAaE+ImsToenFPf6sm9//3idpYbHdqKWeGWHO1FdiNl7Q3IRlvN80jdO6+qptzP/X09+JlfuFlyPbrTffayhcXI8DMnqeVm19zvEZtU9KIAR8SqZR7ERxkaySInIGxDRd5tFEiY7/pLKmxCeAVpRIRy+GEpgyPr67Yj/zzIiMsULSYB07N0XdOXJ2SP9InJ0smpHHo+By47EHIZ2kP6A02jW/h7Hu8duHjiVZPNyPvcAmRAkfjk1T+lkgatvKVzqJGWXfsyPDzZFj837JUZdThpMwadivgI1yRzxFnT8BMZWlakj1gbYRi/toOZAh4/ZQs2TfqCfpWjahMfEgGoo4ucoBeTS272wSnGMbZixNFV13eyG3vKo4yMfz25W3mW+4jWl7Zui4r4rD/WHx0yYMy2WQd5W82RBsHh4or3zWXkQPXxPvZHFjrniRdBRmIHZm2lBLq/WPw8/Y=
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ1PR19MB6138.namprd19.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(7696005)(6506007)(86362001)(38070700005)(2906002)(66556008)(110136005)(8676002)(38100700002)(64756008)(66476007)(66946007)(316002)(66446008)(76116006)(166002)(71200400001)(55016003)(83380400001)(186003)(9686003)(33656002)(5660300002)(122000001)(8936002)(508600001)(9326002)(52536014); DIR:OUT; SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?tJnrSKW6shFePdoYbqs45oL1W9Fd5tI5L7BW5hvNhax+89wl6xlKH7Y9HWxF?= =?us-ascii?Q?+3MG2HkvWhWD0h36ImbxQuXvPLtT9bmoDJJ5i9qYptyfLkGrCRwHj3HSAaLB?= =?us-ascii?Q?L/KoKlW7+ZXf6XtDbMe0H7CUgAb/HIGnt4ewRSm29Yemfii4tWpIjPkMODHo?= =?us-ascii?Q?NHEpBzE00xyzyF6nbDNSkRGMjvryXx5ZrPThHQSB7B5oK6uw6WUzSqLQ9Owc?= =?us-ascii?Q?UGggutEZlz3A4NZlQQC8SrMJmlJqCeGmrM8feYZ+pYjFdrlJTB54IgkUYhkG?= =?us-ascii?Q?4CdHyljcGnfSN3n6qXNtLpmizKwibSOXFr4hDnqzRWrSk3IAN5tH3sqg7e6a?= =?us-ascii?Q?7MRg12j66QX+Zaeg2yVHGdp/rrvoMcilvB1qf31xDD9WyQmHPSGznVzm6Fff?= =?us-ascii?Q?6AE72zLLnmpqD1a3dTgxFAFAON8nGOedRZSVBBSAm/G8y4qAyqIIcr4T8cbp?= =?us-ascii?Q?vT+JRTyroIBe3jxgpsvVPUmdBQxBZKECDVcgThSar0gX8fin1cVxdE/mTxFp?= =?us-ascii?Q?wG+5C6a9Q5AIl6kNbXQj3esckWq32phXUUWlm1nugsNsxPY4xxDjGy0ZxCgc?= =?us-ascii?Q?jKgRtVeuKxBh6dfiq35CQbFHF8RlRObZ8c/njc9+ynFIOR/orHMoLLMwAMQD?= =?us-ascii?Q?aQRAxpEpY4IwnPcdHfMS3ANUg0lVaeRoio8mh0aiaKbKDx1OhCCBMtaAEcSB?= =?us-ascii?Q?CNvPG2/7/TxqaVPFT9IUTwAlGVnStIx9U5T9srlL9o/3YSXJLx0+RZ1i2m2i?= =?us-ascii?Q?huonf2Xqye6yzwAqh5U+2yPafGSqXRNVwb63dStWihBgz46tcls2I+uS+p/I?= =?us-ascii?Q?hci3BfhkHd4cjb9cjLq4nfDy0x9zNMCGuZL+hbavPynXtN+5fF/oE2yS0TKc?= =?us-ascii?Q?8XirsJzk4QDxrolaIU3R5ilrbTxdaZiy0HtJEkCLY/5WdmPBRigPF1kzNG4g?= =?us-ascii?Q?W0pDA5UOYNRg34Gfne1czwgSXWchgmMGSHYoK3Ec0p0/D3n9CRytIVTr1EBS?= =?us-ascii?Q?ZCRrnOfDSTxuZ126DvPvlIUAmjziXRovcbZ6oEBcV5Mg2IgS/XMA91OgyXy2?= =?us-ascii?Q?7VuvFS/hM9kBq/WlCU5R2dlrzGh+OEtdpnVRfOh4ekCjWmxZx8oImRocgHb8?= =?us-ascii?Q?DD9pfCGBg45b2yNU9zKt4fpDaurk8BJiOuiC2Aqq5ddeKQ8gI1Ln5AkMkMJj?= =?us-ascii?Q?tYpt0dMHmWfvuZNNr3I4ksERVWrQ14tdT71bXC/XgjSpBo2qfD0vUL2ENw4L?= =?us-ascii?Q?EXT72lJOfHPjnBVvNdfPgQeC7cPQ+V/HUOG1IbmVboMFKLGi9WHQneUXpLlZ?= =?us-ascii?Q?YLTQY/sGFU76kGovI59Nzgrr1usVJRdIAsGE9iTtyN7v8u0mviXSoE1sukCy?= =?us-ascii?Q?SX5EJ8yGKaAqcmH96aUan7AboTppfXAG9nv5I8XXm55hvikZfD9nzPqUIo83?= =?us-ascii?Q?mXbSQABJsftprk2K8c/lcimfOalNyfL7VMymV1+vbjPnUuCUzGWvRtb+tYp/?= =?us-ascii?Q?EaLpBZD7ZlVgF+3eLHI8TQR1J2APBqATsHLdMQt0J93oWZqHa/wn/Wa+zc+O?= =?us-ascii?Q?eJPc5Y3eWpa7NEqnjXNVI+2DM6ID2MMHRm3Hw7Lab9OjhC3zpE6SpiFq04K5?= =?us-ascii?Q?U/+O0AybTCCx6NVUN91vBTjpQJ4o6HBecrrQL4KKqRtV4gUadlvmpuFBSp8m?= =?us-ascii?Q?bKwskvVZ1e89KalAAsPuVBU+2O8i7s+myFH5n4nnBaKfxIZnn+XoLAPlFqQD?= =?us-ascii?Q?J8h1lHcrbyLkAFUn4nryySnKYEkRW9g=3D?=
Content-Type: multipart/alternative; boundary="_000_SJ1PR19MB6138AE6E460D530FC279688BE1DF9SJ1PR19MB6138namp_"
MIME-Version: 1.0
X-OriginatorOrg: oneidentity.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SJ1PR19MB6138.namprd19.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: cefff8c1-f5cc-436a-2431-08da4369bafd
X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Jun 2022 00:57:41.3813 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 91c369b5-1c9e-439c-989c-1867ec606603
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: RiGpQ2yVwyPRE/KqcIC9ozOVu4Kv976NPxbyW93fYxrxpNSNeJIkWXcBNfGaa3Kzc9kW2cRpjd/1z4CQlybNOQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR19MB4407
Archived-At: <https://mailarchive.ietf.org/arch/msg/scim/m7-ss0ylpqrWpwChj6l9WXAuzU8>
Subject: Re: [scim] Concerns Regarding Cursor-Based Pagination
X-BeenThere: scim@ietf.org
X-Mailman-Version: 2.1.34
Precedence: list
List-Id: Simple Cloud Identity Management BOF <scim.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/scim>, <mailto:scim-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/scim/>
List-Post: <mailto:scim@ietf.org>
List-Help: <mailto:scim-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/scim>, <mailto:scim-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 01 Jun 2022 00:57:49 -0000

Hi Bojitha,

It's important to note that the cursor-based pagination draft is... a draft, which means it hasn't received the full rigor of peer review.  All 3 of your questions are reasons enough for a new version of the draft to be submitted.

Question 1
In the Cursor-based Pagination of SCIM Resources draft<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-peterson-scim-cursor-pagination-00%23section-2&data=05%7C01%7Cmatt.peterson%40oneidentity.com%7C15ec43f5057c4cf25ece08da41fdbd2b%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637894855326559034%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=yOfNhvjHZM%2Bc8i3S1TMJGt%2F7m9eEWuzqsosop2EkIOM%3D&reserved=0> [1], it hasn't been mentioned how to handle an unspecified count. Am I right to assume this can follow the same, as per RFC - 7644<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Frfc7644%23section-3.4.2.4&data=05%7C01%7Cmatt.peterson%40oneidentity.com%7C15ec43f5057c4cf25ece08da41fdbd2b%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637894855326559034%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=GRrNcrT1wAUUbnSninzHGh5qXb76icnr7l8W9KRKeMg%3D&reserved=0> [2] which states, "If unspecified, the maximum number of results is set by the service provider"?

Correct, the draft assumes that if count is omitted, the service provider will set the count as specified in RFC 7644.  It would be better if the draft explicitly mentioned this.

Question 2
In a situation where we wish to provide both cursor based pagination and index based pagination in the same SCIM service provider, how would you suggest the user decide the pagination approach? RFC - 7644<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Frfc7644%23section-3.4.2.4&data=05%7C01%7Cmatt.peterson%40oneidentity.com%7C15ec43f5057c4cf25ece08da41fdbd2b%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637894855326559034%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=GRrNcrT1wAUUbnSninzHGh5qXb76icnr7l8W9KRKeMg%3D&reserved=0> [2], states that both the startIndex value and the count are optional parameters when making the first offset based request. startIndex will be defaulted to 1 and the count will be defaulted to the value specified by the service provider. Furthermore, Cursor-based Pagination of SCIM Resources draft<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-peterson-scim-cursor-pagination-00%23section-2&data=05%7C01%7Cmatt.peterson%40oneidentity.com%7C15ec43f5057c4cf25ece08da41fdbd2b%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637894855326559034%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=yOfNhvjHZM%2Bc8i3S1TMJGt%2F7m9eEWuzqsosop2EkIOM%3D&reserved=0> [1], states that the cursor value of the initial request should be omitted.
eg:
Possible first request for offset pagination: /scim2/Users?count=20 -> Offset will be defaulted to 1.
Possible first request for cursor pagination: /scim2/Users?count=20 -> Initial cursor is omitted.
The SCIM service provider fails to distinguish the pagination mechanism that the user wants for these requests.

This is an ambiguity that I did not think of when writing the draft.   I assumed the ServiceProvider would detect index-based pagination by the existence of startIndex in the query.  You are correct, however, that startIndex is OPTIONAL.  The draft should be modified to address this ambiguity.  Do you have a recommendation?

Question 3
When querying for resources using a POST request combined with the "/.search" path extension, the query params will be contained in the request body. How should the cursor attribute be passed? This point is not mentioned in the Cursor-based Pagination of SCIM Resources draft [1]

{
  "schemas": [
    "urn:ietf:params:scim:api:messages:2.0:SearchRequest"
  ],
  "filter": "userName sw A",
  "domain": "PRIMARY",
  "count": 10,
  "cursor": "eyJ2YWx1ZSI6IlNDSU0yQ1"
}

Correct, this is how the cursor and count should be passed.  Again, the draft should be modified to use of cursor pagination with "/.search"

Once there is feedback on your question #2, I will resubmit a new revision of the draft.

--
Matt Peterson