Re: [scim] SCIM Synchronization Problem

"Matt Peterson (mpeterso)" <Matt.Peterson@oneidentity.com> Thu, 19 August 2021 17:43 UTC

Return-Path: <Matt.Peterson@oneidentity.com>
X-Original-To: scim@ietfa.amsl.com
Delivered-To: scim@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 781CD3A104B for <scim@ietfa.amsl.com>; Thu, 19 Aug 2021 10:43:01 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.101
X-Spam-Level:
X-Spam-Status: No, score=-2.101 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=oneidentity.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UGzVt2DdEk20 for <scim@ietfa.amsl.com>; Thu, 19 Aug 2021 10:42:56 -0700 (PDT)
Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2134.outbound.protection.outlook.com [40.107.244.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F3C803A1042 for <scim@ietf.org>; Thu, 19 Aug 2021 10:42:55 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NWmhYqWhGtWzRyz5X64u5I58jl2yj1UzAk6CCh//W/znC0YykuUtRRUgGxfoxaQ2NRml/TKVzWe0ieOROMHXn6OHkd7vGoSR1f1HABzV7qH82ER/DUviwnA/A7RB0jvx74XA0R2WFfhjCK1YWBFTSGKG8rcQYcNQd5kLauSKIy6V1jslXQuXTHMLTJNE94zYuXOXbqu2cSokGf+todtZIrUSajp8+H3j0J0Mii/lM0bCAEgWUbGIAIteRHfnPmRkZHqbN4fDNUx8StSAHj+I2kyOdOS5k82kn8nW5pVYS5DZxZ/lR+LRqMdI/M4EO0UkV1tXYbs4uO4d5Sli8Wy1yA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PMhevUSesWuc+ambTdDb9EOyHdolQ+uoVUhlbP4xxwI=; b=YdDzB76nRc4qto0JCz0UBDXSi72qjK93gH2oE6JnBdhaD4khjNLupl1XKZsti2q1GP6r3WVcc+0bgvHhXgrb0ZRG1h/lwFUDF/sQFl3bF58cRlY80OiTPFzsj+9qWZ3ZnfM6YIXRWZw7DOA21ALCjwrj8SX2qOfo3jxMPt8wsC7VO2NuUTWUiVRKBWEWIoxB6Yt5KNaWLTsAtJ6lDIwwPQyCVMQlsTvcMUkC/C8WSl4PlDYzzKGpYDeXlhNhPHtrd7HbIcholspF0dnDhkuhBvtuBQLGkwp6876wBHWsO70O+2+bjHEy5qg92KmUxyaMX2mx/SB4yVWjThDPh3EA/w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oneidentity.com; dmarc=pass action=none header.from=oneidentity.com; dkim=pass header.d=oneidentity.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oneidentity.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PMhevUSesWuc+ambTdDb9EOyHdolQ+uoVUhlbP4xxwI=; b=iBb0PYpYSiJ+BZsMeNnG5G/GSOL5BtlfXi3TmqjQPw+pRnFm813Gy/ELxJTrERQURqloI7N08RjQaI5wTkBJd+/MuyBN4PGAXHUWKEJHNUjOPa+lRNutG6j3BUzHTGvsNEwJsR5lhyhdi4uoqh7BxjzVjBTXXehcrxt82XakXeQN9EvlCbO6geq/09zXIynRqZYDx3sx3hJkPckURO3AlnM4lpquAxoW35efERjDyohyNAFz68WAstWotMeR26zVwo/fhYFOX5JI9QwLfJCk5kCAJrGsGdgnfRFYfQU/eD1oxK6EAE85OrXf63wQSuKN1ZlPBa1NDoUCJxXX28Q4hQ==
Received: from MWHPR19MB0957.namprd19.prod.outlook.com (2603:10b6:300:a4::16) by MWHPR19MB1087.namprd19.prod.outlook.com (2603:10b6:300:a2::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4436.19; Thu, 19 Aug 2021 17:42:54 +0000
Received: from MWHPR19MB0957.namprd19.prod.outlook.com ([fe80::4880:4967:c535:5783]) by MWHPR19MB0957.namprd19.prod.outlook.com ([fe80::4880:4967:c535:5783%4]) with mapi id 15.20.4415.025; Thu, 19 Aug 2021 17:42:54 +0000
From: "Matt Peterson (mpeterso)" <Matt.Peterson@oneidentity.com>
To: Danny Mayer <mayer@pdmconsulting.net>, SCIM WG <scim@ietf.org>
Thread-Topic: [scim] SCIM Synchronization Problem
Thread-Index: AQHXlD4yOJYx7decRU+u2K9sFyDQQ6t7GC/Q
Date: Thu, 19 Aug 2021 17:42:53 +0000
Message-ID: <MWHPR19MB095771672B28345FA22D6399E1C09@MWHPR19MB0957.namprd19.prod.outlook.com>
References: <e8f9d66c-f356-61b8-d38a-b5288fb9c518@pdmconsulting.net>
In-Reply-To: <e8f9d66c-f356-61b8-d38a-b5288fb9c518@pdmconsulting.net>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: pdmconsulting.net; dkim=none (message not signed) header.d=none;pdmconsulting.net; dmarc=none action=none header.from=oneidentity.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 3cf9f26b-0eee-4d0b-c455-08d96338c60e
x-ms-traffictypediagnostic: MWHPR19MB1087:
x-microsoft-antispam-prvs: <MWHPR19MB10870332DB8F8C6EBC2D59D6E1C09@MWHPR19MB1087.namprd19.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8273;
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: LTtGm3ez5qXukLpInUbVnOI7jmMqnQXuzj2FRkWpVsixwu5AyziwfyjQgbreN/HEP7okdZUq7QVND2JILsrROtGv940DY7hgN2TmdTc4mZNqKnbhFXGVstXU6ZmDrPQtULGFz+kaz5FbJqnkkzk2JdzVHmfoPqoYxBn+2F1GIq5k4lumdtdMzEecO9n8ouoviosZepjrS1cTqG/pri/RDZOhAMBfzh4wDq0u8drFyQBVWBt1k7I/reoQeiVHuU4CZyn4to24KLwULDAa3YMXvlLBSf9ZyF8KdM+gTx6uc/JZX7LQ4bLdmRS2RNg8iOcLhwn20UE46CbY+sSu01askx2ULbJYV3fEfv2vk3FOZxixh32mtkFqUbjpRS108BXmBgDCm7gnIIhDuDzBlm3EXHzLIdIsfpiTNntd/tgE6AC9HhMzbP/ftb8q5IRxPGlu0ryBCWmJ4gA1JIq2cA//qIeY4OyBI2bvcAj5PdbBpaeKbiss2JPx9Ln61fbf+iODnbtphdoqntG627wh2BKN9L94jiyCnOkmjsXXYGtU24jSPd9abGgGN60GRNlbDcU/eo0y/7AUh1Lddow8pv2K13r/11M2ZuNkyBc7P6h3U2AsJYNsDwoYMwx3acHvHUANOllbk1koRTgtSWoLYn75GgMDiv6w6pGHj4xp5ZdLAPzifWjiuRkWfmLcIfDbHPDLbbLlw08qwXosgXEBeVdt0/pwnrAx0lQ4v5KyLaHCWY/WzMh/xFRWmLS2E1X1gvYIiukQOY0MRg2RikJfEnG81k9CFQoywZclbjG/8s93DF0=
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MWHPR19MB0957.namprd19.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(366004)(396003)(39850400004)(136003)(376002)(33656002)(478600001)(53546011)(6506007)(45080400002)(110136005)(7696005)(316002)(122000001)(38100700002)(38070700005)(66556008)(66476007)(76116006)(8936002)(64756008)(71200400001)(66446008)(9686003)(66946007)(52536014)(5660300002)(2906002)(83380400001)(55236004)(966005)(8676002)(55016002)(26005)(186003)(86362001); DIR:OUT; SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?8rYoExd7GE3fLCaYiAh9abd+1rmpZjR5peqwcUNCdxwuL9+1SOZ6zr0UiULb?= =?us-ascii?Q?cxHRVOKJ68z48yd46G/C7U/RVugeeezo3NeQ3i+GGtbe15BwSSdjZIp9GMBm?= =?us-ascii?Q?32f2OivsL5l5nU2lTAZtsVN9UJUfDiDLaJxOlc91SCKKFDyJ5J31mf2aj28m?= =?us-ascii?Q?Drrc7iNe6c38z1dyaz4TCy9gQOoMhG6Xnv9GYmLO/Z2uEHhQmmZ4bn7/Q5WM?= =?us-ascii?Q?Iue/IizYLKdNS/myfDDwfqeH2pFQT1r565C9YVPikzdPJqVp6sooVxqGp4Ik?= =?us-ascii?Q?ammy+1C1qEaha/xgG/RoHAFMuIJ2f8cx0fAEYTGz0XB+RJqXfIM2cluiokpI?= =?us-ascii?Q?6IUV3Pzack+il4jhQ0GTSsgXnZH8ArfLU1q/z23Uy+AEq9OGiRggMtay0YH3?= =?us-ascii?Q?E0uMof6Tm/84glRmBkYKQwUf19flB7Oxslifj2agyxftTLhJbeqEPZN6bWAE?= =?us-ascii?Q?bLHe9J3VQUBd1WmMJ00W2ie9Cq9JxNZ61q90YHqUuy3QKDUS50bwO/YWyF/W?= =?us-ascii?Q?JU+czH9Cc+L4pfgiiPMn0o6xLdsi/PcT8re9LOud4rOdwiq0njLU7R72sf97?= =?us-ascii?Q?cLB5SOoYmP12TCxGXG9+rgHM+x1O/qgM+bUhyF6/SWUbBtqwfdhc3NAmv3XH?= =?us-ascii?Q?lxnmGMio/67oDHjsTP58xLzRVWxcMMpMQesef1vAz7a5UNkTI5HmgE+aFV5Z?= =?us-ascii?Q?hbRNlQS9PGhTTisZigutz0s9SqNQBob+ZAdZJFHaJ/2d3kc3tHziMO14RbVr?= =?us-ascii?Q?CzGibXYQ8QJDcxnAvcqsipc4/aC9u0EpOd5KRBygwvbkx9uK3YEVU962VKFM?= =?us-ascii?Q?qU0HXVtcRhzEt85JwEHLG0b3CfpfZmhAnLg5HmdHajflrKhFVJ70d95JB0p3?= =?us-ascii?Q?SasP4/juopqwP5xmkPmMLlliUDihrVLa+hY4SYijkxY5a0mpawbUQ4emfc0r?= =?us-ascii?Q?A1NbA19H0yDkrtkkt6oc1vj9xieThOr/53PKyDoKiuQsKW20QEX32Er7TKyf?= =?us-ascii?Q?s7Z94er8QfdkopcwKb2/vgxqLAQJkqr1xTfnvt9x0snPQhLP62R7mcTQY9pv?= =?us-ascii?Q?PnZq2T4ZD24cQuw3LD6Qj7sI1Ceb8xzKSMsjetMFmwpPy4LX2fWXHpZAZLj+?= =?us-ascii?Q?gF9qSVc1cnVir7kQPJ/chD82ACdS51+edUB2RS2ZkY0+ApjoAV1fnQIuxdNb?= =?us-ascii?Q?4xZC8Ff3dz51A2FnQNHZGbrooTC+XWeNnOVLn9X2YsXXZ9wNJVtz1/AMUEfK?= =?us-ascii?Q?uehJmfTl8VGM/1C6ypiPbNOcjUp8dprnoFxj3oGlrjxhUJ0dPGZ6Pg2MzcR3?= =?us-ascii?Q?m7BY6s2S7NYYCQPb6VYE05Dl?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: oneidentity.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR19MB0957.namprd19.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3cf9f26b-0eee-4d0b-c455-08d96338c60e
X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Aug 2021 17:42:54.0351 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 91c369b5-1c9e-439c-989c-1867ec606603
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: tahaYRqyWVAlnQothYXPixh7Boy5gIkfPWbUEP5bbcJO5asfD1PI4RgP8TANDBrrUYamo4yqYwuOxzHmjlw1ng==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR19MB1087
Archived-At: <https://mailarchive.ietf.org/arch/msg/scim/jm7jR98cZh3wliOme1quZu4VDwE>
Subject: Re: [scim] SCIM Synchronization Problem
X-BeenThere: scim@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Simple Cloud Identity Management BOF <scim.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/scim>, <mailto:scim-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/scim/>
List-Post: <mailto:scim@ietf.org>
List-Help: <mailto:scim-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/scim>, <mailto:scim-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 19 Aug 2021 17:43:02 -0000

Danny,

To help me understand in your post is the "SCIM Client" and what is the "SCIM Server" can you tell me which of your components (the Management Server or the Application Server) implements SCIM endpoints?   

The component that acts as a "SCIM Server" is the component that provides the of WebAPI endpoints that conform to the SCIM spec.  For example, the  /user, /group SCIM endpoints. 

The component(s) that uses these endpoints to query (GET)  users/groups or to create (POST) user/groups is the "SCIM client".

--
Matt 

-----Original Message-----
From: scim <scim-bounces@ietf.org> On Behalf Of Danny Mayer
Sent: Wednesday, August 18, 2021 8:34 AM
To: SCIM WG <scim@ietf.org>
Subject: [scim] SCIM Synchronization Problem

CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.


I decided that this needs it's own thread and not be part of the meeting minutes.

I have had a great deal of experience dealing with the user account synchronization problem. Here's my view of the problems.

I will be calling one system Management Server and the other system Application Server. I found client/server labels confusing. The Management Server is what I am defining to be the server that sends updates to add/update/remove users and groups to the Application server whose account, groups and access permissions are being managed.

First some definitions of user accounts. There are usually more than one of each of these:
1. Builtin accounts
2. Special-purpose accounts
3. Employee
4. Contractor
5. Agent
6. Customer

There may be more.

1. Builtin accounts: These are accounts that applications have and there may be more than one. There is always an admin account which can do anything, for example the administrator account in Active Directory or a database admin account. The application may have more accounts for other purposes.

2. Special-purpose accounts: These may be set up to provide access to other applications, for example a SCIM request to a SCIM REST API should be handled by a special account which cannot be used to login via a UI interface and only be able to perform certain functions. In addition there may be accounts set up to listen for topics or queues on a message queue among other possibilities. Keeping separate accounts like this are important for tracking in logs and applications.

3. Employee: These are accounts that employees may login to the application.

4. Contractors: These accounts that a contractor performing work for the company may use to log into an application. Unlike Employee accounts these would have an expiration date.

5. Agent: Accounts like this are for external users who may need to manage information for their own customers. An example of this is an insurance agent logging in to handle an insurance policy for their clients.

6. Customers: These are where the customers are using the application directly. For a bank it's likely to be millions of customers. The management platform should not be involved in managing these accounts.

Let's now look at a few example applications.

1. Helpdesk
All employees and contractors will need to be able to log into a helpdesk application and enter tickets. This means loading information about all employees and contractors. For a company with only 1000 employees that's manageable. For a company with 100K employees, it's a bigger challenge.

2. Customer Support
Only employees or contractors in the department providing customer support need access plus a few other employees. In addition identified customers may need accounts.

3. Expenses
Not all employees or contractors will be submitting expenses so it may not be necessary to have accounts for all possible users. This is something that the application owner needs to decide.

Now let's look at logistics.

Bulk load:
Each application will need an initial set of accounts set up and for something like a helpdesk this could involve load 1000-100,000 accounts.
The information needed could come from either the management server or separately, say from an HR system. Many servers that I have encountered limit the number of records to something like 1000, so the pagination requirement is needed for this. Even when dealing with a limited subset of employees or contractors you can run into this need.

Synchronization
An application that is bulk-loaded above may need to be synchronized to the management server if the data did not come from the management server.

Change Management
This is really a synchronization issue as well. Changes happen all the time and new employees/contractors need to be added, terminated ones removed and updates happen all the time. The best way of dealing with this may be to set up a message queue that each application can subscribe to and they can take the needed action when it's convenient for that application. It's not the only method but it's the one I found to be the most helpful. There are two ways of doing that: 1. send the complete user information for new accounts, send just the change for updating accounts, send the ID for terminated accounts along with some meta information. The other method which I have used is just to send the ID and whether it's new, updated or terminated.

I hope this is helpful to the discussion.

Danny


_______________________________________________
scim mailing list
scim@ietf.org
https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fscim&amp;data=04%7C01%7Cmatt.peterson%40quest.com%7C1fb1fe803b8c439026bd08d9625553ba%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637648940889425726%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=riJQy3jEHltc6ZFCaEe%2B6%2F2ecXN0SWBXOxIZAKLxum4%3D&amp;reserved=0