Re: [scim] SCIM Compliance Test Tools?

Danny Mayer <mayer@ntp.org> Tue, 08 December 2020 18:33 UTC

Return-Path: <mayer@ntp.org>
X-Original-To: scim@ietfa.amsl.com
Delivered-To: scim@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E16193A10A3 for <scim@ietfa.amsl.com>; Tue, 8 Dec 2020 10:33:32 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.224
X-Spam-Level:
X-Spam-Status: No, score=-1.224 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, NICE_REPLY_A=-0.001, SPF_SOFTFAIL=0.665, T_SPF_HELO_TEMPERROR=0.01, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pAt8MwTqp0-G for <scim@ietfa.amsl.com>; Tue, 8 Dec 2020 10:33:28 -0800 (PST)
Received: from chessie.everett.org (chessie.everett.org [66.220.13.234]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5BD0D3A109F for <scim@ietf.org>; Tue, 8 Dec 2020 10:33:28 -0800 (PST)
Received: from l34097ous.fios-router.home.rpega.com (unknown [198.22.153.34]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by chessie.everett.org (Postfix) with ESMTPSA id 4Cr82H3XF0zMNC7; Tue, 8 Dec 2020 18:33:27 +0000 (UTC)
To: "Karaimin, Aleyidin" <aleyidin.karaimin@sap.com>, Phil Hunt <phil.hunt@independentid.com>, "scim@ietf.org" <scim@ietf.org>
References: <0ade1a37-55ea-a16a-0a96-96aab3770ac1@ntp.org> <73E248E2-5A66-4AF8-A36C-F048B3ED6EF1@independentid.com> <83F8AC8F-0BC0-454B-B6E2-999C08715C2D@sap.com>
From: Danny Mayer <mayer@ntp.org>
Message-ID: <a066fb93-c100-3816-ebe1-51ae6196ce80@ntp.org>
Date: Tue, 8 Dec 2020 13:33:26 -0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.5.1
MIME-Version: 1.0
In-Reply-To: <83F8AC8F-0BC0-454B-B6E2-999C08715C2D@sap.com>
Content-Type: multipart/alternative; boundary="------------A3D7FD49A111860FA6A917EB"
Content-Language: en-US
Archived-At: <https://mailarchive.ietf.org/arch/msg/scim/IR671TUa-A8o0YkYZxB8YHXM5So>
Subject: Re: [scim] SCIM Compliance Test Tools?
X-BeenThere: scim@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Simple Cloud Identity Management BOF <scim.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/scim>, <mailto:scim-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/scim/>
List-Post: <mailto:scim@ietf.org>
List-Help: <mailto:scim-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/scim>, <mailto:scim-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 08 Dec 2020 18:33:33 -0000

I have managed to find a number of testing tools. Here's a summary.


  SCIM Testing

Here are a number of tools that can be used for testing SCIM 2 compliance.


    1.WSO-incubator

https://github.com/wso2-incubator/scim2-compliance-test-suite 
<https://github.com/wso2-incubator/scim2-compliance-test-suite>

This is a java implementation that creates a war file which needs to be 
installed on a Tomcat or similar server.

Currently this tool complains about Bulk testing even though our 
implementation does not support it and attempts to remove the Bulk 
testing part have made no difference.


    2.Suvera

https://github.com/suvera/scim2-compliance-test-utility 
<https://github.com/suvera/scim2-compliance-test-utility>

This is a java implementation and creates a jar file that is run locally 
and you just specify the port to use. This has been useful in testing 
but requires you to look at the output logging from the command line to 
see exactly where the errors are.


    3.SAP

https://github.com/SAP/scimono/tree/master/scimono-compliance-tests 
<https://github.com/SAP/scimono/tree/master/scimono-compliance-tests>

This has not yet been looked at.

Thanks everyone for their input.

Danny

On 12/7/20 11:30 AM, Karaimin, Aleyidin wrote:
>
> Hello,
>
> Maybe I noticed this thread too late. However, I would like to suggest 
> tool that our team exposed to the open source. 
> https://github.com/SAP/scimono/tree/master/scimono-compliance-tests 
> <https://github.com/SAP/scimono/tree/master/scimono-compliance-tests>
>
> Basically, it is a java library helping implementors to build SCIM 
> v2.0 API’s. In addition, it contains tool that can be run against any 
> target system to verify whether it is SCIM v2.0 compliant. It is very 
> helpful for our use cases. Furthermore, I will be glad to receive any 
> additional feedback.
>
> Best Regards
>
> Aleydin
>
> *From: *Phil Hunt <phil.hunt@independentid.com>
> *Date: *Friday, 13 November 2020, 21:56
> *To: *Danny Mayer <mayer@ntp.org>
> *Cc: *SCIM WG <scim@ietf.org>
> *Subject: *Re: [scim] SCIM Compliance Test Tools?
>
> Danny,
>
> I am not aware of any compliance/testing tools yet. It has certainly 
> been talked about.
>
> SCIM is basically a RESTful API wth  JSON documents. I just use tools 
> like Postman, PAW etc.
>
> Also, check out simplecloud.info <http://simplecloud.info> for the 
> list of implementations.
>
> Best,
>
> Phil Hunt
>
> @independentid
>
> phil.hunt@independentid.com <mailto:phil.hunt@independentid.com>
>
>
>
>     On Nov 13, 2020, at 10:55 AM, Danny Mayer <mayer@ntp.org
>     <mailto:mayer@ntp.org>> wrote:
>
>     I've been trying to find tools that I can use to figure out my
>     SCIM V2.0 implementation and I'm having trouble finding anything
>     useful. The client provisioner is failing but it's not clear why.
>     Are there any recommended tools that might help with this to
>     pinpoint the failures?
>
>     Thanks,
>
>     Danny
>
>     _______________________________________________
>     scim mailing list
>     scim@ietf.org <mailto:scim@ietf.org>
>     https://www.ietf.org/mailman/listinfo/scim
>