Re: [httpapi] Header for idempotency

"Martin J. Dürst" <duerst@it.aoyama.ac.jp> Tue, 25 May 2021 07:26 UTC

Return-Path: <duerst@it.aoyama.ac.jp>
X-Original-To: httpapi@ietfa.amsl.com
Delivered-To: httpapi@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CEFA33A18BB for <httpapi@ietfa.amsl.com>; Tue, 25 May 2021 00:26:17 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.001
X-Spam-Level:
X-Spam-Status: No, score=-2.001 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, MSGID_FROM_MTA_HEADER=0.001, NICE_REPLY_A=-0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, URIBL_SBL=0.5, URIBL_SBL_A=0.1] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=itaoyama.onmicrosoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yA1vzpAPuOj7 for <httpapi@ietfa.amsl.com>; Tue, 25 May 2021 00:26:11 -0700 (PDT)
Received: from JPN01-TY1-obe.outbound.protection.outlook.com (mail-eopbgr1400115.outbound.protection.outlook.com [40.107.140.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9DBAF3A18B9 for <httpapi@ietf.org>; Tue, 25 May 2021 00:26:10 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jvRJKqnlw9uJkYQPP2goLp43oz62BqIgQIGHd+7W9CYJq2L278c/66/XeQGU81UqM98vUvCC+ZPs2ut4xw6cgn94jvVR8kMASGNGucafU0T2bogsC35lVUh9jtmTD7HtLhDlHI3zcSWNQoWroHVwThb31lGvQ96jQjANTEXGpUTcprxxaKnO7LSIinmLfkKoeVcsqHQp+RdR92KUgqOPIdm4lWnRstlmGKrfeERkrnW3jK+f96JVesrGx0ISJ9AshhBTDgCPc23ScY2Hiu60Rb35N5VW8ZtyBM157aI79vJpFAlDsXUWBtty/pFwOJuTTZCpnC4osESJkgBefRyk9g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=X2KXsViPw7DTuSJFAFYth8mMCy4/+220+2hfrYSbdGg=; b=ivtjbrC+on3YBLbpYEDc0h8cW8zQaJmFaja0q4vNr/Binm+EAuvDqb5z3aYXVCOGlMsZjF9nwyhYo7gn2x9jEHWDDTQd/gtRYGKnLJTmWh4pIKPB11ZTAg74rgnWh+G8RFAZiO7iRht74huN+smxm1KjlfILSMiCtTSkY1TPn70+BbfupHU3sOtHhfSzPEInuO1GLbGpVN3VnDBW6blzbPcma7jvM4GWHAiE5aQEs7/P9D5koayjYCPol19pn9p7Jthy+PbYP4s7w0xqQBOoN/aKzJlxPrKxV6PsXXIjf8zsE3tYTuKiwqCLfEz5jOTDbz7HyXbwh/zHa96zx7zJxQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=it.aoyama.ac.jp; dmarc=pass action=none header.from=it.aoyama.ac.jp; dkim=pass header.d=it.aoyama.ac.jp; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=itaoyama.onmicrosoft.com; s=selector2-itaoyama-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=X2KXsViPw7DTuSJFAFYth8mMCy4/+220+2hfrYSbdGg=; b=hN0oRtPFqZXP+kos9HpUkxMc+R5VNIB//Dg1XgtTbF+kTW9oi1iaCNVyTWSl3Yjn5NtV9aAagrslolCrVaGlNbw3qKBEod5IIQMRTtgjtHtAw2LctQn4+MlAjDL50N/WpNakK4WJVA0ErlAEYWvGTn7F5vQ4kQBClkpGwb8MKOA=
Authentication-Results: paypal.com; dkim=none (message not signed) header.d=none; paypal.com; dmarc=none action=none header.from=it.aoyama.ac.jp;
Received: from TYAPR01MB5689.jpnprd01.prod.outlook.com (2603:1096:404:8053::7) by TYAPR01MB6459.jpnprd01.prod.outlook.com (2603:1096:400:86::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4150.23; Tue, 25 May 2021 07:26:07 +0000
Received: from TYAPR01MB5689.jpnprd01.prod.outlook.com ([fe80::7c68:2926:ee00:a511]) by TYAPR01MB5689.jpnprd01.prod.outlook.com ([fe80::7c68:2926:ee00:a511%5]) with mapi id 15.20.4150.027; Tue, 25 May 2021 07:26:06 +0000
To: Sanjay Dalal <sanjay.dalal@cal.berkeley.edu>, httpapi@ietf.org
Cc: Jayadeba Jena <jjena@paypal.com>
References: <CAC5fHGN_GbQo9fPm=f82QoqfVv7vtt6DNL0kE2FWXzJ_pFHxrQ@mail.gmail.com>
From: "Martin J. Dürst" <duerst@it.aoyama.ac.jp>
Organization: Aoyama Gakuin University
Message-ID: <664c3366-0960-4869-7728-99dcc3190315@it.aoyama.ac.jp>
Date: Tue, 25 May 2021 16:26:05 +0900
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.2
In-Reply-To: <CAC5fHGN_GbQo9fPm=f82QoqfVv7vtt6DNL0kE2FWXzJ_pFHxrQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"; format="flowed"
Content-Language: en-US
Content-Transfer-Encoding: 8bit
X-Originating-IP: [133.2.0.87]
X-ClientProxiedBy: TYAPR04CA0016.apcprd04.prod.outlook.com (2603:1096:404:15::28) To TYAPR01MB5689.jpnprd01.prod.outlook.com (2603:1096:404:8053::7)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
Received: from [100.70.17.231] (133.2.0.87) by TYAPR04CA0016.apcprd04.prod.outlook.com (2603:1096:404:15::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4150.23 via Frontend Transport; Tue, 25 May 2021 07:26:06 +0000
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 87455426-bc89-4743-a943-08d91f4e5c51
X-MS-TrafficTypeDiagnostic: TYAPR01MB6459:
X-Microsoft-Antispam-PRVS: <TYAPR01MB64591B288DD99F9D234B478ACA259@TYAPR01MB6459.jpnprd01.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: LMM6ehqkcgb8a+nSJCb1PqcV4bwGAADAXzzN2JBkpA02LjemQA6w26FkQyhfUpj0/j+W1FUUPbfaPjOWSgtuT7/8LolGSvqalDEoMbxdaOZDOUOfJR7zhwBljuzG41otZyB9boSg4avXKMXn9Z1u4UIUPPjvSN3NFan8kJinaZ2yTsQOsIXKBtoEP86Wskhqyhc/nlDAvpWp7wkhMg2Xzy/St3GZhe31Whw6+i8Otp3GreRVzj/savdhlrAcWWOto2hJZUfmEDMB7qH0f4aa3+WDOM2D8GdIGse00sfUBAkOa5ai+xwanaYrEV//ssJm4ubJ7h9ma4bqn8cAAc3s6qA2jjWgM8EY8Pxi7B2XbC1d01vTrcA7jCaaKLVZ3EAsq23XQRG2po/9uIBTR+zLYtBKQNqg5xHs+qD3J0slB9IbU4L4GlVntvpmK1UTqM4wp4ZRr6j8T75HjtzPtDT14X1mEXdDeeOhZxVwZKWAMmimiJpPKEvTeI5l6T2od56x2hEDVwO2OYwLUONcTfD9DVOqd4pcEXHoXeSvXrnNJxZ+9FHoVvEri5KWKbAuqXV7gdFPwXs43gsGea5gTym3C/3+eK7HPhUdQfQsOTWGzf8IkrKwS7M2lN34bbXFK9cZ5ATsFaidmg7x2VYvgviTRbaXmY8pzf0hh/McuWswkZkgy705v2IdVcOOlUrc5K6B2IkJVh6gqWj8OtYQMIPGjtcAEtq8b0NBLiEvYOM4vN2CqdcBualpRBJVwE4khuGACXgft5a4nCZM/ReNOzut/wOTckUrU3LPfOlypLvzMu1BZDOxZcMY14bdV9tcDPVpTTCpBAmHhAR6Hlj8ndeOlwxoC+fprXpV4pRvciSRBUc=
X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:TYAPR01MB5689.jpnprd01.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(376002)(396003)(346002)(136003)(39850400004)(786003)(316002)(66476007)(86362001)(16576012)(66574015)(8676002)(38350700002)(31686004)(966005)(36916002)(52116002)(38100700002)(66946007)(478600001)(8936002)(6706004)(2906002)(5660300002)(83380400001)(16526019)(53546011)(26005)(31696002)(4326008)(186003)(956004)(6486002)(2616005)(66556008)(78286007)(43740500002)(45980500001); DIR:OUT; SFP:1102;
X-MS-Exchange-AntiSpam-MessageData: tvumCJlYenNFHR6IJgyYvO9v4qadBPCT0z5UgjZ7J1u1teKmIrs8r0pAqH54exeD26HlQXZ25xkwjXtOk0AN7+zfBBC85JJ0gfN3oSYvPEjRJ2svLLlsR9gyKwd7ckaRLRrb2x8BS6xlaC39LIW/5lqn0B5u6HngyMnKCeECzm4P+m5Pvv6ozcI5d2OSl2ikk5aEqJv+NhKPTfrjrYG4TnaqAc6CXJN7QAKKsGCzWpIT8J5ZQdoAOCQz5RLCkqP6pntlCRoqBj5KxLw/eDWj/I4ZGx20w3TtmCBgxncn0mV69Jg25K2QbWbxTj73jr1qrPDJNCYj8ZVhvQK0kMNWEu+c2fM4MXn5E+AT5fdHoMu5X4SwBr3LY5rtZ2E1DKYMP1ZLjzqXVYMOfi4vcs33zKwYvsxKkfrsdYvCtoGAO8yhWMceEEBTQDbI5M1XrjtTpGQ8CEKA/CO2DOwhyyhVjWH3wbYc1FSgpKacct0iI0VSk+DYHWpY0FyEZnIB+l7Jr8W6lSh5WGyskAfmG6a3xfSRjFVobXoyf8DsNgonRZ86MmlADK8Y/8w7C8g0T8dNOs8MvE8qU1vYaguen/fr+5wwr5YV/PVViiXxDrDGGLEjokiM1Vf0k2IeBU//YU/ZAfOV82KPDLpSSZBQf9Zb1JQoxyXKt5nWS3xqkDweKRzPp+s8Mc03llCEAvqVQWjE6gyipoulyy1iuPcdSMmhiH6ndVI/QZ8qkmIaqtdF9/4Wb9oC3YOvfH/Q0H9ZiX6X8Oo+TLhE7iTTVLK8+s41R831ZItngQ31OMrUkMQaGP/rBpbeBWOozv3568eNvP3gkMb/GEr3gbNQ0OhZTEqL/pgbBBN+6BUutEsfFoVjBCVdAHriPDxl8bxQV4UeInqQOnmiatAgTR/VX5wnULyEFsOPv/F5q7w6ugHUbTf0HrCl0qzLA0tSATmKIEC2NbEKH5HAdt6Qhv3kEIwt5EoPaimGzy2rqOVHebvQ1gGbSD2dQAyZWISsvtPHEIflDfxQvCt5jST7xFpmnd1avzB+Gg8aiNFEIoRGX+ZiDa2N0XIGOZI9zBhdHqPz6JV3lH67zj7tV57hTi0VlAs7vA07OrxlFf3bFY2NFXwdK5TZ22uroKM+CTnohsoliLk7lBRH/QujBb+dIJSeWJlgZFhbJWsq8ovshBG1nCL1WQldUMpjNhy7GMBdLhC5OW9lFHmNnZNWhkmzdXRb3weSgIlGem7pHBPbbWjwhYioikPUhvCs98vM3aaYzjTY4Y0GJjfu4icVVD4y6dpdSUuZfB1gu89wpQN7uE7mhHUyLztwnHXZTXvkTK18Mw2halIWZ6TO
X-OriginatorOrg: it.aoyama.ac.jp
X-MS-Exchange-CrossTenant-Network-Message-Id: 87455426-bc89-4743-a943-08d91f4e5c51
X-MS-Exchange-CrossTenant-AuthSource: TYAPR01MB5689.jpnprd01.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 May 2021 07:26:06.8532 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: e02030e7-4d45-463e-a968-0290e738c18e
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: b0RVniqi7tGb50WhWp2N4QGI65U4qKGjJDvtifZrqyZEKvBrLBhtixEM68gVfcQdqjo7GufH2pq3jVCwNhKQYg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: TYAPR01MB6459
Archived-At: <https://mailarchive.ietf.org/arch/msg/httpapi/7k5HhOixlrf__2m63og0nrkH4Jc>
Subject: Re: [httpapi] Header for idempotency
X-BeenThere: httpapi@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Building Blocks for HTTP APIs <httpapi.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/httpapi>, <mailto:httpapi-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/httpapi/>
List-Post: <mailto:httpapi@ietf.org>
List-Help: <mailto:httpapi-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/httpapi>, <mailto:httpapi-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 25 May 2021 07:26:18 -0000

Hello Sanjay, others,

I had a quick look. I'm not familiar with the details.

However, I think the title of this draft should not be
"The Idempotency HTTP Header Field", but "The Idempotency-Key HTTP 
Header Field". Same for some of the section titles in the draft.

Also, putting BCP 14 keywords in quotes in the actual text looks really 
strange. Maybe the text

    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
    "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
    "OPTIONAL" in this document are to be interpreted as described in BCP
    14 [RFC2119] [RFC8174] when, and only when, they appear in all
    capitals, as shown here.

should be changed to

    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
    "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
    "OPTIONAL" in this document are to be interpreted as described in BCP
    14 [RFC2119] [RFC8174] when, and only when, they appear in all
    capitals, as shown here, and without quotes.

Some text such as

    The idempotency key that is supplied as part of every "POST" request
    MUST be unique and "MUST" not be reused with another request with a
    different request payload.

is otherwise difficult to interprete. What's the difference between MUST 
and "MUST"? It looks like "MUST" isn't meant seriously.

Regards,    Martin.

On 2021-05-25 14:11, Sanjay Dalal wrote:
> Hello all,
> 
> Jayadeba and I would like to submit a draft for The Idempotency HTTP Header
> Field <https://datatracker.ietf.org/doc/html/draft-idempotency-header-01>
> to this working group for further consideration.
> 
> We have listed below some organizations using this header. Many of these
> were already using the header before we wrote the draft. Many are using the
> concept in their APIs but differently (listed below as well). Predominantly
> this header is used by fintech companies and financial orgs but we have
> found usage elsewhere too including in Django and https.org (HTTP for
> Scala).
> 
> Let us know how we can proceed. See the abstract and implementation status
> below.
> 
> thanks,
> sanjay
> 
> https://datatracker.ietf.org/doc/html/draft-idempotency-header-01
> 
> *Abstract*
> 
> The HTTP Idempotency request header field can be used to carry idempotency
> key in order to make non-idempotent HTTP methods such as POST or PATCH
> fault-tolerant.
> 
> 
> *Implementation Status*
> 
> Organization: Stripe
> 
>     o  Description: Stripe uses custom HTTP header named "Idempotency-
>        Key"
>     o  Reference: https://stripe.com/docs/idempotency
> 
>     Organization: Adyen
> 
>     o  Description: Adyen uses custom HTTP header named "Idempotency-Key"
>     o  Reference: https://docs.adyen.com/development-resources/api-
>        idempotency/
> 
>     Organization: Dwolla
> 
>     o  Description: Dwolla uses custom HTTP header named "Idempotency-
>        Key"
>     o  Reference: https://docs.dwolla.com/
> 
>     Organization: Interledger
> 
>     o  Description: Interledger uses custom HTTP header named
>        "Idempotency-Key"
>     o  Reference: https://github.com/interledger/
> 
>     Organization: WorldPay
> 
>     o  Description: WorldPay uses custom HTTP header named "Idempotency-
>        Key"
>     o  Reference: https://developer.worldpay.com/docs/wpg/idempotency
> 
>     Organization: Yandex
> 
>     o  Description: Yandex uses custom HTTP header named "Idempotency-
>        Key"
>     o  Reference: https://cloud.yandex.com/docs/api-design-
>        guide/concepts/idempotency
> 
> *Implementing the Concept*
> 
>     This is a list of implementations that implement the general concept,
>     but do so using different mechanisms:
> 
>     Organization: Django
> 
>     o  Description: Django uses custom HTTP header named
>        "HTTP_IDEMPOTENCY_KEY"
> 
>     o  Reference: https://pypi.org/project/django-idempotency-key
> 
>     Organization: Twilio
> 
>     o  Description: Twilio uses custom HTTP header named "I-Twilio-
>        Idempotency-Token" in webhooks
> 
>     o  Reference: https://www.twilio.com/docs/usage/webhooks/webhooks-
>        connection-overrides
> 
>     Organization: PayPal
> 
>     o  Description: PayPal uses custom HTTP header named "PayPal-Request-
>        Id"
> 
>     o  Reference: https://developer.paypal.com/docs/business/develop/
>        idempotency
> 
>     Organization: RazorPay
> 
>     o  Description: RazorPay uses custom HTTP header named "X-Payout-
>        Idempotency"
> 
>     o  Reference: https://razorpay.com/docs/razorpayx/api/idempotency/
> 
>     Organization: OpenBanking
> 
>     o  Description: OpenBanking uses custom HTTP header called "x-
>        idempotency-key"
> 
>     o  Reference: https://openbankinguk.github.io/read-write-api-
>        site3/v3.1.6/profiles/read-write-data-api-profile.html#request-
>        headers
> 
>     Organization: Square
> 
>     o  Description: To make an idempotent API call, Square recommends
>        adding a property named "idempotency_key" with a unique value in
>        the request body.
> 
>     o  Reference: https://developer.squareup.com/docs/build-basics/using-
>        rest-api
> 
>     Organization: Google Standard Payments
> 
>     o  Description: Google Standard Payments API uses a property named
>        "requestId" in request body in order to provider idempotency in
>        various use cases.
> 
>     o  Reference: https://developers.google.com/standard-payments/
>        payment-processor-service-api/rest/v1/TopLevel/capture
> 
>     Organization: BBVA
> 
>     o  Description: BBVA Open Platform uses custom HTTP header called "X-
>        Unique-Transaction-ID"
> 
>     o  Reference:
>        https://bbvaopenplatform.com/apiReference/APIbasics/content/x-
>        unique-transaction-id
> 
>     Organization: WebEngage
> 
>     o  Description: WebEngage uses custom HTTP header called "x-request-
>        id" to identify webhook POST requests uniquely to achieve events
>        idempotency.
> 
>     o  Reference: https://docs.webengage.com/docs/webhooks
> 
> 

-- 
Prof. Dr.sc. Martin J. Dürst
Department of Intelligent Information Technology
College of Science and Engineering
Aoyama Gakuin University
Fuchinobe 5-1-10, Chuo-ku, Sagamihara
252-5258 Japan