Re: [mpls] Concerns about ISD

Tony Li <tony.li@tony.li> Mon, 18 April 2022 20:32 UTC

Return-Path: <tony1athome@gmail.com>
X-Original-To: mpls@ietfa.amsl.com
Delivered-To: mpls@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 700A63A04BC for <mpls@ietfa.amsl.com>; Mon, 18 Apr 2022 13:32:34 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.512
X-Spam-Level:
X-Spam-Status: No, score=-1.512 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.248, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.248, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GZ-F3IoMsxT0 for <mpls@ietfa.amsl.com>; Mon, 18 Apr 2022 13:32:31 -0700 (PDT)
Received: from mail-pg1-x52f.google.com (mail-pg1-x52f.google.com [IPv6:2607:f8b0:4864:20::52f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B3E5B3A07A4 for <mpls@ietf.org>; Mon, 18 Apr 2022 13:32:31 -0700 (PDT)
Received: by mail-pg1-x52f.google.com with SMTP id k14so20938063pga.0 for <mpls@ietf.org>; Mon, 18 Apr 2022 13:32:31 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=IJ3CC+bnYTrJWKm59MTub9ln4NWJ7wCE7PY2z1YFwhI=; b=mLIFjNM6fga1WHUsqKzvzis9iH7m0LOJLhN1S6H3VmT+tZ9PIkrJULmIkpDH3iFH68 697/jh8+B9duZ6bzhYjOHFsXJLRLLJL0tWeH0QkXvLr/B7NxkvqTmcSblSJ6Kbz+8ooz y9I0R306xQAQNIUhArbFZZResnBFR5E0GMAeXrY35D+A0BQbn+50gjf1MrlaAx1DfI5f BolSLsJzumOTY4+W0RS3upo6/EQuWaqXF6FGD2+rh72uqwMNkOZThvFRmXsWHZOe4t5I WTyljDXIzvaa6PQrlU9BRulC5vL1uschYhc49lTq15/V3E3SZ70MDY7oCK4ExFn8Q3Kl h/xA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=IJ3CC+bnYTrJWKm59MTub9ln4NWJ7wCE7PY2z1YFwhI=; b=djJU2u6e77mgQNYateN6ePVAjN7y8lO/Y6Gq5Nqw/QXhzRjEBsSG379PBGewvWCb8O LaLsT7v5e9L5vm/UQ3Gi7XsOj0ji8tWmyEJcw+mnYiKjWYsaXbIeE9xO1JGEJmXW3tA6 HPEQeiETiCs6L4yoAuPQr4ZBjQPdXDtjz4A5XpIqhiUNxI6sdDh61CBmSmJBaVJXYOGR H+oNlk1DOrzLUeps5irbx5iMRFv7FzcMP8w+P1n2YuUFn1GzgpClMO6NPSFngLoK2jVP yOUJOyQwqGyVRt2UCeMCs7X2xZoYKau0OwC7d43/yMu9njU2qYl6xEu8dPHd6iW79ljo Pr/Q==
X-Gm-Message-State: AOAM5328CaxxGpOxW3bGigiT7eiEcy9AL06SHx5UnD80sSW4SpUeXBK5 mSvNCOLmhSbWjAWXLAAz4pGdCsdairI=
X-Google-Smtp-Source: ABdhPJzZ91BchpbXizwwW0sQi6+pb5frZS8drmRQLZf4utWd2NdcORQ7acVVxi015qY79osymP8iBQ==
X-Received: by 2002:a63:6c0a:0:b0:398:6bd2:a16a with SMTP id h10-20020a636c0a000000b003986bd2a16amr11728058pgc.191.1650313950541; Mon, 18 Apr 2022 13:32:30 -0700 (PDT)
Received: from smtpclient.apple (c-67-169-103-239.hsd1.ca.comcast.net. [67.169.103.239]) by smtp.gmail.com with ESMTPSA id w14-20020a17090a4f4e00b001cb510021ecsm17479236pjl.49.2022.04.18.13.32.29 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 18 Apr 2022 13:32:29 -0700 (PDT)
Sender: Tony Li <tony1athome@gmail.com>
From: Tony Li <tony.li@tony.li>
Message-Id: <90F25F5A-E953-45F4-A594-CC56DA796309@tony.li>
Content-Type: multipart/alternative; boundary="Apple-Mail=_2AB704CA-5B01-4A3A-B992-F3D1BB3A1FB0"
Mime-Version: 1.0 (Mac OS X Mail 15.0 \(3693.60.0.1.1\))
Date: Mon, 18 Apr 2022 13:32:29 -0700
In-Reply-To: <BY3PR13MB478725F22B60E681ABF94BC79AF39@BY3PR13MB4787.namprd13.prod.outlook.com>
Cc: John E Drake <jdrake=40juniper.net@dmarc.ietf.org>, Tianran Zhou <zhoutianran=40huawei.com@dmarc.ietf.org>, "mpls@ietf.org" <mpls@ietf.org>
To: Haoyu Song <haoyu.song@futurewei.com>
References: <6cc272447d2f4c779e85d5c42d3b3c6c@huawei.com> <8623637D-A32E-47A4-B5FC-4D2CF40BEDD1@tony.li> <6199e0e886f9437c95ef9b70719b00ec@huawei.com> <BCFD3F4A-36D6-47C2-B907-FC40B402F97C@tony.li> <3fb1f261ddff48deb0c2ea083cdbd16f@huawei.com> <6B96F21B-9331-4FA8-AD7B-84A4CA8B6FAB@tony.li> <903c57a48280454091495673ec2fe275@huawei.com> <BD5C1BE7-4633-4B51-BAC1-B2AE1C537F36@tony.li> <ad6b8c42b0aa4880b9dee02516f5e46f@huawei.com> <F5BB2CEB-CC8C-4E71-A2E7-B4212878C3B1@tony.li> <aa9c4b913d844410b2af90c8db78c194@huawei.com> <BY3PR05MB8081937B52E657713E8293BFC7ED9@BY3PR05MB8081.namprd05.prod.outlook.com> <a29c96be774845e582a66700d2264f7b@huawei.com> <BY3PR05MB8081870EF67C551727BBE2CFC7EC9@BY3PR05MB8081.namprd05.prod.outlook.com> <d5521b3972dd43e38276afbbdc7c2bda@huawei.com> <BY3PR05MB80813C7CAD7F2C12C36FB513C7EE9@BY3PR05MB8081.namprd05.prod.outlook.com> <BY3PR13MB47879EB8A582437DE936688C9AEE9@BY3PR13MB4787.namprd13.prod.outlook.com> <C493D0B8-4B57-4D19-BC27-70ABD7F50356@tony.li> <BY3PR13MB47878B227A37AAA06625194B9AEE9@BY3PR13MB4787.namprd13.prod.outlook.com> <0318B3A3-2884-4FD6-B5EF-377481D2657B@tony.li> <BY3PR13MB4787752FB6D147281A7150789AEE9@BY3PR13MB4787.namprd13.prod.outlook.com> <602D6128-3BE3-4A2D-B5C2-019AE0FADF09@tony.li> <BY3PR13MB47876188B5927A51BD4F4E739AEE9@BY3PR13MB4787.namprd13.prod.outlook.com> <BCB99042-ECA3-40C6-8581-FA1656DDF987@tony.li> <BY3PR13MB4787468DAA96610B9933E1659AF19@BY3PR13MB4787.namprd13.prod.outlook.com> <EB04096F-70B7-4FF0-973F-6C7C1FDDE837@tony.li> <BY3PR13MB47874EBD4E397AB18CD8AF819AF39@BY3PR13MB4787.namprd13.prod.outlook.com> <7451CC1D-88A7-4D13-9BE5-44BCBE95337A@tony.li> <BY3PR13MB478725F22B60E681ABF94BC79AF39@BY3PR13MB4787.namprd13.prod.outlook.com>
X-Mailer: Apple Mail (2.3693.60.0.1.1)
Archived-At: <https://mailarchive.ietf.org/arch/msg/mpls/jDUW5PkUTnlGtGmUrQ5Ui-QwiPM>
Subject: Re: [mpls] Concerns about ISD
X-BeenThere: mpls@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Multi-Protocol Label Switching WG <mpls.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/mpls>, <mailto:mpls-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/mpls/>
List-Post: <mailto:mpls@ietf.org>
List-Help: <mailto:mpls-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/mpls>, <mailto:mpls-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Apr 2022 20:32:35 -0000

Hi Haoyu,


> [HS] Here “expensive” means the performance impact. NP is struggling to squeeze every cycle out of the processing, since the throughput is reversely proportional to the number of cycles a packet can finish its processing, and NP’s throughput already falls behind ASIC. Engineers need to handcraft the code to save a bit here and there. There’s never enough. The cycle is really a precious resource which can’t afford to squander. 


First off, network processors and switches are ASICs.  It’s all just silicon. How you get there may vary, but at the end of the day, the performance that you get out given a specific process technology corresponds directly to the effort that you put into optimizing it.

Yes, you’re correct, every cycle is precious.  Memory reads are also precious. And in most every modern architecture, a memory read is far more expensive than an ALU operation. For example, in my history, memory reads took around 12ns and operations took 1ns.  Thus, spending a memory read to save a compute operation was a bad trade-off.

As a result, memorizing the memory reads is far more important that worrying about operations.


> Yes, I have.  The memory reads for parsing a header chain are extremely high.  No thank you.  The bandwidth requirements for the header chain are extremely high.  No thank you.
>  
> [HS] Not sure what you mean here. In ASIC, memory access is really fast. In software, every instruction and data need to be read from memory/cache. The more “if” in the code, the more memory accesses. And why header chain parsing requires higher bandwidth?


I’m sorry, but that’s simply not true. Memory accesses are limited by SRAM cycle times. If it’s in DRAM, it’s far worse than that. That is an order of magnitude higher than ALU operation times. Yes, instructions do need to be fetched. This is why, in some architectures, there are multiple operations per instruction.

Header chain parsing requires more bandwidth because you’ve distributed all of the information over memory.  Let’s look at the example again:

EHI: 4 octets
HEH: 4 octets
Entropy: 8 octets
GISS: 8 octets

Total: 24 octets

This requires that we perform at least 4 reads just for parsing. There’s two more reads to get the EL and GISS values into registers.  And this is again assuming that all PSD is readable. If your architecture doesn’t keep that much of the packet in its context, you’re in tough shape.

If we look at FAI:

AI: 4 octets
NFFRR: included in the above, so 0 octets
Entropy: 4 octets (30 bits, plus overhead)
GISS: 4 octets (30 bits, plus overhead)

Parsing requires 1 read. Again, there’s two more reads to get the EL and GISS values into registers, so that’s equal.

> Well, I prefer proposals where there is more flexibility.
>  
> [HS] Header chain is more flexible and extensible. It’s used for all the headers AFAIK.


Except that now you’ve decided that SOME operations get taken out of a fixed number of bits, not out of the header chain.  And once we’ve used that number of bits, we’re stuck, back having this same argument again in 10 years.  No thank you.

Tony