Re: Service Redundancy using BFD

Ashesh Mishra <> Wed, 29 November 2017 00:56 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 241DD127B5A for <>; Tue, 28 Nov 2017 16:56:19 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -0.01
X-Spam-Status: No, score=-0.01 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=1.989, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id ZNztsSwptg_6 for <>; Tue, 28 Nov 2017 16:56:15 -0800 (PST)
Received: from ( [IPv6:2a01:111:f400:fe4c::830]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 723F7120724 for <>; Tue, 28 Nov 2017 16:56:15 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=vAL8SMzQyZ+dC/98yEFrS6wS0tNGp9SQrxd5YsoIRO0=; b=fYO7wY4hpkXTAcKWxdtHjCQdWJyslk81zfydOI1LWluw2CzSVtw/arWvsTBWnCDwe2dotEJ1mXmf/TRyhlcFYHYFiIidYB2x0Tk42A7VXr7I77Mhd7ZYJsMKtjkPYV03nmlZ9nVToL/B8tmm9jDNYO1A1Nhpe4b1/Ovb82XNu1QQGqtMvL+1nUpE93C6451Kx7nBrVRmrsSNGbeDys56MtxMRoNmWrJZl1MiibSwMGRMOUzj4vf/Ze6GqjtSY93XfT9GFd/Ay7nXqx/kMzCkZpyrRYMKRtlEvEfjLKdhCm3w4ulFEN4qn7EMIb4tEo9XsxGKl0GJQG+tM3S4RYAC6A==
Received: from ( by ( with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id; Wed, 29 Nov 2017 00:55:57 +0000
Received: from ( by ( with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id via Frontend Transport; Wed, 29 Nov 2017 00:55:57 +0000
Received: from ([]) by ([]) with mapi id 15.20.0260.006; Wed, 29 Nov 2017 00:55:57 +0000
From: Ashesh Mishra <>
To: Ankur Dubey <>, Sami Boutros <>, "" <>
CC: Reshad Rahman <>
Subject: Re: Service Redundancy using BFD
Thread-Topic: Service Redundancy using BFD
Thread-Index: AQHTZ/M82vlye4FAgk+FmQYa86ul9aMpXmmAgAA3kICAAGH9AP//z8oAgAA42wD//9WuAIAART2A///YzwAAAPTXgAAHPY8A
Date: Wed, 29 Nov 2017 00:55:57 +0000
Message-ID: <>
References: <> <> <> <> <> <> <> <> <> <>
In-Reply-To: <>
Accept-Language: en-US
Content-Language: en-US
x-incomingtopheadermarker: OriginalChecksum:E2714CADB60FA50071F08552725F2436FC32A3108372CF48A2B1DACF52BD4D23; UpperCasedChecksum:142F07D8CC893D1852E40BBD6017ECCD782BE38ED6FBC41B58BB02D9860DAE59; SizeAsReceived:7521; Count:47
x-ms-exchange-messagesentrepresentingtype: 1
x-tmn: [cvWM528YiFwU+xKuao2KmYQy/pE/Bs5x]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; SN1NAM04HT139; 6:I3GWPpLDF/YyiUsZEuea5XQ8dcGstEqThGzbB+8VAjjjNhIsoA0o+MnjQGGpNGFJnKCHg0lsO31FkUqun4BAVgsSCP6LlBI1DG7w8S/Q+sx5KquzjBKN9MtW6U6T/aQg6lTbKQESKsjhFBlUCBcKuRpFD8xRBguwM7yxI7RLB9KUdCk81mPiP1lEnslz1oF6pMt+2c8ZeOuUfMztzFNsNuWy1HXrNQzZnJJ8CYwUmqsV6kCuX6mvfWXaKzMBd70vEdg6OAWJi0z48JF2xTCSHFScmi0EaM5rfYjFObWdjO+XqqaUxmp8LAsxuFHOr9f0xXdHG4VHkohZF1qQ3xz2zN2ASq7n9zih2jqqu29tp14=; 5:h5Kf4yZ1ca4cHRzyi0X5ieDn4tdCnZq7Z7o+FoS+IodD+0VTdg5RdSLlvCYpCLnA+kzb2cKVpsd/cnnVybniyl7v/jPpWXmFXKRnXXYcBu6gFBR4vrEzzawja/OGvSLZSHBrdAryHd9k6vSVZ/WRse3ZY3K+PqxlrQe+VhdtoqI=; 24:p/RT0RrXRlkPGkIUVZj6PDbgq7uTBzjcdXSt/0NhaVMzqT15XVFxxG/FdJZ5Oo6y3lSA2+4sOvKcV1p+mOtlcpWowDO+ZB84XBYeV9r6MJY=; 7:sdAVAsV/hXwPiS2bo/VGBXErDZ5TfnX00fkdXrKtPA8XGExmadO6MeUIQrdFlnIvSfBgdgoljL5N4Kc2iRjEFE/+Hfarz8xpnuHHS05O5jRHNpHE7K02PfJHr9cl/wmiIiIbrpp3sSsPZI4sVaYOmc3x3Rtwbx5BLJNZ1gvOyEspR+98B2vDglhCntX2hX3DhxmA1VrH0WvRxXftlVsMK737IC+MCXUI1hwyK42ALZyM9+B57Jj/miy5mtI7HRMf
x-incomingheadercount: 47
x-eopattributedmessage: 0
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(201702061074)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031324274)(2017031323274)(2017031322404)(1601125374)(1603101448)(1701031045); SRVR:SN1NAM04HT139;
x-ms-traffictypediagnostic: SN1NAM04HT139:
x-ms-office365-filtering-correlation-id: 1e2629aa-2d96-4381-93f5-08d536c3f3bf
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(444000031); SRVR:SN1NAM04HT139; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:SN1NAM04HT139;
x-forefront-prvs: 05066DEDBB
x-forefront-antispam-report: SFV:NSPM; SFS:(7070007)(98901004); DIR:OUT; SFP:1901; SCL:1; SRVR:SN1NAM04HT139;; FPR:; SPF:None; LANG:;
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: multipart/alternative; boundary="_000_7C9430DCB97A4F429774CD4F74B47A44outlookcom_"
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 1e2629aa-2d96-4381-93f5-08d536c3f3bf
X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Nov 2017 00:55:57.7953 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Internet
X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1NAM04HT139
Archived-At: <>
X-Mailman-Approved-At: Wed, 29 Nov 2017 04:29:15 -0800
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: "RTG Area: Bidirectional Forwarding Detection DT" <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 29 Nov 2017 00:56:19 -0000


The bitmap that you mentioned in your previous email to demultiplex the services needs more clarity. Where is that bitmap added in the BFD frame? How does a bitmap represent a subset of the services? I feel there is an underlying assumption in the use-case that’s not clear in the proposal that simplifies the service structure.


From: Ankur Dubey <>;
Date: Tuesday, November 28, 2017 at 7:28 PM
To: Ashesh Mishra <>;, Sami Boutros <>;, ""; <>;
Cc: Reshad Rahman <>;
Subject: Re: Service Redundancy using BFD


In case you meant that C and D are network nodes (not services) peering with A-B which are providing redundancy for any L2/L3/L4-L7 services, I’d like to clarify the following:

The mechanism to indicate to C&D which node (A or B) should attract traffic for a given service is not described in this draft. Like Sami mentioned in another email, there can be many ways to do that depending on the deployment scenario.

The solution described in this draft helps to establish understanding between the network nodes proving redundancy (A and B) regarding which node should be Active for a given service.


From: Ankur Dubey <>;
Date: Tuesday, November 28, 2017 at 4:01 PM
To: Ashesh Mishra <>;, Sami Boutros <>;, ""; <>;
Cc: Reshad Rahman <>;
Subject: Re: Service Redundancy using BFD

Hi Ashesh,

Yes, multiple services can be running between A & B. The indication of Active is needed on BFD packet only when the backup node is acting as the active for a given non-revertive service.

If all non-revertive services (lets say C and D in your example) are Active on a backup node (lets say B), the new diag code is sufficient to indicate the Active status for those services.

If some non-revertive services are active on A, while others on B, the bitmap indicating active services is needed in the payload.


From: Ashesh Mishra <>;
Date: Tuesday, November 28, 2017 at 3:21 PM
To: Sami Boutros <>;, Ankur Dubey <>;, ""; <>;
Cc: Reshad Rahman <>;
Subject: Re: Service Redundancy using BFD


Thanks for the clarification. In a typical scenario, this will look like:

A <-------------\
            \           \
             C            D
            /           /
B <-------------/

were there are two sets of services. The BFD session between A and B in this case will be overloaded with the states for the two sets of sessions. It’s not clear from the proposal if this scenario is addressed (and how).

From: Sami Boutros <>;
Date: Tuesday, November 28, 2017 at 5:13 PM
To: Ashesh Mishra <>;, Ankur Dubey <>;, ""; <>;
Cc: Reshad Rahman <>;
Subject: Re: Service Redundancy using BFD

Hi Ashesh,

The topology is more like the following:

A <—\
|         \
BFD      C
|         /

A and B are nodes providing L2 and L3 services for C, with A/S redundancy.

A can be active and B standby, if A goes down then B start providing the services.


From: Ashesh Mishra <<>>
Date: Tuesday, November 28, 2017 at 1:45 PM
To: Sami Boutros <<>>, Ankur Dubey <<>>, "<>" <<>>
Cc: Reshad Rahman <<>>
Subject: Re: Service Redundancy using BFD

Okay. That makes sense now.

So in a scenario where you have a primary overlay service between A and B, and a backup overlay service between C and D, the BFD sessions in question will be between A and C, and B and D (so that the backup can send diag code to primary)?

A <------- primary service --------->B
|                                                           |
BFD                                                    BFD
|                                                           |
C<-------- backup service ---------->D


From: Sami Boutros <<>>
Date: Tuesday, November 28, 2017 at 4:21 PM
To: Ashesh Mishra <<>>, Ankur Dubey <<>>, "<>" <<>>
Cc: Reshad Rahman <<>>
Subject: Re: Service Redundancy using BFD

Hi Ashesh,

A service is an overlay service running on a routing node, this could be a L2 or L3 VPN service running on set of links connected to 2 or more nodes, where one node is active for a service at a given point in time, and one node is standby.

Now, BFD is running on underlay links between the 2 nodes active and standby, once BFD goes down, the standby assumes that the active went down and activates the services that it shares with the active. On the BFD session the standby would signal to the old active when it came back up that it activated the non-preemptive services via this diag code saying that it didn’t fail, so the old active node doesn’t activate those non-preemptive services.


From: Ashesh Mishra <<>>
Date: Tuesday, November 28, 2017 at 1:14 PM
To: Sami Boutros <<>>, Ankur Dubey <<>>, "<>" <<>>
Cc: Reshad Rahman <<>>
Subject: Re: Service Redundancy using BFD

Thanks for the response, Sami. I think our disconnect lies in the definition of a service. From a BFD perspective, I expect the service to be established across two nodes, at the very least, so that BFD can monitor its liveness. Can you elaborate on

-          What, in the context of this draft, a service is?

-          How does BFD signal for a service that it is not monitoring the liveness for?


From: Sami Boutros <<>>
Date: Tuesday, November 28, 2017 at 1:23 PM
To: Ashesh Mishra <<>>, Ankur Dubey <<>>, "<>" <<>>
Cc: Reshad Rahman <<>>
Subject: Re: Service Redundancy using BFD

Hi Ashesh,

Thanks for your comments.

For your first comment the draft applies to both single hop or what you call interface BFD and multi hop BFD too. And yes the per service could be per interface too if this is a single hop BFD, we can clarify that in the draft.

For your second comment, I am not sure I understand. The service will be active only on one node, if the service is associated with the whole node, then the BFD session is monitoring the node liveness. And when the service is associated with an interface the BFD session will monitor the interface connectivity as well. So, a primary service can’t be active at the 2 node endpoints hosting the BFD session.


From: Ashesh Mishra <<>>
Date: Tuesday, November 28, 2017 at 4:04 AM
To: Ankur Dubey <<>>, "<>" <<>>
Cc: Reshad Rahman <<>>, Sami Boutros <<>>
Subject: Re: Service Redundancy using BFD

Hi Ankur,

This is a good proposal to pursue within the BFD-wg.

Couple of comments:

-          BFD can only signal this diag code for the interface that it is monitoring (the IP next hop, MPLS LSP, etc.). You mention per-service (which I assume means per-service-per-interface) failover in the draft but it may be worthwhile defining behavior on per-service-type-per-interface as well.

-          There still needs to be a method for the primary and backup pairs (two BFD end-points on primary service and two on backup service) to communicate with each other (primary-to-primary and backup-to-backup) if the service is active or standby. This is useful in the scenario when the primary cannot communicate with backup nodes (it is a failure condition after all).

Again, at 10k ft, I like the idea of signaling active/standby using BFD.


From: Rtg-bfd <<>> on behalf of Ankur Dubey <<>>
Date: Monday, November 27, 2017 at 9:47 PM
To: "<>" <<>>
Cc: Reshad Rahman <<>>, Sami Boutros <<>>
Subject: Service Redundancy using BFD

Hi all,

Please review and provide comments for the following draft:<>

Summary of draft:

This draft proposes a new BFD diag code via which a node running a BFD session with another node, can inform the other node after a BFD session times out, that it didn’t go down and did live through the failure.

Such notification is useful for a set of nodes providing Active/Standby redundancy. When these nodes are running multiple L2/L3/L4-L7 services  in non-revertive mode of redundancy, the standby node taking over as active for non-revertive services after BFD times out needs to indicate in the BFD packet that it outlived the other failed old active node. The new diag code will be used for this purpose. When this diag code is set in the BFD packets, it will provide an indication to the failed old active node that it MUST NOT activate the non-revertive services when it comes up.

For providing a per service level failover, a node activating certain non-revertive services needs to indicate that it is Active ONLY for those non-revertive services. This can be done by using a unique bitmap where each bit position is uniquely identifying a service. This unique bitmap is configured on all nodes by a network controller. When there is at least one non-revertive service for which a node is not active AND it is active for at least 1 non-revertive service, this node will set bits identifying the active services in the bitmap and send it in the payload of the BFD packet.