Re: [Ice] Re-nomination and candidate pair switching in RFC 5245bis - PROPOSAL
Peter Thatcher <pthatcher@google.com> Wed, 30 August 2017 16:28 UTC
Return-Path: <pthatcher@google.com>
X-Original-To: ice@ietfa.amsl.com
Delivered-To: ice@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 282CD132055 for <ice@ietfa.amsl.com>; Wed, 30 Aug 2017 09:28:28 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.7
X-Spam-Level:
X-Spam-Status: No, score=-2.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IEiYjN4dk0i1 for <ice@ietfa.amsl.com>; Wed, 30 Aug 2017 09:28:24 -0700 (PDT)
Received: from mail-oi0-x231.google.com (mail-oi0-x231.google.com [IPv6:2607:f8b0:4003:c06::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B02FD13202D for <ice@ietf.org>; Wed, 30 Aug 2017 09:28:24 -0700 (PDT)
Received: by mail-oi0-x231.google.com with SMTP id k77so55341870oib.2 for <ice@ietf.org>; Wed, 30 Aug 2017 09:28:24 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=btYBiikLAc4h3SvqxvG2MLJ5jDqj+QINIjt19zxriy0=; b=C1TfPVSp+Z+GSZ8VJX42GNDqr76wEIiSyIRMVdjo9hI3nYym86XjgndbdcMBUm7tfD ipkT2bnGlbIrXY+R7c7ldlOXTy7I4ZNVDoLs0HHMHFbGrYA3KLvD/90/heQbIj3xhrU3 0EWw2GFiooNtNJ8faNMnAwkS1+15fsS8hYueJMwPQOvdm0HUL33GTdQCA68rc01soYbP GeRNZrwpx5hOKnOC+MreUcLs2WYtBCJDg7PuBvTcjVdO43P+fFbNfnNWA3qK/H8eryVv OjcQ+RPFs9Tsw03ZwtQtoMaPlKmj7chrr6nAUg9rXWQyu1vHP2Cfb+nlNplchFp6KyKU dB/Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=btYBiikLAc4h3SvqxvG2MLJ5jDqj+QINIjt19zxriy0=; b=QFKgT/P5IaIKerGx+R8NKjcjvFYUk01f5Ys4abByWeAPxYPYJVaaXj9QP28utNC/L+ B1j/li3FEWQBJYBB2opqRhp+dt5Cv6NN8Bwxc8rQY873Ky9gKKyyWx1BZlJ5pTS83w6P CS7O8CJQYPIY3sZYd5WtksOLHcCL7UDZsfg7pYYmW3CpBEKIa9OonC0MzS/NBPXZfBCj W11puap4NtO1KvSWPHjtaSMUamrZP6hHRT43PJ+FcOBm14RKm/KoakpKm3OMbQDxfH9n PxiZ++pTmybrKNfUjB/Q5sY/k/GP3VSq/xawTq6Lr9Z7pUsF5eMahePb72IjPMR+5l/5 ffLA==
X-Gm-Message-State: AHYfb5hNzZqVL9GgoVomoKAV2rDKz5Q3k25qEArQ3yG6feKSxeePc55g /HL8trNm2SV91DPgIMPrsJl+XdD4xp905Fs=
X-Google-Smtp-Source: ADKCNb6tY0mcwG5sdMxB5F+nv7vzG1JoF4dPxg48HwjszCP8BYB1ERMaj/BLnSFttvQs+J1qSN9f0oZnCHypfHEKZfw=
X-Received: by 10.202.105.198 with SMTP id e189mr1903541oic.195.1504110503618; Wed, 30 Aug 2017 09:28:23 -0700 (PDT)
MIME-Version: 1.0
References: <7594FB04B1934943A5C02806D1A2204B4CC9ED94@ESESSMB109.ericsson.se> <CAJrXDUHGYkivt5+dnX48C8TkR9W7afoUWAPv8+MbafYNsZsFmA@mail.gmail.com> <CAOW+2duR54wenv088kZSmKJj56u8j=Qi6KzNUNrjA0=o5qhekg@mail.gmail.com> <CAJrXDUEKWCa7GsL_bSrdvVN-erROSgHJt5DO5VOZ2Qx3PQr15g@mail.gmail.com>
In-Reply-To: <CAJrXDUEKWCa7GsL_bSrdvVN-erROSgHJt5DO5VOZ2Qx3PQr15g@mail.gmail.com>
From: Peter Thatcher <pthatcher@google.com>
Date: Wed, 30 Aug 2017 16:28:12 +0000
Message-ID: <CAJrXDUH6vvauP8Bj2k+e5B=reTM=5C+vwD0+WidtdTRW5MpdAA@mail.gmail.com>
To: Bernard Aboba <bernard.aboba@gmail.com>
Cc: Christer Holmberg <christer.holmberg@ericsson.com>, "ice@ietf.org" <ice@ietf.org>
Content-Type: multipart/alternative; boundary="001a1141b266461aaf0557fb047f"
Archived-At: <https://mailarchive.ietf.org/arch/msg/ice/zLdaJCg49fFtJFB3noEJin1KlL8>
Subject: Re: [Ice] Re-nomination and candidate pair switching in RFC 5245bis - PROPOSAL
X-BeenThere: ice@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: "Interactive Connectivity Establishment \(ICE\)" <ice.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ice>, <mailto:ice-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ice/>
List-Post: <mailto:ice@ietf.org>
List-Help: <mailto:ice-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ice>, <mailto:ice-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 30 Aug 2017 16:28:28 -0000
Oops, hit send too soon. That's what https://www.ietf.org/archive/id/draft-thatcher-ice-renomination-01.txt does: it negotiates "renomination". And that's what Chrome's implementation of WebRTC does. On Wed, Aug 30, 2017 at 9:26 AM Peter Thatcher <pthatcher@google.com> wrote: > As far as "just enough" so that it isn't a hindrance: that's why I wanted > to say that agents *may* switch. It allows a renomination in the future. > Saying they MUST NOT is a hindrance. I would be fine if the *may* requires > negotiation. That's what > > On Wed, Aug 30, 2017 at 9:08 AM Bernard Aboba <bernard.aboba@gmail.com> > wrote: > >> Peter said: >> >> "Your reasoning for saying that an agent can't switch candidate pairs is >> that it prevents the remote side from being able to release resources. But >> the switching side would only switch if the candidate pair recently >> received an ICE check response (in other words, it knows the candidate pair >> works), in which case it knows that resources have not been released." >> >> [BA] There is a dicotomy between the situation prior to nomination and >> after it that isn't explained well in RFC 5245bis. Prior to nomination, >> the agent can switch between validated candidate pairs, and hopefully we >> can make the language clear enough so that this will work well. >> >> While the same logic applies after nomination as well (e.g. it only makes >> sense to switch a valid pair), the text doesn't clarify the meaning of >> nomination with respect to resource release, nor does it talk about consent >> (presumably the mechanism that would be used to maintain pair validity >> after nomination), so that it's not clear how pair validity could be >> maintained. Since there is much RFC 5245 text still remaining, the overall >> document reads as though while it represents an update to RFC 5245 it still >> is based on a pre-WebRTC mindset. >> >> All this makes me wonder whether we should limit our aspirations for RFC >> 5245bis to clarifying things "just enough" so that it isn't a hindrance. >> People interested in building a modern ICE implementation for use in WebRTC >> will probably focus more on Trickle-ICE, where implementation of consent >> can be assumed, and perhaps the meaning of nomination within a WebRTC >> context could be better explained. >> >> >> >> On Wed, Aug 30, 2017 at 11:19 AM, Peter Thatcher <pthatcher@google.com> >> wrote: >> >>> #2 and #3 are fine, but I'm not a fan of #1. >>> >>> Your reasoning for saying that an agent can't switch candidate pairs is >>> that it prevents the remote side from being able to release resources. But >>> the switching side would only switch if the candidate pair recently >>> received an ICE check response (in other words, it knows the candidate pair >>> works), in which case it knows that resources have not been released. >>> >>> In other words, if the remote side chose to release resources, the local >>> side wouldn't do the switch. So the remote side is free to release >>> resources; it's not prevented from doing so. If it does, then the local >>> side will notice and not do a switch. But if the remote side doesn't >>> release the resources, then the local side may switch. >>> >>> In other words, I think we can allow switching and resource release >>> both. >>> >>> On Thu, Jul 27, 2017 at 4:01 AM Christer Holmberg < >>> christer.holmberg@ericsson.com> wrote: >>> >>>> Hi, >>>> >>>> >>>> >>>> Would anyone disagree with the following: >>>> >>>> >>>> >>>> 1) explicitly indicating that re-nomination is **NOT** allowed >>>> without ICE restart; and >>>> >>>> 2) once a pair has been selected, agents need to be able to send * >>>> *AND** receive media using that pair – but not using any other pair >>>> (read: resources associated with other pairs may be released); and >>>> >>>> 3) PRIOR to selection, agents need to be able to send **AND** >>>> receive media on any valid pair (RFC 7675 adds restrictions, but that’s >>>> outside the scope of 5245bis) >>>> >>>> >>>> >>>> Regards, >>>> >>>> >>>> >>>> Christer >>>> >>>> >>>> >>>> *From:* Ice [mailto:ice-bounces@ietf.org] *On Behalf Of *Christer >>>> Holmberg >>>> *Sent:* 25 July 2017 10:17 >>>> *To:* Bernard Aboba <bernard.aboba@gmail.com> >>>> *Cc:* ice@ietf.org >>>> *Subject:* Re: [Ice] Re-nomination and candidate pair switching in RFC >>>> 5245bis >>>> >>>> >>>> >>>> Hi Bernard, >>>> >>>> >>>> >>>> Regarding sending media PRIOR to nomination, we have previously agreed >>>> that any valid pair can be used for that. Perhaps it needs more >>>> clarification. >>>> >>>> >>>> >>>> Regarding receiving media after nomination, it was discussed in Prague, >>>> as it is covered by Peter’s PR. I don’t have access to the PR/minutes right >>>> now, but I think the outcome was that an agent is only expected to receive >>>> media on the nominated pair (otherwise it cannot free resources). >>>> >>>> >>>> >>>> Regards, >>>> >>>> >>>> >>>> Christer >>>> >>>> >>>> >>>> *From:* Bernard Aboba [mailto:bernard.aboba@gmail.com >>>> <bernard.aboba@gmail.com>] >>>> *Sent:* 25 July 2017 02:29 >>>> *To:* Christer Holmberg <christer.holmberg@ericsson.com> >>>> *Cc:* ice@ietf.org >>>> *Subject:* Re: [Ice] Re-nomination and candidate pair switching in RFC >>>> 5245bis >>>> >>>> >>>> >>>> Christer said: >>>> >>>> >>>> >>>> "The whole discussion began when I was given a comment that the text >>>> above should be modified, to clarify that the pair used for media can >>>> change after a pair has been selected. >>>> >>>> But, if the outcome is that the pair can NOT change, maybe we need to >>>> clarify THAT instead :)" >>>> >>>> >>>> >>>> [BA] Currently, use of the "ice2" ICE option forestalls use of >>>> aggressive nomination (e.g. setting the nominated flag on more than one >>>> pair). Since only the selected pair can be used to send media, that would >>>> seem to rule out changing the pair used for media after a pair has been >>>> selected: >>>> >>>> >>>> >>>> Once a candidate pair has been selected >>>> >>>> only that candidate pair (referred to as selected pair) is used for >>>> >>>> sending media. >>>> >>>> >>>> >>>> What about changing the pair used for media prior to selection? On >>>> this point, the text seems less clear than it could be. >>>> >>>> >>>> >>>> Prior to nomination, the specification allows the sending of media on a >>>> successful pair: >>>> >>>> >>>> >>>> o Once there is at least one nominated pair in the VALID LIST for >>>> >>>> every component of at least one media stream and the state of the >>>> >>>> CHECK LIST is Running: >>>> >>>> >>>> >>>> ... >>>> >>>> >>>> >>>> * The agent MUST continue to respond to any checks it may still >>>> >>>> receive for that media stream, and MUST perform triggered >>>> >>>> checks if required by the processing of Section 6.3 <https://tools.ietf.org/html/draft-ietf-ice-rfc5245bis-10#section-6.3>. >>>> >>>> >>>> >>>> * The agent MAY begin transmitting media for this media stream as >>>> >>>> described in Section 11.1 >>>> <https://tools.ietf.org/html/draft-ietf-ice-rfc5245bis-10#section-11.1> >>>> . >>>> >>>> >>>> >>>> However, the specification is not clear enough about the receiving >>>> side; while it recommends that implementations be prepared to receive prior >>>> to nomination, it does not require this. From Section 11.2: >>>> >>>> >>>> >>>> ICE implementations SHOULD by default be >>>> >>>> prepared to receive media on any of the candidates provided in the >>>> >>>> most recent candidate exchange with the peer. >>>> >>>> >>>> >>>> What happens if an implementation is NOT prepared to receive media? >>>> >>>> In WebRTC, an implementation cannot send without consent, which >>>> >>>> suggests that perhaps an unwilling receiver could use consent to >>>> >>>> influence the potential sender. >>>> >>>> >>>> >>>> However, the specification does not even reference RFC 7675, >>>> >>>> so it is left unclear about how this is to be done. >>>> >>>> For example, a receiver might not reply to a consent >>>> >>>> request if the inability to receive is temporary ("I'm not ready yet"), >>>> >>>> but that might cause consent to time out prior to nomination and >>>> >>>> might even influence pair selection inappropriately. >>>> >>>> >>>> >>>> Another choice might be to revoke consent (which would >>>> >>>> invalidate the pair). But that's pretty drastic unless the >>>> >>>> pair is truly unacceptable. >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >[BA] RFC 5245bis Section 7.1.1 continues to imply a single selected >>>> pair: >>>> > >>>> > Eventually, there will be only a single nominated pair in the VALID >>>> > LIST for each component. Once the state of the CHECK LIST is set to >>>> > Completed, that exact pair is selected by ICE for sending and >>>> > receiving media for that component. >>>> > >>>> >Based on that text, an implementation might still release resources >>>> (e.g. unused TURN candidates) post-nomination. Given this, the "ice2" ICE >>>> option doesn't address >potential interoperability issues resulting from >>>> different resource release behaviors (although it does clear indicate lack >>>> of support for aggressive nomination): >>>> >>>> The whole discussion began when I was given a comment that the text >>>> above should be modified, to clarify that the pair used for media can >>>> change after a pair has been selected. >>>> >>>> But, if the outcome is that the pair can NOT change, maybe we need to >>>> clarify THAT instead :) >>>> >>>> > NOTE: A controlling agent that does not support this specification >>>> > (i.e. it is implemented according to RFC 5245) might nominate more >>>> > than one candidate pair. This was referred to as aggressive >>>> > nomination in RFC 5245. The usage of the 'ice2' ice option by >>>> > endpoints supporting this specifcation should prevent such >>>> > controlling agents from using aggressive nomination. >>>> > >>>> >Christer also said: >>>> > >>>> >"Also, my understanding was that endpoints supporting RFC 7675 might >>>> maintain consent on pairs currently not >>>> >used for media, in order to be able to re-nominate in case consent for >>>> the currently nominated pair expires. However, >>>> >RFC 7675 does not explicitly say anything about that." >>>> > >>>> >[BA] RFC 7675 Section 5 says: >>>> > >>>> > Initial consent to send traffic is obtained using ICE [RFC5245]. An >>>> > endpoint gains consent to send on a candidate pair when the pair >>>> > enters the Succeeded ICE state. >>>> > >>>> >Given this, an RFC 5245bis implementation might request consent to >>>> send to >>>> >multiple remote peer candidates, so as to keep them alive. However, >>>> >there is nothing in RFC 7675 that requires the responder to grant >>>> >consent for that. For example, based on the text in RFC 5245bis >>>> >Section 7.1.1, a conforming implementation might well revoke >>>> >consent on local candidates other than the local candidate in the >>>> >selected pair. >>>> >>>> Sure - the responder is not mandated to grant consent to multiple >>>> candidates after nomination. But, the option to do seems to be there >>>> (unless I've understood the RFC wrong), and the only reason to do so would >>>> be possible re-nomination. >>>> >>>> Anyway, I don't have any strong feelings which way we go, but we do >>>> need to make it clear in the spec whether re-nomination is allowed or not. >>>> >>>> Regards, >>>> >>>> Christer >>>> >>>> >>>> >>>> On Mon, Jul 24, 2017 at 4:33 PM, Christer Holmberg < >>>> christer.holmberg@ericsson.com> wrote: >>>> >>>> Hi >>>> , >>>> >[BA] RFC 5245bis Section 7.1.1 continues to imply a single selected >>>> pair: >>>> > >>>> > Eventually, there will be only a single nominated pair in the VALID >>>> > LIST for each component. Once the state of the CHECK LIST is set to >>>> > Completed, that exact pair is selected by ICE for sending and >>>> > receiving media for that component. >>>> > >>>> >Based on that text, an implementation might still release resources >>>> (e.g. unused TURN candidates) post-nomination. Given this, the "ice2" ICE >>>> option doesn't address >potential interoperability issues resulting from >>>> different resource release behaviors (although it does clear indicate lack >>>> of support for aggressive nomination): >>>> >>>> The whole discussion began when I was given a comment that the text >>>> above should be modified, to clarify that the pair used for media can >>>> change after a pair has been selected. >>>> >>>> But, if the outcome is that the pair can NOT change, maybe we need to >>>> clarify THAT instead :) >>>> >>>> > NOTE: A controlling agent that does not support this specification >>>> > (i.e. it is implemented according to RFC 5245) might nominate more >>>> > than one candidate pair. This was referred to as aggressive >>>> > nomination in RFC 5245. The usage of the 'ice2' ice option by >>>> > endpoints supporting this specifcation should prevent such >>>> > controlling agents from using aggressive nomination. >>>> > >>>> >Christer also said: >>>> > >>>> >"Also, my understanding was that endpoints supporting RFC 7675 might >>>> maintain consent on pairs currently not >>>> >used for media, in order to be able to re-nominate in case consent for >>>> the currently nominated pair expires. However, >>>> >RFC 7675 does not explicitly say anything about that." >>>> > >>>> >[BA] RFC 7675 Section 5 says: >>>> > >>>> > Initial consent to send traffic is obtained using ICE [RFC5245]. An >>>> > endpoint gains consent to send on a candidate pair when the pair >>>> > enters the Succeeded ICE state. >>>> > >>>> >Given this, an RFC 5245bis implementation might request consent to >>>> send to >>>> >multiple remote peer candidates, so as to keep them alive. However, >>>> >there is nothing in RFC 7675 that requires the responder to grant >>>> >consent for that. For example, based on the text in RFC 5245bis >>>> >Section 7.1.1, a conforming implementation might well revoke >>>> >consent on local candidates other than the local candidate in the >>>> >selected pair. >>>> >>>> Sure - the responder is not mandated to grant consent to multiple >>>> candidates after nomination. But, the option to do seems to be there >>>> (unless I've understood the RFC wrong), and the only reason to do so would >>>> be possible re-nomination. >>>> >>>> Anyway, I don't have any strong feelings which way we go, but we do >>>> need to make it clear in the spec whether re-nomination is allowed or not. >>>> >>>> Regards, >>>> >>>> Christer >>>> >>>> >>>> >>>> >>>> >>>> On Fri, Jul 21, 2017 at 12:33 PM, Christer Holmberg < >>>> christer.holmberg@ericsson.com> wrote: >>>> Hi Bernard, >>>> >>>> Support of 5245bis is also negotiated, using the “ice2” ICE option. >>>> >>>> Also, my understanding was that endpoints supporting RFC 7675 might >>>> maintain consent on pairs currently not used for media, in order to be able >>>> to re-nominate in case consent for the currently nominated pair expires. >>>> However, RFC 7675 does not explicitly say anything about that. >>>> >>>> Regards, >>>> >>>> Christer >>>> >>>> From: Ice [mailto:ice-bounces@ietf.org] On Behalf Of Bernard Aboba >>>> Sent: 20 July 2017 14:22 >>>> To: ice@ietf.org >>>> Subject: [Ice] Re-nomination and candidate pair switching in RFC 5245bis >>>> >>>> During the ICE WG meeting today, there was discussion of whether >>>> RFC5245bis should indicate that it is possible to re-nominate pairs >>>> (proposed by Peter), or whether it is possible to switch from one interface >>>> to another (Cullen). While these capabilities are desirable, attempting to >>>> add them to RFC 5245bis without negotiation has the potential to break >>>> interoperability with existing RFC 5245 implementations. >>>> >>>> In my experience, this is an area where RFC 5245 implementations have >>>> very different interpretations. For example, some implementations (e.g. >>>> ones that did not support aggressive) discard non-selected candidate pairs >>>> after nomination. These implementations (e.g. particularly ones included in >>>> previous product releases) cannot be assumed to change their behavior after >>>> RFC 5245bis is published. This raises the possibility that that >>>> interoperability could be impacted. >>>> >>>> Since in practice the desired candidate pair switching capabilities are >>>> most likely to be supported in WebRTC implementations supporting Trickle >>>> ICE, my recommendation is to think of candidate pair switching as a Trickle >>>> ICE capability. Since Trickle-ICE support is negotiated, clarifications >>>> relating to candidate-pair switching can be linked to that negotiation. >>>> >>>> This provides a potential way forward that bypasses potential >>>> interoperability issues. For example, if text on candidate-pair switching >>>> is to be added to (either to RFC 5245bis or Trickle-ICE) then the text >>>> could say that support for these behaviors can only be assumed if they are >>>> explicitly negotiated. The Trickle-ICE document could then create normative >>>> requirements for support of the new behaviors by stating that support for >>>> them is mandatory when supporting full-Trickle. >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> Ice mailing list >>>> Ice@ietf.org >>>> https://www.ietf.org/mailman/listinfo/ice >>>> >>> >>
- Re: [Ice] Re-nomination and candidate pair switch… Christer Holmberg
- Re: [Ice] Re-nomination and candidate pair switch… Bernard Aboba
- Re: [Ice] Re-nomination and candidate pair switch… Christer Holmberg
- Re: [Ice] Re-nomination and candidate pair switch… Peter Thatcher
- Re: [Ice] Re-nomination and candidate pair switch… Bernard Aboba
- Re: [Ice] Re-nomination and candidate pair switch… Peter Thatcher
- Re: [Ice] Re-nomination and candidate pair switch… Peter Thatcher
- Re: [Ice] Re-nomination and candidate pair switch… Bernard Aboba
- Re: [Ice] Re-nomination and candidate pair switch… Peter Thatcher
- Re: [Ice] Re-nomination and candidate pair switch… Bernard Aboba
- Re: [Ice] Re-nomination and candidate pair switch… Christer Holmberg
- Re: [Ice] Re-nomination and candidate pair switch… Bernard Aboba
- Re: [Ice] Re-nomination and candidate pair switch… Christer Holmberg
- Re: [Ice] Re-nomination and candidate pair switch… Christer Holmberg
- Re: [Ice] Re-nomination and candidate pair switch… Bernard Aboba
- Re: [Ice] Re-nomination and candidate pair switch… Christer Holmberg
- Re: [Ice] Re-nomination and candidate pair switch… Christer Holmberg
- Re: [Ice] Re-nomination and candidate pair switch… Bernard Aboba
- Re: [Ice] Re-nomination and candidate pair switch… Christer Holmberg
- Re: [Ice] Re-nomination and candidate pair switch… Peter Thatcher
- Re: [Ice] Re-nomination and candidate pair switch… Peter Saint-Andre
- Re: [Ice] Re-nomination and candidate pair switch… Nils Ohlmeier