Re: [DNSOP] New Draft Charter

[Dean Anderson <[e-mail]>]

On Fri, 21 Mar 2008, bill fumerola wrote:


> if i felt the use of anycast by root server operators is causing
> instability, i would take that up with the individual root server
> operator(s) that i was experiencing instability with. but i'm not.

Are you using TCP DNS? Most people don't use TCP DNS. That is changing,
though.  I guess I don't recall who your work for, or what kind of
service you assert to be anycasting. But please do send me the details
of your anycasted service.

Over the last 3 years, a number of reports have come out that indicate
there are indeed stability problems with TCP Anycast.  Mark Kosters
(then at Verisign)  reported that he detected the same IP addresses at
multiple Anycast root DNS servers over short time frames, indicating
either fast path changes or load balancing.  Theory also indicates
(RFC1812) and states explicitly (RFC1546) that TCP Anycast isn't stable
and must be avoided.

The rest of the Anycast users I've come across aren't actually using
anycast, but have confused Anycast (RFC1546) with load balancing, as in
the kind you get from an F5.  Stateful loadbalancing ala F5,Cisco, etc
loadbalancers is not Anycast.

> this From [e-mail]  Tue Mar 25 14:35:48 2008
Return-Path: <[e-mail]>
X-Original-To: [e-mail]
Delivered-To: [e-mail]
Received: from localhost (localhost [127.0.0.1])
	by core3.amsl.com (Postfix) with ESMTP id F2FA03A6B36;
	Tue, 25 Mar 2008 14:35:47 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -101.518
X-Spam-Level: 
X-Spam-Status: No, score=-101.518 tagged_above=-999 required=5
	tests=[AWL=-1.081, BAYES_00=-2.599, FH_RELAY_NODNS=1.451,
	HELO_MISMATCH_ORG=0.611, RDNS_NONE=0.1, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([64.170.98.32])
	by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id VBu1+W+v4eK6; Tue, 25 Mar 2008 14:35:46 -0700 (PDT)
Received: from core3.amsl.com (localhost [127.0.0.1])
	by core3.amsl.com (Postfix) with ESMTP id B34B23A6A45;
	Tue, 25 Mar 2008 14:35:46 -0700 (PDT)
X-Original-To: [e-mail]
Delivered-To: [e-mail]
Received: from localhost (localhost [127.0.0.1])
	by core3.amsl.com (Postfix) with ESMTP id 13D613A6A5E
	for <[e-mail]>; Tue, 25 Mar 2008 14:35:45 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
Received: from mail.ietf.org ([64.170.98.32])
	by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id xzt7pnazD7LL for <[e-mail]>;
	Tue, 25 Mar 2008 14:35:43 -0700 (PDT)
Received: from cirrus.av8.net (cirrus.av8.net [130.105.36.66])
	by core3.amsl.com (Postfix) with ESMTP id A8BD33A6928
	for <[e-mail]>; Tue, 25 Mar 2008 14:35:43 -0700 (PDT)
Received: from citation2.av8.net (citation2.av8.net [130.105.12.10])
	(authenticated bits=0)
	by cirrus.av8.net (8.12.11/8.12.11) with ESMTP id m2PLXK7G002474
	(version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO);
	Tue, 25 Mar 2008 17:33:21 -0400
Date: Tue, 25 Mar 2008 17:33:20 -0400 (EDT)
From: Dean Anderson <[e-mail]>
X-X-Sender: [e-mail]
To: bill fumerola <[e-mail]>
In-Reply-To: <[e-mail]>
Message-ID: <[e-mail]>
MIME-Version: 1.0
Cc: Peter Koch <[e-mail]>, IETF DNSOP WG <[e-mail]>
Subject: Re: [DNSOP] New Draft Charter
X-BeenThere: [e-mail]
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IETF DNSOP WG mailing list <dnsop.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/dnsop>,
	<mailto:[e-mail]?subject=unsubscribe>
List-Archive: <http://www.ietf.org/pipermail/dnsop>
List-Post: <mailto:[e-mail]>
List-Help: <mailto:[e-mail]?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/dnsop>,
	<mailto:[e-mail]?subject=subscribe>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Sender: [e-mail]
Errors-To: [e-mail]

On Fri, 21 Mar 2008, bill fumerola wrote:


> if i felt the use of anycast by root server operators is causing
> instability, i would take that up with the individual root server
> operator(s) that i was experiencing instability with. but i'm not.

Are you using TCP DNS? Most people don't use TCP DNS. That is changing,
though.  I guess I don't recall who your work for, or what kind of
service you assert to be anycasting. But please do send me the details
of your anycasted service.

Over the last 3 years, a number of reports have come out that indicate
there are indeed stability problems with TCP Anycast.  Mark Kosters
(then at Verisign)  reported that he detected the same IP addresses at
multiple Anycast root DNS servers over short time frames, indicating
either fast path changes or load balancing.  Theory also indicates
(RFC1812) and states explicitly (RFC1546) that TCP Anycast isn't stable
and must be avoided.

The rest of the Anycast users I've come across aren't actually using
anycast, but have confused Anycast (RFC1546) with load balancing, as in
the kind you get from an F5.  Stateful loadbalancing ala F5,Cisco, etc
loadbalancers is not Anycast.

> this WG certainly doesn't have the authority to directly instruct
> individual root server operators how to configure their networks.

It does, (or perhaps "it did").  Previously, this WG had authority over
RFC2870. RFC2870 does give direct instructions for root server operators
on how to e.g., configure their networks, physical security, and a lot
of other things. The ICANN/IANA received this technical advice from the
IETF, and now imposes it on root server operators. I previouly opposed
removing the root server operations element from its charter. But now I
think this WG and the IETF shouldn't have that charge.

		--Dean

P.S.

The rest is about history or what ought to be. But I don't think the
future of root DNS server management ought to be the subject of this
working group, so just stop reading now unless you like history.

> IANA could intervene. however, i would imagine IANA would want to see
> agreement amongst multiple respected experts in the areas of routing and
> dns operations. they'd need proof that your claims of root server
> instability as a result of the use of anycast.

Actually, it seems odd that anycast operations were allowed to begin
without proof of stability in the first place.  I've read all the
scientific research on Anycast. All but one paper are non-commital or
didn't address stateful anycast. These are all after 2002.  There is
nothing from 2002 or before when Anycast was first deployed.

The root server operators seem to (or ought to) have the obligation to
prove stability, both in theory and in experiment before they undertake
risky changes to the architecture of the internet.

Indeed, this seems to be another reason to update RFC2870, if it truly
allows root server operators to make such risky and architectural
changes without proof of stability.

> given the previous results of threads started/hijacked by you for this
> purpose in this WG, i'm not sure what sort of verbage in the charter
> would grant you the soapbox on top of which you wish to preach.

Perhaps then you support having root DNS operators give me NSID keys so
that I can do anycast testing on roots, directly. My tests so far have
been indirect, or else not on roots. But of the tests so far, I can
detect anycast'd recursors on the internet.  So indirectly, I have
proven the Anycast DNS Instability by experiment.

[I did so by getting different packets returned from each (several
caches each cache a different response---I just wrote a special server
and a client to keep track of each unique answer sent and the responses
to the queries. When you see the different and previous answers from the
same IP address, there are multiple caches and anycast has been
detected.  TCP packets in the same situation would result in a
connection failure.]

> if you want to prove anycast [in]stability i'd suggest submitting and
> presenting a technical paper to a respected org (USENIX comes to mind,
> there are others). even within the IETF, you don't even need a WG
> charter to submit an internet draft. individual submissions are made
> all the time. often, a WG picks them up after sufficient discussion
> and review.

Vixie and companies have prevented the IETF from hearing about Anycast.  
Just 2 years ago, I was PR-actioned for talking about Anycast on this
very working group by a conflicted A.D. (This is still the subject of
ongoing legal actions, so I can't say just too much, but we can't forget
the history)

> reasonable minds cannot ignore solid evidence. surely every avenue
> (many exist beyond this WG) hasn't been so jaded by past experiences
> as to reject your valid technical arguments, if they exist.

Indeed. It must be the case that either the minds are unreasonable or
else there is no evidence.  There is substantial evidence in anecdote,
theory, and experiment. So now why are the minds unreasonable?  Hmm?
Money is known to make people act that way. Could it be that people make
money on Anycast?  Surely not here! Surely there is no bad faith and
conflict of interest!

> -- bill
> 
> [ full disclosure: i work for a company that uses anycWG certainly doesn't have the authority to directly instruct
> individual root server operators how to configure their networks.

It does, (or perhaps "it did").  Previously, this WG had authority over
RFC2870. RFC2870 does give direct instructions for root server operators
on how to e.g., configure their networks, physical security, and a lot
of other things. The ICANN/IANA received this technical advice from the
IETF, and now imposes it on root server operators. I previouly opposed
removing the root server operations element from its charter. But now I
think this WG and the IETF shouldn't have that charge.

		--Dean

P.S.

The rest is about history or what ought to be. But I don't think the
future of root DNS server management ought to be the subject of this
working group, so just stop reading now unless you like history.

> IANA could intervene. however, i would imagine IANA would want to see
> agreement amongst multiple respected experts in the areas of routing and
> dns operations. they'd need proof that your claims of root server
> instability as a result of the use of anycast.

Actually, it seems odd that anycast operations were allowed to begin
without proof of stability in the first place.  I've read all the
scientific research on Anycast. All but one paper are non-commital or
didn't address stateful anycast. These are all after 2002.  There is
nothing from 2002 or before when Anycast was first deployed.

The root server operators seem to (or ought to) have the obligation to
prove stability, both in theory and in experiment before they undertake
risky changes to the architecture of the internet.

Indeed, this seems to be another reason to update RFC2870, if it truly
allows root server operators to make such risky and architectural
changes without proof of stability.

> given the previous results of threads started/hijacked by you for this
> purpose in this WG, i'm not sure what sort of verbage in the charter
> would grant you the soapbox on top of which you wish to preach.

Perhaps then you support having root DNS operators give me NSID keys so
that I can do anycast testing on roots, directly. My tests so far have
been indirect, or else not on roots. But of the tests so far, I can
detect anycast'd recursors on the internet.  So indirectly, I have
proven the Anycast DNS Instability by experiment.

[I did so by getting different packets returned from each (several
caches each cache a different response---I just wrote a special server
and a client to keep track of each unique answer sent and the responses
to the queries. When you see the different and previous answers from the
same IP address, there are multiple caches and anycast has been
detected.  TCP packets in the same situation would result in a
connection failure.]

> if you want to prove anycast [in]stability i'd suggest submitting and
> presenting a technical paper to a respected org (USENIX comes to mind,
> there are others). even within the IETF, you don't even need a WG
> charter to submit an internet draft. individual submissions are made
> all the time. often, a WG picks them up after sufficient discussion
> and review.

Vixie and companies have prevented the IETF from hearing about Anycast.  
Just 2 years ago, I was PR-actioned for talking about Anycast on this
very working group by a conflicted A.D. (This is still the subject of
ongoing legal actions, so I can't say just too much, but we can't forget
the history)

> reasonable minds cannot ignore solid evidence. surely every avenue
> (many exist beyond this WG) hasn't been so jaded by past experiences
> as to reject your valid technical arguments, if they exist.

Indeed. It must be the case that either the minds are unreasonable or
else there is no evidence.  There is substantial evidence in anecdote,
theory, and experiment. So now why are the minds unreasonable?  Hmm?
Money is known to make people act that way. Could it be that people make
money on Anycast?  Surely not here! Surely there is no bad faith and
conflict of interest!

> -- bill
> 
> [ full disclosure: i work for a company that uses anycast for dns and
> 		   other services, without instability. if anything, it
> 		   is in my best interests to know about a wolf at the
> 		   door from using anycast. i've just never heard anything
> 		   more than loud barking.

Hmm. Interesting metaphor. Perhaps it is one's interests to ignore the
wolf while one profits from the door being open. I guess that case, one
must be a coyote or some other predator to make the metaphor work.


-- 
Av8 Internet   Prepared to pay a premium for better service?
www.av8.net         faster, more reliable, better service
617 344 9000   


_______________________________________________
DNSOP mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/dnsop


ast for dns and
> 		   other services, without instability. if anything, it
> 		   is in my best interests to know about a wolf at the
> 		   door from using anycast. i've just never heard anything
> 		   more than loud barking.

Hmm. Interesting metaphor. Perhaps it is one's interests to ignore the
wolf while one profits from the door being open. I guess that case, one
must be a coyote or some other predator to make the metaphor work.


-- 
Av8 Internet   Prepared to pay a premium for better service?
www.av8.net         faster, more reliable, better service
617 344 9000   


_______________________________________________
DNSOP mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/dnsop