IETF Logo Mail Archive

Re: [weirds] [Regops] Search Engines Indexing RDAP Server Content

"Hollenbeck, Scott" <shollenbeck@verisign.com> Fri, 29 January 2016 16:54 UTC

Return-Path: <shollenbeck@verisign.com>
X-Original-To: weirds@ietfa.amsl.com
Delivered-To: weirds@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 18F3E1A87C8 for <weirds@ietfa.amsl.com>; Fri, 29 Jan 2016 08:54:41 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level:
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WeCkCXqGgkHI for <weirds@ietfa.amsl.com>; Fri, 29 Jan 2016 08:54:39 -0800 (PST)
Received: from mail-qg0-x262.google.com (mail-qg0-x262.google.com [IPv6:2607:f8b0:400d:c04::262]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 44C901A87C7 for <weirds@ietf.org>; Fri, 29 Jan 2016 08:54:39 -0800 (PST)
Received: by mail-qg0-x262.google.com with SMTP id i20so4605300qgd.2 for <weirds@ietf.org>; Fri, 29 Jan 2016 08:54:39 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=verisign-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:thread-topic:thread-index:date:message-id :references:in-reply-to:accept-language:content-language :content-type:content-transfer-encoding:mime-version; bh=p5eAeop13g9YIhieSsn2DZ9fHAlvQTFt8vXTzXkte/M=; b=uIzCbPajVZ8FDQV/RVVnXCzAwhi54UOl9aFSoq8NnVGytoSY+a7xmAoCLdk0L5m/tq Myh0SAPPMJzrD+69elUmkh8zEVcMKFQj1n9MHfT17aueiC1gUrTJSK3TRML5tpl730Ca iSxpwVcAkp2yiZhCAUTsW/i4xhxB7prBSJvTOqZFMtKAQxed2N/+LPxCppmPGFAz4f1N nH6tyu6MAFR7KmhhMcP9ffeOUc5tEDnroHEzahKUHPORuo53osguJwWtzfbRA0lCUwg+ jGVAz5IbO/XmARczNXXFDAo1kjgMtI6L8+Bz4C9u9r1GZXLBE8Hbb1bPq/jR3OYZ6pD6 uIOg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:thread-topic:thread-index:date :message-id:references:in-reply-to:accept-language:content-language :content-type:content-transfer-encoding:mime-version; bh=p5eAeop13g9YIhieSsn2DZ9fHAlvQTFt8vXTzXkte/M=; b=Q4z3/MilOzmovTafJJRP3l5wJCtzwCECS5CeRhpb1uJVRMzCPwEMq8HplVOgzw96gX ogLc5ymUh/Nx6naWaG2Oskvuz5aeYMFMgqbGhKCOyyACQqFIJpxkI4joAJO8MOKxRV76 bwWNx+wgKIWYWjifwOGM4wCqPxh446uzVfZtbCARRiH2v6O09b3Oey0+F5rONhAI/J4+ nOsLQW744L1r851OIIg6EYZGKApokPod6QQGS7/Po8vsm0UoXAUTgPbP0wxwQO7xXCQs REwOgUGekXYOKid0lJNliJQik6Xw2Rd+1XkrbsHLDDZbJD2drIg4vu9L2k64gjsKbwSb TRiw==
X-Gm-Message-State: AG10YOSTVwkzJjTdy7bAEbTeBrePdhd2PtiToGZV1ROwsnAsivIR8UGubwrSGzyMU9C7eURPbrCLACtVccQLAUlnXsJ6tiUG
X-Received: by 10.55.56.134 with SMTP id f128mr11782201qka.92.1454086478404; Fri, 29 Jan 2016 08:54:38 -0800 (PST)
Received: from brn1lxmailout01.verisign.com (brn1lxmailout01.verisign.com. [72.13.63.41]) by smtp-relay.gmail.com with ESMTPS id d123sm1486056qka.12.2016.01.29.08.54.38 (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 29 Jan 2016 08:54:38 -0800 (PST)
X-Relaying-Domain: verisign.com
Received: from BRN1WNEXCHM01.vcorp.ad.vrsn.com (brn1wnexchm01 [10.173.152.255]) by brn1lxmailout01.verisign.com (8.13.8/8.13.8) with ESMTP id u0TGsbS9010103 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Fri, 29 Jan 2016 11:54:37 -0500
Received: from BRN1WNEXMBX01.vcorp.ad.vrsn.com ([::1]) by BRN1WNEXCHM01.vcorp.ad.vrsn.com ([::1]) with mapi id 14.03.0174.001; Fri, 29 Jan 2016 11:54:35 -0500
From: "Hollenbeck, Scott" <shollenbeck@verisign.com>
To: John R Levine <johnl@taugh.com>, "weirds@ietf.org" <weirds@ietf.org>
Thread-Topic: [weirds] [Regops] Search Engines Indexing RDAP Server Content
Thread-Index: AQHRWrTPLeNZLPJOwEOzSCpB2y+X758StVFw
Date: Fri, 29 Jan 2016 16:54:35 +0000
Message-ID: <831693C2CDA2E849A7D7A712B24E257F4A14C08E@BRN1WNEXMBX01.vcorp.ad.vrsn.com>
References: <alpine.OSX.2.11.1601291146570.26863@ary.lan>
In-Reply-To: <alpine.OSX.2.11.1601291146570.26863@ary.lan>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [10.173.152.4]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Archived-At: <http://mailarchive.ietf.org/arch/msg/weirds/kWv0wI3QYfdfrjyAfLlYQrRDS_c>
Subject: Re: [weirds] [Regops] Search Engines Indexing RDAP Server Content
X-BeenThere: weirds@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "WHOIS-based Extensible Internet Registration Data Service \(WEIRDS\)" <weirds.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/weirds>, <mailto:weirds-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/weirds/>
List-Post: <mailto:weirds@ietf.org>
List-Help: <mailto:weirds-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/weirds>, <mailto:weirds-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 29 Jan 2016 16:54:41 -0000

> -----Original Message-----
> From: weirds [mailto:weirds-bounces@ietf.org] On Behalf Of John R
> Levine
> Sent: Friday, January 29, 2016 11:48 AM
> To: weirds@ietf.org
> Subject: Re: [weirds] [Regops] Search Engines Indexing RDAP Server
> Content
> 
> > So I saw a tweet from Gavin Brown (@GavinBrown) that describes how
> one
> > particular search engine has indexed the RDAP server of a gTLD
> registry
> > operator:

[snip]

> The big search engines spider at low speed from hosts all over the
> world
> to avoid overloading the sites they index.  You're not going to keep
> them
> out via authentication without also keeping out everyone else who
> doesn't
> have a password.  I don't think that's a good idea.

We have a difference of opinion, because I tend to think that it's a *good* idea to restrict access to *some* data based on a client's identify and level of authorization. I do not want *my* PII appearing in search engine search results because it's accessible via RDAP.

Scott

v2.23.0 | Report a Bug | By Email