Re: [weirds] Entity name searches

Dave Piscitello <> Mon, 09 November 2015 20:45 UTC

Return-Path: <>
Received: from localhost ( []) by (Postfix) with ESMTP id 564F41B8404 for <>; Mon, 9 Nov 2015 12:45:46 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -3.431
X-Spam-Status: No, score=-3.431 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_NEUTRAL=0.779, T_RP_MATCHES_RCVD=-0.01] autolearn=ham
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id hSuZr5oIc2Ey for <>; Mon, 9 Nov 2015 12:45:41 -0800 (PST)
Received: from ( []) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 68CD51B83FE for <>; Mon, 9 Nov 2015 12:45:40 -0800 (PST)
Received: from ( by ( with Microsoft SMTP Server (TLS) id 15.0.1044.25; Mon, 9 Nov 2015 12:45:38 -0800
Received: from ([]) by PMBX112-W1-CA-1.PEXCH112.ICANN.ORG ([]) with mapi id 15.00.1044.021; Mon, 9 Nov 2015 12:45:38 -0800
From: Dave Piscitello <>
To: Andrew Newton <>
Thread-Topic: [weirds] Entity name searches
Thread-Index: AQHRGNV/4iTet5vJAke73jAZnpXocZ6QDfAAgARqPACAAAj1gIAALtUAgAADnoA=
Date: Mon, 9 Nov 2015 20:45:37 +0000
Message-ID: <>
References: <> <> <> <> <>
In-Reply-To: <>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
x-pgp-agent: GPGMail 2.5.1
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: []
Content-Type: multipart/signed; boundary="Apple-Mail=_FCB7ED55-2FEA-42BD-B418-A7C1A500C7D6"; protocol="application/pgp-signature"; micalg=pgp-sha512
MIME-Version: 1.0
Archived-At: <>
Cc: Brian Mountford <>, Justine Tunney <>, "" <>
Subject: Re: [weirds] Entity name searches
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "WHOIS-based Extensible Internet Registration Data Service \(WEIRDS\)" <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Mon, 09 Nov 2015 20:45:46 -0000

Can we clear up one question before we venture into extensions?

Q: What is the purpose of such an extension versus the ability that is naturally available if one is able to collect structured data into a local data store?

For my example, so long as I’m able to gather the 100,000 structured records, I and others will happily take structured data, incorporate these into SQL or other databases, and search to our hearts’ content. What has inhibited the kinds of _domain name_ investigations my example illustrates in the past was the combination of non-standard, unstructured data, rate limiting, and other completely non-RDAP related matters.

Perhaps I’m missing something, but the extension(s) required to do what I suggested on a query level would require federated {authentication,authorization,auditing… } that would be quite remarkable objective for the numbers world and a monumental undertaking for the domain world, no?

> On Nov 9, 2015, at 3:32 PM, Andrew Newton <> wrote:
> Ok. I understand the use case. It still requires an extension to RDAP.
> That type of search never came up during the standardization process.
> At ARIN we do offer more targeted RESTful searches, but it has been my
> experience they are used almost exclusively for data mining. And I
> suspect that if left wide open, that's how it would be used in RDAP
> for DNRs. So to get a "system" to work, there would need to be an
> authorization system... and since we are talking multiple DNRs, that
> would be federated authorization. And because there are multiple DNRs,
> these queries would have to be run in parallel against each one (using
> the bootstrap to find them). And then you have the problem that some
> domain registries are thin. Those are a lot of problems to solve, and
> I'm not sure who has the will to do it.
> -andy
> On Mon, Nov 9, 2015 at 12:44 PM, Dave Piscitello
> <> wrote:
>> More generally, if you are looking at 100,000 registration records that have
>> been associated with spam, you’d want the ability to search or pivot on any
>> data/string that establish relationships among the domain strings (e.g.,
>> botnet DGA) but as importantly, _any_ element of the registration record
>> that subsets share: POC, creation date, name server…
>> On Nov 9, 2015, at 12:12 PM, Brian Mountford <> wrote:
>> Well, for instance, one might want to search the organization. When I do a
>> WHOIS query for, the contacts have an organization of Google Inc.
>> I might want to search for all contacts with that organization.
>> I can try to tailor the interpretation, but since the search string syntax
>> does not allow for arbitrary suffix searching, it's not clear what tailoring
>> is possible. Are you saying that I could take an entity name search string
>> of "McB*" and interpret that as a search for names any of whose words begin
>> with McB, so that it would find Joe McBride as well as McBurns Simpson? That
>> seems to be playing pretty loose with the partial search string rules, since
>> the RFC takes pains to define that syntax so precisely.
>> Brian
>> On Fri, Nov 6, 2015 at 4:47 PM, Andrew Newton <> wrote:
>>> Welcome to the world of internationalization, where the concept of a
>>> last name vs a first name is not universal, and US ASCII is not
>>> representative all the characters used.
>>> Since you know the data in your database best, you should taylor the
>>> interpretation of the query input to that which works best with your
>>> registry.
>>> On Sat, Nov 7, 2015 at 5:54 AM, Brian Mountford <>
>>> wrote:
>>>> And only names? There is no provision for searching entities by address,
>>>> etc.?
>>> I don't think that ever came up. It would require an RDAP extension. I
>>> do question how useful such a thing would be. Why does anybody care
>>> that a particular registry has contacts living on Mumford Lane in East
>>> Westover? Are they searching all the registries for that information?
>>> Do they need a telephone book instead?
>>> -andy
>> _______________________________________________
>> weirds mailing list