RE: DoS attack ?

Nicolas Popp <nico@realnames.com> Fri, 07 December 2001 17:26 UTC

Return-Path: <ietf-irnss-errors@lists.elistx.com>
Received: from ELIST-DAEMON.eListX.com by eListX.com (PMDF V6.0-025 #44856) id <0GNZ00504HSI3A@eListX.com> (original mail from nico@realnames.com); Fri, 07 Dec 2001 12:26:43 -0500 (EST)
Received: from CONVERSION-DAEMON.eListX.com by eListX.com (PMDF V6.0-025 #44856) id <0GNZ00501HSI38@eListX.com> for ietf-irnss@elist.lists.elistx.com (ORCPT ietf-irnss@lists.elistx.com); Fri, 07 Dec 2001 12:26:42 -0500 (EST)
Received: from DIRECTORY-DAEMON.eListX.com by eListX.com (PMDF V6.0-025 #44856) id <0GNZ00501HSH37@eListX.com> for ietf-irnss@elist.lists.elistx.com (ORCPT ietf-irnss@lists.elistx.com); Fri, 07 Dec 2001 12:26:41 -0500 (EST)
Received: from friendly.realnames.com (friendly.realnames.com [63.251.238.102]) by eListX.com (PMDF V6.0-025 #44856) with SMTP id <0GNZ0036EHSG8N@eListX.com> for ietf-irnss@lists.elistx.com; Fri, 07 Dec 2001 12:26:41 -0500 (EST)
Received: (qmail 23940 invoked by uid 104); Fri, 07 Dec 2001 17:24:09 +0000
Received: from nico@realnames.com by friendly.realnames.com with qmail-scanner-0.96 (. Clean. Processed in 0.833626 secs); Fri, 07 Dec 2001 17:24:09 +0000
Received: from heaven.internal.realnames.com (10.1.5.39) by friendly.realnames.com with SMTP; Fri, 07 Dec 2001 17:24:08 +0000
Received: From RINCON.INTERNAL.REALNAMES.COM (10.1.5.99[10.1.5.99 port:3677]) by heaven.internal.realnames.com Mail essentials (server 2.422) with SMTP id: <164841@heaven.internal.realnames.com> for <newcat@spsoft.co.kr>; Fri, 07 Dec 2001 09:20:34 +0000 (AM)
Received: by rincon.centraal.com with Internet Mail Service (5.5.2653.19) id <XHJ5G7NZ>; Fri, 07 Dec 2001 09:23:51 -0800
Date: Fri, 07 Dec 2001 09:22:30 -0800
From: Nicolas Popp <nico@realnames.com>
Subject: RE: DoS attack ?
To: Nicolas Popp <nico@realnames.com>, =?iso-8859-1?Q?=27Patrik_F=E4lts?= =?iso-8859-1?Q?tr=F6m=27?= <paf@cisco.com>, 'John C Klensin' <klensin@jck.com>, 'YangWoo Ko' <newcat@spsoft.co.kr>
Cc: "'ietf-irnss@lists.elistx.com'" <ietf-irnss@lists.elistx.com>
Message-id: <7FC3066C236FD511BC5900508BAC86FE4364E0@trestles.internal.realnames.com>
MIME-version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-type: text/plain; charset=iso-8859-1
Content-transfer-encoding: QUOTED-PRINTABLE
List-Owner: <mailto:ietf-irnss-help@lists.elistx.com>
List-Post: <mailto:ietf-irnss@lists.elistx.com>
List-Subscribe: <http://lists.elistx.com/ob/adm.pl>, <mailto:ietf-irnss-request@lists.elistx.com?body=subscribe>
List-Unsubscribe: <http://lists.elistx.com/ob/adm.pl>, <mailto:ietf-irnss-request@lists.elistx.com?body=unsubscribe>
List-Archive: <http://lists.elistx.com/archives/ietf-irnss>
List-Help: <http://lists.elistx.com/elists/admin.shtml>, <mailto:ietf-irnss-request@lists.elistx.com?body=help>
List-Id: <ietf-irnss.lists.elistx.com>

By the way, that's why CNRP has a "range" property.
It also has Referrals (not the main driver though).

-Nico

-----Original Message-----
From: Nicolas Popp 
Sent: Friday, December 07, 2001 9:10 AM
To: 'Patrik Fältström'; 'John C Klensin'; YangWoo Ko
Cc: ietf-irnss@lists.elistx.com
Subject: RE: DoS attack ?



>(1)One practical path is to give in the protocol a way for the server to
>say "I'm sorry 

>(2) As soon as you do "paged results", you force the server to keep state.

No. You simply refetch (the referral query sends me back all the facets and
the new range). All large search engines are stateless and handle paged
results. Their caching strategies is typically based on query keyword
frequency (the cache is built as indexing time based on the query
distribution, more than keeping the "last requests" (they actually do a
little bit of that too). Since they are stateless, their response is
consistent with the query (at any time their return the set of results for
the query within the specified range, but that set is not immutable. The
fact that the index is slowing changing makes that strategy acceptable from
a user perspective).

Having said that I don't disagree with (1). I just think you need to do
both.

-Nico



-----Original Message-----
From: Patrik Fältström [mailto:paf@cisco.com]
Sent: Thursday, December 06, 2001 10:34 PM
To: Nicolas Popp; 'John C Klensin'; YangWoo Ko
Cc: ietf-irnss@lists.elistx.com
Subject: RE: DoS attack ?


--On 01-12-06 10.09 -0800 Nicolas Popp <nico@realnames.com> wrote:

> You can also do what most search engines would.
> You return a (small) range of ranked results in the set of results and
> your last result is a referral back to you for the next range in the
> set...Then you try to detect automated crawlers that recursively follow
> the referrals and slow them down to a halt.
> 
> So, just from that standpoint, it could be useful for the protocol to
> support the notion of results set range (query) as well as referral
> (response).

We have been through this when looking at other protocols....and I would
urge you to learn from earlier mistakes (and successes).

(1) One practical path is to give in the protocol a way for the server to
say "I'm sorry, but I will not do that operation you requested. Instead I
did the following". This generic response can be "you only got 10 records
even though the result set is larger".

(2) As soon as you do "paged results", you force the server to keep state.
Depending on whether the protocol is stateful or stateless, it is harder or
easier for the server to know when to remove the cached search. Further, as
soon as you start doing pages results, you end up getting problems with
sorting the result, handling of database changes between the two fetches
(i.e. can the server re-issue the query for the second fetch, or do the
server really have to cache the result set and return the second part at
the second fetch) and million of other problems.

So, my suggestion is "don't go there".

    paf