Re: [OPSEC] Fwd: Last Call for Comments:draft-oreirdan-mody-bot-remediation-06

"Smith, Donald" <Donald.Smith@qwest.com> Wed, 21 April 2010 16:58 UTC

Return-Path: <Donald.Smith@qwest.com>
X-Original-To: opsec@core3.amsl.com
Delivered-To: opsec@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 289A128C1AA for <opsec@core3.amsl.com>; Wed, 21 Apr 2010 09:58:19 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.834
X-Spam-Level:
X-Spam-Status: No, score=-0.834 tagged_above=-999 required=5 tests=[AWL=-0.835, BAYES_50=0.001]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ByHr5YTMlmsb for <opsec@core3.amsl.com>; Wed, 21 Apr 2010 09:58:15 -0700 (PDT)
Received: from suomp64i.qwest.com (suomp64i.qwest.com [155.70.16.237]) by core3.amsl.com (Postfix) with ESMTP id CEAA828C211 for <opsec@ietf.org>; Wed, 21 Apr 2010 09:39:27 -0700 (PDT)
Received: from suomp61i.qintra.com (suomp61i.qintra.com [151.117.69.28]) by suomp64i.qwest.com (8.14.4/8.14.4) with ESMTP id o3LGdE9c029246 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Wed, 21 Apr 2010 11:39:14 -0500 (CDT)
Received: from qtdenexhtm20.AD.QINTRA.COM (localhost [127.0.0.1]) by suomp61i.qintra.com (8.14.4/8.14.4) with ESMTP id o3LGd8jD015423; Wed, 21 Apr 2010 11:39:09 -0500 (CDT)
Received: from qtdenexmbm24.AD.QINTRA.COM ([151.119.91.226]) by qtdenexhtm20.AD.QINTRA.COM ([151.119.91.229]) with mapi; Wed, 21 Apr 2010 10:39:08 -0600
From: "Smith, Donald" <Donald.Smith@qwest.com>
To: 'Jason Livingood' <jason_livingood@cable.comcast.com>, 'Joel Jaeggli' <joelja@bogus.com>, 'Danny McPherson' <danny@tcb.net>
Date: Wed, 21 Apr 2010 10:39:06 -0600
Thread-Topic: [OPSEC] Fwd: Last Call for Comments:draft-oreirdan-mody-bot-remediation-06
Thread-Index: Acqtx42/UyVFz2xtQbKi5RBxrqoIrwAomV5oDL5bnCQAAMqW4A==
Message-ID: <B01905DA0C7CDC478F42870679DF0F1007B32A90B8@qtdenexmbm24.AD.QINTRA.COM>
References: <B01905DA0C7CDC478F42870679DF0F100787A47F9A@qtdenexmbm24.AD.QINTRA.COM> <C7F48B4D.21D25%jason_livingood@cable.comcast.com>
In-Reply-To: <C7F48B4D.21D25%jason_livingood@cable.comcast.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Cc: 'Mike O'Reirdan' <michael_oreirdan@cable.comcast.com>, 'opsec wg mailing list' <opsec@ietf.org>, 'Nirmal Mody' <nirmal_mody@cable.comcast.com>
Subject: Re: [OPSEC] Fwd: Last Call for Comments:draft-oreirdan-mody-bot-remediation-06
X-BeenThere: opsec@ietf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: opsec wg mailing list <opsec.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/listinfo/opsec>, <mailto:opsec-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/opsec>
List-Post: <mailto:opsec@ietf.org>
List-Help: <mailto:opsec-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/opsec>, <mailto:opsec-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Apr 2010 16:58:19 -0000

Thanks Jason, a few minor comments below.
Mostly I agreed with your changes.
Danny there is one comment aimed at you wrt the SP survey you do every year.


(coffee != sleep) & (!coffee == sleep)
Donald.Smith@qwest.com gcia

> -----Original Message-----
> From: Jason Livingood [mailto:jason_livingood@cable.comcast.com]
> Sent: Wednesday, April 21, 2010 9:00 AM
> To: Smith, Donald; Joel Jaeggli; Danny McPherson
> Cc: Nirmal Mody; opsec wg mailing list; Mike O'Reirdan
> Subject: Re: [OPSEC] Fwd: Last Call for
> Comments:draft-oreirdan-mody-bot-remediation-06
>
> I'm just getting around to doing a -08 document update now
> that IETF-77 has concluded.  (Joel's feedback has been
> incorporated into -08 as well.)
>
> Thanks for this feedback, now incorporated into
> draft-oreirdan-mody-bot-remediation-08 (
> http://www.ietf.org/id/draft-oreirdan-mody-bot-remediation-08.txt).
>
> Specific comments/replies inline below if you are so inclined.
>
> Regards
> Jason
>
> On 2/15/10 1:16 PM, "Smith, Donald" <Donald.Smith@qwest.com> wrote:
>
>
>
>       >    This document contains recommendations on how
> Internet Service
>       >    Providers can manage the effects of computers used by their
>       * subscribers, which have been infected with malicious bots, via
>
>       donald: A bot is a tool. It can't really be malicious
> nor benign any more that a gun, or screwdriver is malicious
> however I understand as this is widely used, incorrectly,
> within the industry. But malicious implies intent to do harm.
>       As long as that is addressed in the malicious bot/bot
> definition it's probably acceptable to use it in this way.
>
>       [JL] Quite true.  I have tried to ensure when using
> bots in the doc that I am referring to malicious bots, rather
> than other bots which may have positive uses.  I try to
> define this at length in Terminology, Section 2.1.  (I
> received similar feedback from Robb Topolski in an earlier draft.)
>
>       >    various remediation techniques.  Internet users
> with infected
>       >    computers are exposed to risks such as loss of
> personal data, as well
>       >    as increased susceptibility to online fraud and/or
> phishing.  Such
>       >    computers can also become an inadvertent
> participant in or component
>       >    of an online crime network, spam network, and/or
> phishing network, as
>       >    well as be used as a part of a distributed denial
> of service attack.
>
>       donald: The risks to users of infected systems and
> others are included/defined in many places within this document.
>       It should probably be defined clearly in one place and
> referenced in all other places.
>
>       [JL] Fair point.  I'm going to call that out in my open
> issues list and address in a future version if I don't have
> the chance to get it into -08.
>
>       > 2.1.  Malicious Bots, or Bots
>       >
>       >    A malicious "bot" (derived from the word "robot",
> hereafter simply
>
>       donald: Malicious bot implies intent to do harm, while
> these are really bots that can be used to perform potentially
> harmful tasks for the bot master. Perhaps potentially harmful bot?
>
>       [JL] I've added a bunch of new text to 2.1 to address
> this.  Please let me know if there's anything that could be
> improved upon.
>
>       >    referred to as a "bot") refers to a program that
> is surreptitiously
>
>       donald: Some times they are installed on purpose but
> the user doesn't understand the full functionality of the
> software installed or the amount of remote control the
> application provides to the remote controller (bot master)
>
>       [JL] Ack, and addressed in 2.1 of the doc.
>
>       >    It is important to note that there are 'good', or
> benign bots.  Such
>       >    benign bots are often found in such environments
> such as gaming and
>       >    Internet Relay Chat (IRC) [RFC1459], where a
> continual, interactive
>       >    presence can be a requirement for participating in
> the games,
>       >    interacting with a computing resource, or other
> purposes.  Since such
>       >    benign bots are performing useful, lawful, and
> non-disruptive
>       >    functions, there is no reason for a provider to
> monitor for their
>       >    presence and/or alert users to their presence.
>
>       donald: Again Benign implies intent. Perhaps harmless instead?
>
>       [JL] I've retained benign since it seems to be used a
> great deal but added harmless as well: "Thus, while there may
> be benign, or harmless bots, for the purposes of this document..."
>
>       >    include: identity theft, spam, distributed denial
> of service (DDoS)
>       >    attacks, key-logging, fraudulent DNS (pharming),
> proxy services, fast
>       * flux hosting and click fraud.
>
>       donald: This is a pretty good risk identification. You
> might want to add spim, spit, email address harvesting,
> hosting of highly questionable or illegal content
> (BulletProof hosting), various MITM attacks (banking ssl ala
> Zeus) and web site scraping.
>
>       [JL] Added - good suggestions.
>
>       >
>       >    Infection vectors include un-patched operating
> systems, software
>       >    vulnerabilities, weak/non-existent passwords,
> malicious websites, un-
>       >    patched browsers, malware, vulnerable helper
> applications and social
>       >        engineering techniques to gain access to the
> user's computer.
>
>       donald: The use of vulnerable and un-patched seems
> slightly redundant.
>       An un-patched piece of software is likely to be vulnerable.
>       I think your trying to cover the concept of un-patched
> vs zero day (no patch publicly available) exploitation.
>
>       [JL] Correct. I modified to "... software
> vulnerabilities (which include so-called zero-day
> vulnerabilities where no patch yet exists)"
>
>       >         Initially, bots used IRC to communicate but
> were easy to shutdown if
>
>       donald: Initally SOME bots used IRC, Q, TFN, Trinoo,
> etc... all had non-IRC based control.
>
>       [JL] Modified as suggested
>
>       >    Increasingly, other household systems and devices
> contain embedded
>       >    computers which are connected to or can connect to
> the public
>       >    Internet and/or private IP networks.  However,
> these devices may not
>       >    be under interactive control of the Internet user,
> such as may be the
>       >    case with various smart home and smart grid devices.
>
>       donald: So do you intend for your computer definition
> to include smart ip enabled  devices or not. It's hard to
> tell from this definitation.
>       I would limit the definition to computers that humans
> use as general purpose computers.
>
>       [JL] I did intend to refer to these sorts of devices.
> As I have generalized "computers" to now be "hosts" I've made
> some modifications to make this hopefully more clear in the
> next version.  But I expressly want to include both typical
> PC-type devices as well as, say, Internet-connected picture frames.
>
>       >    can sometimes cause their computer to be infected
> with malware, which
>       >    may include a bot or cause a bot to install
> itself, via inadvertently
>       >    accessing a specific website, downloading a
> specific file, or other
>
>       donald: a website, downloading a file ...
>
>       [JL] Change made
>
>       >    Alternatively, Internet-connected computers may
> become infected with
>       donald: Not really alternatively, since you mention
> vulnerabilities above but I think your trying to say without
> user interaction (scan and spolit) right?
>
>       [JL] Correct.  Changed "Alternatively" to "In other cases"
>
>       >    DNS Fast Flux is typically used in conjunction
> with proxies which
>       >    then route the web requests to the real host which
> serves the data
>       >    being sought.  The proxies are normally hosted on
> compromised user
>       >    computers.  The effect of this is to make the
> detection of the real
>       >    host for a piece of content much more difficult
> and to ensure that
>       >    the site remains up for as long as possible.
>       donald: The backend/hidden site
>
>       [JL] Change made
>
>       >    Threats in 2009 and Beyond].  Bots are frequently
> used as part of
>       >    concerted Distributed Denial of Service (DDoS)
> attacks for criminal,
>       donald: coordinated
>
>       [JL] Change made
>
>       >    operating in their networks.  Furthermore, ISPs
> may also be in a
>       >    unique position to be able to communicate to
> Internet users which are
>       donald: unique position to be able to notify their
> customers of potential infection by
>       bots or other infections.
>
>       [JL] Change made
>
>       >    From an end user perspective, knowing that their
> computer has been
>       donald: From an end user perspective, being notified
> that they may have an infected computer on their network is
> important information.
>
>       [JL] change made
>
>       >    infected with one or more bots is very important
> information.  Once
>       >    they know this, they can take steps to remove the
> bot, protect
>       >    themselves in the future, and resolve any problems
> which may stem
>       >    from the bot infection.  Given that bots can drain
> local computing
>       donald: Given that bots can consume vast amounts of
> local computing and network resources ...
>
>       [JL] change made
>
>       >    As a result, the intent of this document is to
> provide a guide to
>       donald: The intent of this document is to provide
> guidence to ISPs...
>
>       [JL] change made
>
>       >    generally, as well as on individual Internet
> users.  Efforts by ISPs
>       >    and other organizations could therefore, over
> time, reduce the pool
>       donald: and other organizations can over time reduce
> the pool of ...
>
>       [JL] change made
>
>       >    The potential mitigation of bots is accomplished
> through a process of
>       >    detection, notification to Internet users, and
> remediation of that
>       >    bot with a variety of tools, as described later in
> this document.
>       donald: bot infections with a variety ...
>
>       [JL] change made
>
>       >    ability of average users.  Attempts at bot removal
> may frequently be
>       >    unsuccessful, or only partially successful, and
> may leave a user's
>       donald: unsuccessful or only partially successful
> leaving the system ...
>
>       [JL] change made
>
>       >    system in an unstable and unsatisfactory state or
> even in a state
>       >    where it is still infected.  Attempts at bot
> removal can also result
>       >    in side effects ranging from a loss of data or
> other files, all the
>       >    way through partial or complete loss of system usability.
>       donald: Attempts at bot removal can result in side
> effects from a loss of data to partial or complete loss of
> system usablility.
>
>       [JL] change made
>
>       >    In general, the only way a user can be sure they
> have removed some of
>       >    today's increasingly sophisticated malware is by
> 'nuking-and-paving'
>       >    the system: reformatting the drive, reinstalling
> the operating system
>       >    and applications (including all patches) from
> scratch, and then
>       >    restoring user files from a clean backup.  However
> the introduction
>       donald: restoring user files from a known clean backup...
>
>       [JL] chg made
>
>       >    of BIOS-based malware may mean that, in some
> cases, this will not be
>       donald: of persistent memory based malware may meant
> that, in some cases, this may not be ...
>
>       [JL] chg made
>
>       >    enough and may prove to be more than any end user
> can be reasonably
>       >    expected to resolve [Persistent BIOS Infection].
> Experienced users
>       donald: Persistent memory infections.
>
>       [JL] chg made
>
>       >    would have to re-flash BIOS in their machines in
> order to remove
>       >    BIOS-based malware.  However, in some cases, not
> even 'nuking-and-
>       donald: would have to re-flash or re-image persistent
> memory sections on their machines in order ...
>
>       [JL] chg made
>
>       >    Devices with embedded operating systems, such as
> video gaming
>       >    consoles and smart home appliances, will most
> likely be beyond a
>       >    user's capability to remediate by themselves, and
> will typically
>       >    require the aid of vendor-specific advice, updates
> and tools.  Care
>       >    must be taken when imparting remediation advice to
> Internet users
>       >    given the increasingly wide array of computing
> devices that can be,
>       >    or could be, infected by bots in the future.
>       donald: Many such devices have a "reset to factory
> defaults" switch which will be the main method to clear such
> devices if infected.
>
>       [JL] Good suggestion.  Added "However, in some cases,
> such devices will have a function or switch to enable the
> user to reset that device to a factory default configuration,
> which may in some cases enable the user to remediate the infection."
>       >
>       >
>       > 5.  Detection of Bots
>       >
>       >    An ISP must first identify that an Internet user,
> in this case a user
>       >    that is assumed to be their customer or otherwise
> connected to the
>       >    ISP's network, is determined to be infected, or
> likely to have been
>       >    infected with a bot.  The ISP must attempt to
> detect the presence of
>       >    bots using methods, processes, and tools which
> maintain the privacy
>       >    of the personally identifiable information (PII)
> of their customers.
>       >    The ISP also must not block legitimate traffic in
> the course of bot
>       >    detection, and must instead employ detection
> methods, tools, and
>       >    processes which seek to be non-disruptive, as well as being
>       >    transparent to Internet users and end-user applications.
>       donald: You can't be totally transparent nor completely
> non-disruptive.
>       Nothing can be observed without affecting what your observing.
>       Perhaps rewrite with the concept of MINUMIZING such effects?
>
>       [JL] I see what you mean but it's tough to get that
> across.  I think there's enough wiggle room insofar as I have
>  said "seek to be non-disruptive" which means it is really
> simply a goal.
>
>       >
>       >    Detection methods, tools, and processes may
> include things such as
>       donald: may include analysis of specific ...
>
>       [JL] change made
>
>       >    or other Internet users, as well as a wide variety of other
>       >    possibilities.  It is likely that a combination of
> multiple bot
>       donald: In order to validate any bot net detection
> source a combination of multiple bot
>       detection sources has proven to be an effective approach ...
>
>       [JL] made a change to address your concern
>
>       >    detection data points will prove to be an
> effective approach in order
>       >    to corroborate information of varying
> dependability or consistency,
>       >    as well as to avoid or minimize the possibility of
> false positive
>       >    identification of computers.  Detection should
> also, where possible
>       >    and feasible, attempt to classify a bot in order
> to confirm that it
>       donald: to classify the specific bot infection type in
> order to ...
>
>       [JL] chg made
>
>       >    can be stopped.  This may mean that an ISP may
> need to balance the
>       >    desire or need to definitively classify and/or
> confirm a bot, which
>       >    may take an extended period of time, with the
> ability to predict the
>       >    strong likelihood of a bot in a very short period
> of time.  This
>       >    'definitive-vs-likely' challenge is difficult and,
> when in doubt,
>       >    ISPs should probably err on the side of caution by
> communicating when
>       donald: I agree erring on the side of caution BUT that
> should be don't communicate with your customers unless your
> fairly certain they are infected (False positive rate of 5%
> or less)! Notifying customers just on "likely" (ie greater
> then 50%) will cause customers loose faith in such notification.
>
>       [JL] Good point - added text to address this
>
>       >    a likely bot infection has taken place.  This also
> means that
>       >    Internet users may benefit from the deployment of
> client-based
>       >    software protections or other software tools,
> which can enable rapid
>       >    performance of heuristically-based detection bot
> activity, such as
>       donald: There is a risk here in that this teaches users
> to install software on their computer that may not be
> effective given todays botnet + root kit hiding abilities.
> Thus a false sense of security plus you've trained them to
> install software outside of normal OS maintenance. Would this
> even be included if it weren't the approach you choose?
>
>       [JL] It's not a statement that we should only rely on
> host-based protections, merely an argument for layered
> defenses.  I don't want to see things like I used to see in
> enterprises, where folks felt that since a firewall existed
> in the network that hosts could go largely unprotected. In
> this case, just because an ISP may have detection and perhaps
> protection measures in place, I don't want to obviate users
> of the responsibility to run some local software to protect
> themselves (something > nothing).  :-)

Agreed and in some cases I realize providing a link to a tool to find/remove the malware is pretty much a requirement if you want the customer to be able to clean up.
So there is a bit of an conundrum here as bad guys have used fake notifications to socially engineer vicitims to install malware. That is the primary method used for the class of malware refered to as scareware/fakeAV.



>
>       >    the detection of a bot as it starts to communicate
> a bot net and
>       donald: communicate to a bot net ...
>
>       [JL] chg made.
>
>       >    execute some type of command.  Any bot detection
> systems should also
>       donald: execute commands.
>
>       [JL] chg made
>       >    be capable of learning and adapting, either via
> manual intervention
>       >    or automatically, in order to cope with a rapidly
> evolving threat.
>       donald: If done learning is done automatically you have
> just solved the turing test.
>       Computers don't learn.
>
>       [JL] I removed 'learning'
>       >
>       >    As noted above, detection methods, tools, and
> processes should ensure
>       >    that privacy of customers' PII is maintained.
> While bot detection
>       >    methods, tools, and processes are similar to spam
> and virus defenses
>       >    deployed by the ISP for the benefit of their
> customers (and may be
>       >    directly related to those defenses), attempts to
> detect bots should
>       >    take into account the need of an ISP to take care
> to ensure any PII
>       >    collected or incidentally detected is properly
> protected.  The
>       >    definition of PII varies from one jurisdiction to
> the next so proper
>       >    care must be taken to ensure that any actions
> taken comply with
>       >    legislation and good practice in the jurisdiction
> in which the PII is
>       >    gathered.  Finally, depending upon the geographic
> region within which
>       donald: While this is true in many cases the customers
> PII is already at risk from criminal elements that want to
> exploit it. So being overly concerned about the privacy of
> PII (such as hiding PII from internal analysis teams) should
> be recommended against.
>
>       [JL] That's why I say "should" and not "must" since as
> you point out there may be cases where this is not possible.
> Also, what legally constitutes PII is typically fairly small.
>  I added this text into the section though: "This is
> important, as just as spam defenses may involve scanning the
> content of email messages which may contain PII, then so to
> may bot defenses similarly come into incidental contact with PII."
Good thanks.


>
>        a.  Where legally permissible or otherwise an industry accepted
>       >        practice in a particular market region, an ISP
> may in some manner
>       >        "scan" their IP space in order to detect
> un-patched or otherwise
>       >        vulnerable hosts.  This may provide the ISP
> with the opportunity
>       >        to easily identify Internet users who appear
> to already be or are
>       >        at great risk of being infected with a bot.
> ISP's should note
>       >        that some types of port scanning may leave
> network services in a
>       >        hung state or render them unusable due to
> common frailties, and
>       >        that many modern firewall and host-based
> intrusion detection
>       >        implementations may alert the Internet user to
> the scan.  As a
>       >        result the scan may be interpreted as a
> malicious attack against
>       >        the computer.  Vulnerability scanning has a
> higher probability of
>       >        leaving accessible network services and
> applications in a damaged
>       >        state and will often result in a higher
> probability of detection
>       >        by the Internet user and subsequent
> interpretation as a targeted
>       >        attack.  Depending upon the vulnerability
> being scanned, some
>       >        automated methods of vulnerability checking
> may result in data
>       >        being altered or created afresh on the
> Internet users computer
>       >        which be a problem in many legal environments.
>       donald: I would strongly recommend against this as a
> common practice. You provided most of the risks that make
> this a dangerous course.
>
>       [JL] Good point.  If we delete it, we'll just be asked
> later to add it back I fear.  So I have added this to the
> end: "Thus, while we note that this is one technique which
> may be utilized, it is unlikely to be particularly effective
> and it has problematic side effects, which leads the authors
> to recommend against the use of this particular method."
Agreed!


I missed something in b.
" Good examples of this include SNDS from Microsoft, XBL and PBL
  from Spamhaus and the DSHIELD AS tool from the SANS Institute."

That is a list of good examples but there are a lot of other good 3rd party infected client notification resources. Arbor via atlas, ShadowServer, Cymru, SIE, Support Intelligence and many others.
It might be worth having a seperate section near the end that
has a list of 3rd parties that are willing to share this kind of information with ISPs and provide contact info such as was done for conficker.
http://www.confickerworkinggroup.org/wiki/pmwiki.php/SP/ServiceProviders

That list is NOT complete in any sense of the word.
Providing contact information may help other ISPs get a more complete set of third party reports.
I am a bit concerned that some of the 3rd party contact information could change later and be outdated but I believe most of them will use a role based email account or a link.





>
>       >        monitoring to identify network anomalies that
> may be indicative
>       >        of botnet attacks or bot communications.  For
> example, an ISP may
>       >        be able to identify compromised hosts by
> identifying traffic
>       >        destined to IP addresses associated with the
> command and control
>       donald: destined to an IP address and control port
> associated with the ...
>       Most of the time the host associated with c and c has
> other legit traffic so having the control port helps a LOT
> when running potentially infected bot net reports.
>
>       [JL] good point - added some text to address this
>
>       >        of botnets.  In addition, bots can be
> identified when a remote
>       >        host is under a DDoS attack because computers
> participating in
>       >        the attack will likely be infected by a bot,
> frequently as
>       donald: I would caution against FPs due to spoofing of
> source IP addresses.
>       With netflow you can check which router and interface
> the attack traffic is coming in from as method to reduce FPs.
>
>       [JL] added: "...(though ISPs should beware of source IP
> address spoofing techniques to avoid or confuse detection)."
>
>       >    e.  User complaints: Because hosts infected by
> bots are frequently
>       >        used to send spam or participate in DDoS
> attacks, the ISP
>       >        servicing those hosts will normally receive
> complaints about the
>       >        malicious network traffic.  Those complaints
> may be sent to
>       >        RFC2142-specified [RFC2142] role accounts,
> such as abuse@ or
>       >        postmaster@ or to abuse or security addresses
> specified by the
>       donald: postmaster is for email delivery issues and we
> shouldn't be encouraging people to use it for abuse unless no
> other contact info is available.
>       NOC or security should be encouraged instead if abuse@
> doesn't work.
>
>       [JL] made a change to address this
>
>       >        site as part of its WHOIS (or other) contact data.
>       >
>       >    f.  ISPs may also discover likely bot infected
> hosts located at other
>       >        sites; when legally permissible or otherwise
> an industry accepted
>       donald: located on other ISPs networks; when legally ...
>       You mention industry accepted practice here as an or
> legally permissible but I think you mean and not or. It has
> to be legal or you shouldn't do it period. If it is an
> industry accepted practice shouldn't really come in to it.
> The first of us that started share such reports to other ISPs
> we were the first to do so. It wasn't the industry accepted
> practice at that time.
>
>       [JL] Good catch - made a change to address that
>
>       > 6.  Notification to Internet Users
>       >
>       >    Once an ISP has detected a bot, or the strong
> likelihood of a bot,
>       >    steps should be undertaken to inform the Internet
> user that they may
>       >    have a bot-related problem.  Depending upon a
> range of factors, from
>       >    the technical capabilities of the ISP, to the
> technical attributes of
>       >    their network, financial considerations, available
> server resources,
>       >    available organizational resources, the number of
> likely infected
>       >    computers detected at any given time, and the
> severity of any
>       >    possible threats, among many other things, an ISP
> should decide the
>       >    most appropriate method or methods for providing
> notification to one
>
>       donald: Ideally the customer should be notified via as
> many methods as possible at nearly the same time if possible.
> While most ISPs could do more then one notification method
> most of the "bad guys" won't be able to spoof many different
>       notification types within a small amount of time.
>       I don't know anyone that has implemented this but it
> seems like a really good recommendation.
>
>       [JL] Good suggestion, and added text to that effect.
>
>       >    or more of their customers or Internet users.
> Such notification
>       >    methods may include one or more of the following,
> as well as other
>       >    possible methods not described below
>       >
>       >    It is important to note that none of these methods
> are guaranteed to
>       >    be successful, and that each has its own set of
> limitations.  In
>       donald: 100% successful,
>
>       [JL] Ack, added.
>
>       >    addition, in some cases, an ISP may determine that
> a combination of
>       >    two or more methods is most appropriate.  Finally,
> notification is
>       >    also considered time sensitive; if the user does
> not receive or view
>       >    the notification or a timely basis, then a
> particular bot could
>       >    launch an attack, exploit the user, or cause other
> harm.  If
>       >    possible, an ISP should establish a preferred
> means of communication
>       >    when the subscriber first signs up for service.
> As a part of the
>       >    notification process, ISPs should maintain a
> record of the allocation
>       >    of IP addresses to subscribers for such a period
> as allows any bot
>       donald: commonly used bot detection
>
>       [JL] ack, added
>
>       >    detection technology to be accurately able to link
> an infected IP
>       donald: to be able to accurately link an infected ...
>
>       [JL] change made
>
>       >    address to a subscriber.  This record should only
> be maintained for a
>       >    period of time which is necessary, in order to maintain the
>       >    protection of the privacy of an individual subscriber.
>       >
>       >    One important factor to bear in mind is that
> notification to end
>       >    users needs to be defended against spoofing by
> third parties.  This
>       >    must be done to protect against the possibility of
> notifications
>       >    being spoofed and used by bots to deliver
> additional malware.
>       donald: must be done to protect as well as reasonably
> possible against the possibility of legitimate notifications
> being spoofed and used by persons with malicious intent to
> perform additional malicious acts against the victims.
>
>       They may not be bots and they may not use this to
> "infect" with a bot in many cases "they" will be an AD that
> sells fake AV/scareware.
>
>       [JL] Good suggestion - change made
>
>       >
>       > 6.1.  Email Notification
>       >
>       >    This is probably the most common form of
> notification used by ISPs.
>       donald: Is it? Do we have any surveys or stats on this?
> I think your correct but it would be good to have some kind of backup.
>
>       [JL] changed to "common" since I have no data to backup
> that it's the most common.

Good enough. I would still like to see a survey at some point maybe we could get Danny at Arbor to add a set of customer notification questions to their SP survey for 2011.

>
>       >    delete the message or otherwise file it in an
> email folder that the
>       >    user may not check on a regular and/or timely
> basis.  Bot masters
>       >    have also been known to impersonate the ISP or
> trusted sender and
>       >    send fradulent emails to the users.  This
> technique of social
>       donald: fraudulent
>
>       [JL] corrected
>
>       >    engineering, generally referred to as 'phishing',
> often leads to new
>
>       donald: Phishing is generally defined as attempting to
> gain user credentials or other sensitive information by
> spoofing a trusted source.
>       What your describing is a method used by some viruses
> such as beagle to spread by convincing customers to install
> it thinking it was a fix to some malware or other issue. This
> is commonly refered to as social engineering as you state but
> not phishing.
>
>       [JL] That may have been an artifact of an earlier
> draft, but in any case I dropping the phishing part.
>
>       >    as such not welcome it, or not accept the call at
> all.  Finally, even
>       >    if a representative of the ISP is able to connect
> with and speak to a
>       >    user, that user is very likely to lack the
> necessary technical
>       >    expertise to understand or be able to effectively
> deal with the
>       >    threat.
>       donald: Isn't this last statement true for ALL
> notifications methods.
>       I agree it is true, but it probably needs to be a
> caveat for all notification methods. You mention it else
> where but perhaps a special paragraph at the beginning that
> states this is true for all notification types.
>
>       [JL] I'll take that into consideration - added to my
> open issues list for the next version.
>
>       >
>       > 6.3.  Postal Mail Notification
>       >
>       >    This form of notification is probably the least
> popular and effective
>       >    means of communication, due to both preparation
> time, delivery time,
>       >    the cost of printing and paper, and the cost of postage.
>
>       donald: For ISPs that are already billing their
> customers there wouldn't need to be additional cost of
> postage, printing or paper it could be incorporated into the
> bill its self. The biggest risk here is delivery time and the
> fact that some customers don't review their bills (similar to
> the email issue identified above).
>
>       [JL] In most cases, it's not realistic to get that sort
> of info into a billing system.  Thus, a mailer is likely to
> be separate.
Ok, I haven't tried to use billing for notification so you could be correct.
It doesn't sound HARD to me but no real background doing so ...

>       >
>       > 6.4.  Walled Garden Notification
>       >
>       >    Placing a user in a walled garden is another
> approach that ISPs may
>       >    take to notify users.  A walled garden refers to
> an environment that
>       >    controls the information and services that a
> subscriber is allowed to
>       >    utilize and what network access permissions are
> granted.  This is an
>       donald: That is really the definition of a quarantine
> walled garden.
>       Walled garden's don't have to quarantine the user
> (much) to notify the customer.
>       They can be fairly leaky and still notify the customer
> of their infection.
>
>       [JL] I'd like to ponder that more, so have added to our
> open issues list.
Sounds good we can talk about the qwest walled garden seperately if you like to better understand our "leaky" approach:)

>
>       >    not always the case that a user is actively using
> a computer that
>       >    uses a web browser or which has a web browser
> actively running on it.
>       donald: not always the case that a user is using a
> application that used the ports being redirected to the walled garden.
>
>       This is more of an application issue then the "computer".
>
>       [JL] Was modified to 'host.'  Also added the reference
> to other ports.
>
>       >    In one example, a user could be playing a game
> online, via the use of
>       >    a dedicated, Internet-connected game console.  In
> another example,
>       >    the user may not be using a computer with a web
> browser when they are
>       >    placed in the walled garden and may instead be in
> the course of a
>       >    telephone conversation, or may be expecting to
> receive a call, using
>       >    a Voice Over IP (VoIP) device of some type.  As a
> result, the ISP may
>       >    feel the need to maintain a potentially lengthy
> white list of domains
>       >    which are not subject to the typical restrictions
> of a walled garden,
>       >    which could well prove to be an onerous task, from
> an operational
>       >    perspective.
>       donald: Or design a leaky walled garden that allows
> most traffic without interference only redirecting the
> traffic needed to mitigate or notify.
>
>       [JL] Seems related to the question raised before re a
> quarantine WG.  I'll keep it on the open issues list.
>
>       >        This option is suggested when the purpose of the
>       donald: primary purpose of the
>
>       [JL] change made
>
>       >    Once the user acknowledges the notification, then
> the user decides to
>       donald: the notification, decides to remediate and exit
> the walled garden or exit the walled garden without
> remediating the issue.
>
>       [JL] made a chg to address this
>
>       >    either remediate and then exit the walled garden,
> or exit the walled
>       >    garden without addressing the issue.  Another
> approach may be to
>       >    enforce a stricter policy and require the user to
> clean the computer
>       >    prior to permitting the user to exit the walled
> garden, though this
>       >    may not be technically feasible depending upon the
> type of bot,
>       >    obfuscation techniques employed by a bot, and/or a
> range of other
>       >    factors.  Thus, the ISP may also need to support
> tools to scan the
>       >    infected computer and determine whether it is
> still infected or rely
>       >    on user judgment that the bot has been disabled or
> removed.  One
>
>       donald: Scanning is risky and should be identified as such.
>       Port scanning or vulnerability scanning is dangerous as
> you identified else where.
>       Anti-virus scanning is also risky if the ISP initiates
> it as that means you have taken some level of control (and
> responsibility) for that infected machine.
>       A better method would be to use SNORT or similar tool
> to passively watch for packets indicating an continued infection.
>       Another good approach is just to keep track of whom was
> put into the walled garden for which infection and if they
> keep showing up as infected via the trusted third party
> reports then you can assume they are unable/unwilling to
> remediate and take what ever additional actions you deem appropriate.
>
>       [JL] Added a clarifying statement "(in the style of a
> virus scan, rather than a port scan)"
>
>       >    There are several drawbacks with this
> communications method.  There
>       >    is a high probability that subscriber may
> interpret the communication
>       >    to be spam, and as such ignore it.  Also, not
> every user uses IM
>       >    and/or the user may not provide their IM identity
> to the ISP so some
>       >    alternative means have to be used.  Even in those
> cases where a user
>       >    does have an IM address, they may not be signed
> onto that IM system
>       >    when the notification is attempted.  There maybe
> also be a privacy
>       >    concern on the part of users, when such an IM
> notification must be
>       >    transmitted over a third-party network and/or IM
> service.  As such,
>       >    should this method be used, the notification
> should be discreet and
>       >    not include any PII in the notification itself.
>
>       donald: Isn't this true for email, sms and possibly
> other methods.
>       Again if so it could become part of the opening
> statement rather then tied to just one notification type.
>
>       [JL] Good suggestion to ponder  - I will drop that into
> my open issues for a future rev.
>
>
>       > Livingood, et al.        Expires August 16, 2010
>          [Page 15]
>       >
>       > Internet-Draft     Remediation of Bots in ISP
> Networks     February 2010
>       >
>       >
>       >    typically pay on a per-message basis to send SMS
> notifications.  An
>
>       donald: No in most cases the ISP would use the
> customers SMS providers SMTP to SMS gateway and wouldn't
> encore additional costs for an sms msg.
>
>       [JL] I just generalized to the following: "In both
> cases an ISP may incur fees related to SMS notifications,
> depending upon the method used to send the notifications"
Ok

>
>       >    additional downside is that SMS messages sent to a
> user may result in
>       >    a charge to the user by their wireless provider,
> depending upon the
>       >    plat to which they subscribe.  Another minor
> disadvantage is that it
>
>       donald: did you mean platform or feature set here?
>
>       [JL] I meant 'plan' and corrected it.
>
>       >    is possible to notify the wrong user if the
> intended user changes
>       >    their mobile number but forgets to update it with the ISP.
>
>       donald: Again this applies to more then just sms,
> certainly to email accounts and probably other notification methods.
>
>       [JL] True, but given that most users are metered in
> some manner for their SMS messages, this is more pronounced
> of a risk.  An SMS could cost a user a few cents.

Ok but the fact that an email account could be retired and reused is still a risk.
I am sure some of the other notification methods could be directed to the wrong party due to account
reuse also so we may need to add some generic statement to that effect.

I agree sms could cost an invalid end user so we can put some emphisis on that.

>
>       > 6.9.  Considerations for Notification to Network
> Locations Using a
>       >       Shared IP Address
>       >
>       >    Delivering a notification to a location that
> Internet access routed
>       >    through one or more shared public IP addresses may
> be of low value
>       >    since it may be quite difficult to differentiate
> between users when
>       >    providing a notification.  For example, on a
> business network of 500
>       >    users, all sharing one public IP address, it may
> be sub-optimal to
>       >    provide a notification to all 500 users if you
> only need one specific
>       >    user to be notified and take action.  As a result,
> such networks may
>
>       donald: In cases like this to assist the business
> network you describe providing a date/timestamp, source port
> of the infected system, and malicious sites it was visiting
> often helps them find the infected system.
>
>       [JL] Added: However, should an ISP implement some form
> of notification to such networks, it may be better to simply
> send notifications to a designated network administrator at
> the site.
Most ISPs have a "network contact" for their enterprise customers.
That is whom would normally be notified.

That happens in the broadband world via the primary email account
that is used for any type of notification.


> In such a case the local network administrator may
> like to receive additional information in such a
> notification, such as a date and timestamp, the source port
> of the infected system, and malicious sites that may have
> been visited.
malicious sites and ports that may have been visited.

When notifying enterprises they can often find an infected system behind a NAT/PAT system
if they have the date/timestamp, malicious site (ip or url) and port if they have logs.

>
>       >          notice regular disk activity even when you
> are not doing
>       >          anything.  Ignoring this problem is not
> really an option since
>       donald: Ignoring this problem is risky to you and your
> personal information ..
>
>       [JL]  change made
>
>       >    As an additional consideration, it may be useful
> to create a process
>       >    by which users could choose, at their option and
> with their express
>       >    consent, to share data regarding their bot
> infection with their ISP
>       >    and/or another authorized third party.  Such third
> parties may
>       >    include governmental entities that aggregate
> threat data, such as the
>       >    Internet Crime Complaint Center referred to
> earlier in this document,
>       >    to academic institutions, and/or security
> researchers.  While in many
>       >    cases the information shared with the user's ISP
> or designated third
>       >    parties will only be used for aggregated
> statistical analysis, it is
>       >    also possible that certain research needs may be
> best met with more
>       >    detailed data.  Thus, any such data sharing from a
> user to the ISP or
>       >    authorized third party may contain some type of personally
>       >    identifiable information, either by design or
> inadvertently.  As a
>       >    result, any such data sharing must be enabled on
> an opt-in based,
>       >    where users review and approve of the data being
> shared and the
>       >    parties with which it is to be shared.
>
>       donald: Unless the ISP is already required to provide
> said information due local laws or judicial requirements (subpoena)...
>
>       [JL] change made
>
>       >    In addition, regarding notifications, as described
> in the Section 6,
>       >    care should be taken to assure users that
> notifications have been
>       >    provided by a trustworthy site and/or party, so
> that the notification
>       >    is more difficult for phishers to mimic, or that
> the user has some
>
>       donald: I think you are talking social engineers not phishers.
>
>       [JL] changed to include both.
>
>       Thanks again for this highly detailed feedback!!
>
>
>
>
>
>
>
>

This communication is the property of Qwest and may contain confidential or
privileged information. Unauthorized use of this communication is strictly
prohibited and may be unlawful.  If you have received this communication
in error, please immediately notify the sender by reply e-mail and destroy
all copies of the communication and any attachments.