[Sipping] RE: [spitstop] Spit Score Semantics

["Saverio Niccolini" <[e-mail]>]

Hi Cullen,

all we were trying to do was to answer the question about semantics
raised by Henning some time ago. He was arguing that without semantics
the spam score has no meaning and that is why we answered to it (after
thinking quite some time...).

As you know some examples of architecture are out already together with a
reference scenario (draft expired):
http://tools.ietf.org/html/draft-tschofenig-sipping-framework-spit-reduction-02
http://tools.ietf.org/id/draft-niccolini-sipping-spitstop-00.txt

Moreover some reference architecture for prototypes/products exist as
well (we demonstrated at last IETF our demo to interested people)
and there are scientific results in the literature that would justify
the semantics we are reporting here.

If an architecture needs to be defined then let's define it, let's
have this work started, it seems there is enough interest around the
topic and people willing to work on it :-)

Looking forward to meeting you in Vancouver.

Cheers,
Saverio

============================================================
Dr. Saverio Niccolini
Senior Researcher
NEC Laboratories Europe, Network Research Division	
Kurfuerstenanlage 36, D-69115 Heidelberg
Tel.     [phone]
Fax:     [phone]
e-mail:  [e-mail] <-- !!! NEW ADDRESS !!!
============================================================
NEC Europe Limited Registered Office: NEC House, 1 Victoria
Road, London W3 6BL Registered in England 2832014
 
  

> -----Original Message-----
> From: [e-mail] 
> [mailto:[e-mail]] On Behalf Of 
> Cullen Jennings
> Sent: Thursday, November 22, 2007 7:51 PM
> To: Jan Seedorf
> Cc: Eric Rescorla; [e-mail]; 
> [e-mail]; [e-mail]
> Subject: Re: [spitstop] Spit Score Semantics
> 
> 
> This is a questions of architecture and without an 
> architecture it s pretty hard to decide what to do. If your 
> architecture requires the combination of two scores generated 
> by two different sources - then you need them to mean 
> something more than an ordinal value. Negative log 
> probabilities are pretty popular in the bayesian world but 
> there are many other things such as bounded error rates that 
> could work too. If on the other hand the number is generated 
> by one source and it the only thing is to execute a threshold 
> on it, then it could just be an arbitrary range. Anyways, 
> before we understand what the basic architecture is, it seems 
> hard to sort out the semantics we need.
> 
> 
> On Nov 22, 2007, at 3:17 AM, Jan Seedorf wrote:
> 
> > Dear all,
> >
> > We (Saverio Niccolini and Jan Seedorf) believe there is a need to 
> > signal SPIT scores (as calculated with various methods at single 
> > signalling hops) along the signalling path to protect against SPIT.
> > In order to make such a signalling of SPIT scores compatible among 
> > heterogeneous system, it must be standardised in some way. The 
> > draft-wing-sipping-spam-score addresses this issue by proposing 
> > additional headers/values for SPIT scores that could be 
> passed along 
> > the downstream (and possibly upstream) signalling path. One 
> question 
> > is whether it makes sense to have a SPIT score ranging from 
> 0-100. The 
> > concern raised by Henning Schulzrinne on the SPITSTOP 
> mailing list was 
> > that a scale from 0-100 has no clear semantics defined for values 
> > other than 0 ("I am sure this is not
> > SPIT") and 100 ("I am sure this is SPIT"). We thought about how to 
> > define semantics for SPIT scores and would like to receive 
> feedback on 
> > our thoughts.
> >
> > First, we believe there are SPIT detection methods resulting in a 
> > binary score (0="not SPIT",1="SPIT"). For such methods with 
> a binary 
> > result range it is quite easy to come up with well-defined 
> semantics. 
> > For instance, for methods like "whitelist" or "blacklist", 
> it could be 
> > defined that "0" means "not on list" and "1" means "on 
> list". Similar, 
> > for a CAPTCHA-test you can image "0"
> > for passed (i.e., not SPIT according to this test) and "1" 
> for failed 
> > (i.e., SPIT according to this test).
> >
> > Second, we believe it is also possible to come up with 
> semantics for 
> > statistical methods (which generally do not produce 0/1 results), 
> > resulting in a score between 0 and 100. For anomaly-based 
> statistical 
> > methods, the semantics of a score x could be that x percent of 
> > previous messages where below/above (depending on the
> > method) the result for this method compared to the current 
> message.  
> > For instance, think of a call-rate detection method that checks how 
> > many calls are initiated from a particular SIP-URI (or IP-address) 
> > within a time-frame. A result of 85 for such a test would mean that 
> > the call-rate for the SIP-URI in the From-header of the SIP 
> message in 
> > the time-frame is higher than 84% of all calls monitored in 
> the past. 
> > Note, as Henning pointed out, this implies nothing directly on the 
> > SPIT probability for that message. It is simply a way to precisely 
> > define the semantics for the score of anomaly-based statistical 
> > modules.
> >
> > For statistical modules the score could also be based on user- 
> > feedback. Then the semantics of score x would be that in the past x 
> > percent of messages with the same method-result were marked 
> as SPIT by 
> > users (using some user-feedback mechanism). This has probably more 
> > relation to a SPIT probability. However, this is not important. The 
> > point is that the score is well-defined.
> >
> > In general, we think that for EACH method it needs to be (publicly) 
> > well-defined what the corresponding score means. This score 
> does NOT 
> > (necessarily) imply a SPIT-probability. Instead, it has a 
> per- method 
> > semantic. Then, for each method it is clearly defined what the 
> > signalling of corresponding SPIT-scores means and any SPIT- 
> prevention 
> > device can make a local policy decision on these scores at its own 
> > discretion. In particular, whether the SPIT-prevention device using 
> > scores from previous hops makes arithmetic calculations on these 
> > scores or how it combines them to a final 0/1 decision is 
> out of the 
> > scope of standardisation. In addition, any hop can express its 
> > accumulated result for the single values in a total score. 
> This total 
> > is based on this hops calibrated algorithm on SPIT detection and is 
> > either 0="not SPIT" or 1="SPIT".
> >
> > To give an example (based on the current draft-wing-sipping-spam- 
> > score, whether the Via header is the correct place for SPIT-score 
> > signalling or not is a different issue and not part of this
> > discussion):
> >
> >        Via: SIP/2.0/UDP biloxi.com;branch=z9hG4bKnashds8
> >          ;received=192.0.2.1
> >   -->    ;spam-detail="callrate-anomaly=30;iprange- 
> > userfeedback=45;whitelist=0;blacklist=1"
> >   -->    ;spam=1
> >
> > This would imply that proxy biloxi.com calculated a 
> callrate value of 
> > 30 for this message, based on anomaly detection, meaning 30 
> percent of 
> > all previous messages had a lower or equal call-rate, 
> meaning nothing 
> > more and nothing less. Also, the iprange method for this message 
> > resulted in a score of 45, based on user-feedback.
> > This means that 45% of all times the result of the iprange 
> method for 
> > this message was marked as SPIT by users. The message was 
> not on the 
> > whitelist of the proxy but on its blacklist. With its own 
> (proprietary 
> > and secret) algorithm, the proxy would aggregate these scores to a 
> > "this is SPIT" decision.
> >
> > We are looking forward to receiving comments from others 
> how they view 
> > the problem of assigning semantics to SPIT scores and on 
> our thoughts 
> > towards a solution. We are thinking of writing these 
> thoughts into a 
> > separate draft or inside the draft-wing-sipping- spam-score to have 
> > the semantic of SPIT scores defined.
> >
> >  - Jan & Saverio
> >
> > ============================================================
> > Jan Seedorf
> > Research Scientist
> > NEC Europe Ltd., NEC Laboratories Europe, Network Division	
> > Kurfuerstenanlage 36, D-69115 Heidelberg
> > Tel.     [phone]
> > Fax:     [phone]
> > e-mail:  [e-mail]
> > ============================================================
> > NEC Europe Limited Registered Office: NEC House,
> > 1 Victoria Road, London W3 6BL Registered in England 2832014
> _______________________________________________
> spitstop mailing list
> [e-mail]
> https://listserv.netlab.nec.de/mailman/listinfo/spitstop
> 


_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [e-mail] for questions on current sip
Use [e-mail] for new developments of core SIP