Re: non-UTF-8 sequences in Sieve scripts

[Alexey Melnikov <[e-mail]>]

Ned Freed wrote:

>> Ned Freed wrote:
>
>> >> Hi everyone,
>> >
>> >> I think there is rough consensus to have an extension that would
>> >> introduce new strings with proper escaping mechanism. The new strings
>> >> will allow for UTF-8 (as is) + non-UTF-8 can be escaped. As nobody
>> >> volunteered to do this, I will write a short proposal on this.
>> >
>> >> But I would like to get general feeling of the Working Group about
>> >> non-UTF-8 sequences in existing 3028bis strings.
>> >> draft-ietf-sieve-3028bis-09.txt doesn't prohibit them. However, what
>> >> should our long term solution be? I see two choices:
>> >
>> >> 1). draft-ietf-sieve-3028bis stays as it is now, i.e. it allows for
>> >> non-UTF-8 sequences, but recommends UTF-8. Eventually implementations
>> >> will implement new strings with proper escaping mechanism, but 
>> existing
>> >> strings will stay as described till the end of time.
>> >
>> >> 2). draft-ietf-sieve-3028bis gets updated to only describe UTF-8
>> >> sequences with no escaping (*). The draft can describe existing
>> >> situation with non-UTF-8 sequences, or it can remain completely 
>> silent
>> >> on the issue. This wouldn't affect existing implementations, as 
>> long as
>> >> non-UTF-8 sequences in scripts are not explicitly prohibited. The 
>> draft
>> >> will also explicitly state that future revisions of this document 
>> would
>> >> not allow for non-UTF-8 sequences.
>> >
>> >> =====
>> >> So basically 2) would state the desire to deprecate use of unescaped
>> >> non-UTF-8 sequences in scripts, while 1) would allow for them
>> >> indefinitely.
>
[...]

>> > Once we have the alternative defined I think the right approach is
>> > somewhere between (1) and (2). (2) goes too far IMO by saying that 
>> deprecation will
>> > happen in the future. I think the correct course is to say that the
>> > ability to use non-UTF-8 _may_ be abandoned in the future and to plan
>> > accordingly.
>
>> Right. In general case it is not possible to put requirements on future
>> specs.
>
>> Ok, if (2) is replaced with what you've proposed, which choice would you
>> prefer?
>
> Definitely (2). People need to be warned that this may change in the 
> future.

Thinking more about this. I think replacing existing ABNF in 3028bis to 
only allow for valid UTF-8 sequences would be a drastic change at this 
point, as it will pretty much declare all existing implementations non 
compliant. I've already suggested some warning text about phasing out 
non-UTF-8 sequences and you agreed with my text. I think the suggested 
text is sufficient for this round of work on 3028bis.

Comments?



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8ULPmhM051929; Sat, 30 Sep 2006 14:25:48 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8ULPmRw051928; Sat, 30 Sep 2006 14:25:48 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from rufus.isode.com (rufus.isode.com [62.3.217.251]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8ULPkX8051922 for <[e-mail]>; Sat, 30 Sep 2006 14:25:47 -0700 (MST) (envelope-from [e-mail])
Received: from [172.16.2.167] (shiny.isode.com [62.3.217.250])  by rufus.isode.com (submission channel) via TCP with ESMTPA  id <[e-mail]>; Sat, 30 Sep 2006 22:25:45 +0100
Message-ID: <[e-mail]>
Date: Sat, 30 Sep 2006 22:25:22 +0100
From: Alexey Melnikov <[e-mail]>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: Ned Freed <[e-mail]>
CC: MTA filtering mailing list <[e-mail]>
Subject: Re: non-UTF-8 sequences in Sieve scripts
References: <[e-mail]> <[e-mail]> <[e-mail]> <[e-mail]>
In-Reply-To: <[e-mail]>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

Ned Freed wrote:

>> Ned Freed wrote:
>
>> >> Hi everyone,
>> >
>> >> I think there is rough consensus to have an extension that would
>> >> introduce new strings with proper escaping mechanism. The new strings
>> >> will allow for UTF-8 (as is) + non-UTF-8 can be escaped. As nobody
>> >> volunteered to do this, I will write a short proposal on this.
>> >
>> >> But I would like to get general feeling of the Working Group about
>> >> non-UTF-8 sequences in existing 3028bis strings.
>> >> draft-ietf-sieve-3028bis-09.txt doesn't prohibit them. However, what
>> >> should our long term solution be? I see two choices:
>> >
>> >> 1). draft-ietf-sieve-3028bis stays as it is now, i.e. it allows for
>> >> non-UTF-8 sequences, but recommends UTF-8. Eventually implementations
>> >> will implement new strings with proper escaping mechanism, but 
>> existing
>> >> strings will stay as described till the end of time.
>> >
>> >> 2). draft-ietf-sieve-3028bis gets updated to only describe UTF-8
>> >> sequences with no escaping (*). The draft can describe existing
>> >> situation with non-UTF-8 sequences, or it can remain completely 
>> silent
>> >> on the issue. This wouldn't affect existing implementations, as 
>> long as
>> >> non-UTF-8 sequences in scripts are not explicitly prohibited. The 
>> draft
>> >> will also explicitly state that future revisions of this document 
>> would
>> >> not allow for non-UTF-8 sequences.
>> >
>> >> =====
>> >> So basically 2) would state the desire to deprecate use of unescaped
>> >> non-UTF-8 sequences in scripts, while 1) would allow for them
>> >> indefinitely.
>
[...]

>> > Once we have the alternative defined I think the right approach is
>> > somewhere between (1) and (2). (2) goes too far IMO by saying that 
>> deprecation will
>> > happen in the future. I think the correct course is to say that the
>> > ability to use non-UTF-8 _may_ be abandoned in the future and to plan
>> > accordingly.
>
>> Right. In general case it is not possible to put requirements on future
>> specs.
>
>> Ok, if (2) is replaced with what you've proposed, which choice would you
>> prefer?
>
> Definitely (2). People need to be warned that this may change in the 
> future.

Thinking more about this. I think replacing existing ABNF in 3028bis to 
only allow for valid UTF-8 sequences would be a drastic change at this 
point, as it will pretty much declare all existing implementations non 
compliant. I've already suggested some warning text about phasing out 
non-UTF-8 sequences and you agreed with my text. I think the suggested 
text is sufficient for this round of work on 3028bis.

Comments?



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8UKohFG047830; Sat, 30 Sep 2006 13:50:43 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8UKohBH047829; Sat, 30 Sep 2006 13:50:43 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from mauve.mrochek.com (206.117.180.234.brandx.net [206.117.180.234] (may be forged)) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8UKogpI047823 for <[e-mail]>; Sat, 30 Sep 2006 13:50:43 -0700 (MST) (envelope-from [e-mail])
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <[e-mail]> for [e-mail]; Sat, 30 Sep 2006 13:50:39 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <[e-mail]>; Sat, 30 Sep 2006 13:50:34 -0700 (PDT)
Cc: [e-mail]
To: Alexey Melnikov <[e-mail]>
Message-id: <[e-mail]>
Date: Sat, 30 Sep 2006 13:50:25 -0700 (PDT)
From: Ned Freed <[e-mail]>
Subject: Re: Text suggesting that non-UTF-8 sequenced may be abandoned in the future
In-reply-to: "Your message dated Sat, 30 Sep 2006 21:18:38 +0100" <[e-mail]>
MIME-version: 1.0
Content-type: TEXT/PLAIN; charset=ISO-8859-1; format=flowed
References: <[e-mail]>
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

> Hi everyone,
> 3028bis, section 2.1. (Form of the Language) says:

>  >   With the exceptions of strings and comments, the language is limited
>  >   to US-ASCII characters.  Strings and comments may contain octets
>  >   outside the US-ASCII range.  Specifically, they will normally be in
>  >   UTF-8, as specified in [UTF-8].  NUL (US-ASCII 0) is never permitted
>  >   in scripts, while CR and LF can only appear as the CRLF line ending.

> As per recent discussion on the mailing list I suggest adding the
> following text to the end of this paragraph:

> Note that Sieve WG is working on an extension that would allow for 7bit
> escaping
> of non-UTF-8 data. When and if this extension is deployed, the ability
> to use
> non-UTF-8 data unescaped in Sieve scripts may be abandoned.


> Comments?

Seems OK to me.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8UKJ2dO045152; Sat, 30 Sep 2006 13:19:02 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8UKJ20B045150; Sat, 30 Sep 2006 13:19:02 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from rufus.isode.com (rufus.isode.com [62.3.217.251]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8UKJ1OB045137 for <[e-mail]>; Sat, 30 Sep 2006 13:19:02 -0700 (MST) (envelope-from [e-mail])
Received: from [172.16.2.167] (shiny.isode.com [62.3.217.250])  by rufus.isode.com (submission channel) via TCP with ESMTPA  id <[e-mail]>; Sat, 30 Sep 2006 21:18:58 +0100
Message-ID: <[e-mail]>
Date: Sat, 30 Sep 2006 21:18:38 +0100
From: Alexey Melnikov <[e-mail]>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915
X-Accept-Language: en-us, en
To: [e-mail]
Subject: Text suggesting that non-UTF-8 sequenced may be abandoned in the future
MIME-version: 1.0
Content-type: text/plain; charset="ISO-8859-1"; format="flowed"
Content-transfer-encoding: 7bit
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

Hi everyone,
3028bis, section 2.1. (Form of the Language) says:

 >   With the exceptions of strings and comments, the language is limited
 >   to US-ASCII characters.  Strings and comments may contain octets
 >   outside the US-ASCII range.  Specifically, they will normally be in
 >   UTF-8, as specified in [UTF-8].  NUL (US-ASCII 0) is never permitted
 >   in scripts, while CR and LF can only appear as the CRLF line ending.

As per recent discussion on the mailing list I suggest adding the 
following text to the end of this paragraph:

Note that Sieve WG is working on an extension that would allow for 7bit 
escaping
of non-UTF-8 data. When and if this extension is deployed, the ability 
to use
non-UTF-8 data unescaped in Sieve scripts may be abandoned.


Comments?

Alexey



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8RIltNx010621; Wed, 27 Sep 2006 11:47:55 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8RIltUe010620; Wed, 27 Sep 2006 11:47:55 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from out1.smtp.messagingengine.com (out1.smtp.messagingengine.com [66.111.4.25]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8RIlsoO010614 for <[e-mail]>; Wed, 27 Sep 2006 11:47:55 -0700 (MST) (envelope-from [e-mail])
Received: from frontend3.internal (frontend3.internal [10.202.2.152]) by frontend1.messagingengine.com (Postfix) with ESMTP id 6E1C9DA9C8F; Wed, 27 Sep 2006 14:47:51 -0400 (EDT)
Received: from heartbeat2.internal ([10.202.2.161]) by frontend3.internal (MEProxy); Wed, 27 Sep 2006 14:47:54 -0400
X-Sasl-enc: 61pAvkbidcCT56aEuQRfeK6yvv59GyqDCMS0VN4FId9c 1159382874
Received: from [192.168.1.102] (banana.groupd.com [64.139.6.95]) by mail.messagingengine.com (Postfix) with ESMTP id 11125BC66; Wed, 27 Sep 2006 14:47:53 -0400 (EDT)
Message-ID: <[e-mail]>
Date: Wed, 27 Sep 2006 11:47:55 -0700
From: Matthew Elvey <[e-mail]>
User-Agent: Thunderbird 1.5.0.7 (Macintosh/20060909)
MIME-Version: 1.0
To: [e-mail]
CC: Alexey Melnikov <[e-mail]>
Subject: draft-ietf-sieve-3028bis-09
X-Habeas-SWE-1: winter into spring
X-Habeas-SWE-2: brightly anticipated
X-Habeas-SWE-3: like Habeas SWE (tm)
X-Habeas-SWE-4: Copyright 2002 Habeas (tm)
X-Habeas-SWE-5: Sender Warranted Email (SWE) (tm). The sender of this
X-Habeas-SWE-6: email in exchange for a license for this Habeas
X-Habeas-SWE-7: warrant mark warrants that this is a Habeas Compliant
X-Habeas-SWE-8: Message (HCM) and not spam. Please report use of this
X-Habeas-SWE-9: mark in spam to <http://www.habeas.com/report/>.
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

Minor.

Would it make sense to add this or something like it to the security
section?
   Security issues associated with UTF-8 are fully
   discussed in the security consideration section of [RFC3629]. This
   document is believed not to introduce any additional security
   considerations in this general area.

What's Cyrus' new email?



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8P9FjlC011521; Mon, 25 Sep 2006 02:15:45 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8P9FjeB011520; Mon, 25 Sep 2006 02:15:45 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from rufus.isode.com (rufus.isode.com [62.3.217.251]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8P9FgQw011512 for <[e-mail]>; Mon, 25 Sep 2006 02:15:43 -0700 (MST) (envelope-from [e-mail])
Received: from [172.16.1.99] (shiny.isode.com [62.3.217.250])  by rufus.isode.com (submission channel) via TCP with ESMTPA  id <[e-mail]>; Mon, 25 Sep 2006 10:15:41 +0100
Message-ID: <[e-mail]>
Date: Sun, 24 Sep 2006 22:06:44 +0100
From: Alexey Melnikov <[e-mail]>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915
X-Accept-Language: en-us, en
To: Ned Freed <[e-mail]>
CC: Michael Haardt <[e-mail]>, [e-mail]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
References: <[e-mail]> <[e-mail]> <[e-mail]> <[e-mail]>
In-Reply-To: <[e-mail]>
MIME-version: 1.0
Content-type: text/plain; charset="ISO-8859-1"; format="flowed"
Content-transfer-encoding: 7bit
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

Ned Freed wrote:

>> [...]
>
> I have to agree with Michael - changing the basic syntax of sieve is
> unacceptable.
>
>> >I don't think the base spec allows that.
>> >
>> IMHO, if it explicitly prohibit such change, we need to change the base
>> spec.
>
> One of the key strengths of sieve is that the core syntax is both simple
> and immutable. This makes it possible to perform syntax checks even if
> you don't understand all the extensions a given script uses. Breaking 
> this would IMO be hugely damaging.

Ok, good point.

>> >Sieve has a very
>> >small syntax, moving most stuff to the semantical layer for being very
>> >extensible, pretty much like LISP does.
>> >
>> >The variables extension, for example, introduces the concept of 
>> expanding
>> >strings, but only at the semantic level.  Syntactically, strings still
>> >look the same.
>
>> The problem is that RFC 3028 said that '\ <octet>' for any <octet> other
>> than <\> or <"> SHOULD be interpreted as <octet>.
>> This effectively means that we can't fix quoted strings. So I had to
>> introduced a new quoted string.
>
> First of all, having an extension that changes the interpretation of, 
> say, \x
> followed by two hex digits would IMO be far less damaging than an 
> extension
> that changes the core syntax.

Fair enough.

>> Do you have any better suggestions?
>
> There are all sorts of ways to do it. Here's an obvious one: If the
> octet-value extension is enabled, any occurances of ${x} where x takes
> the form of a space-separated list of decimal values is replaced with
> a sequence of octets corresponding to each value. Hex is a bit more
> difficult since the sequences could be confused with a variable names, so
> if you want values in hex the simplest thing would be to invent a 
> different
> escape convention, e.g. something like $%x%.

After thinking more about this (and reviewing list of options provided 
by Philip), I would like to propose a variables-like syntax close to 
what you suggested above:
 $%<hex>
controlled by a new Sieve capability "string-escape".
This would give existing implementations an ability to provide the 
escaping functionality in both quoted and multiline strings, without the 
need to implement variables.
Any objections?
If people don't like this option, my next favorite would be to change 
the interpretation of \x.

Alexey




Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8J8J1IZ032615; Tue, 19 Sep 2006 01:19:01 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8J8J1dQ032614; Tue, 19 Sep 2006 01:19:01 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from rufus.isode.com (rufus.isode.com [62.3.217.251]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8J8IxrX032607 for <[e-mail]>; Tue, 19 Sep 2006 01:19:00 -0700 (MST) (envelope-from [e-mail])
Received: from [172.16.1.99] (shiny.isode.com [62.3.217.250])  by rufus.isode.com (submission channel) via TCP with ESMTPA  id <[e-mail]>; Tue, 19 Sep 2006 09:18:57 +0100
Message-ID: <[e-mail]>
Date: Tue, 19 Sep 2006 09:18:47 +0100
From: Alexey Melnikov <[e-mail]>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: Aaron Stone <[e-mail]>
CC: [e-mail]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
References: <[e-mail]>	 <[e-mail]>	 <[e-mail]> <[e-mail]>	 <[e-mail]>	 <[e-mail]> <1158615865.15870.27.camel@localhost>
In-Reply-To: <1158615865.15870.27.camel@localhost>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

Aaron Stone wrote:

>On Mon, 2006-09-18 at 15:12 -0600, Philip Guenther wrote:
>  
>
>>On Mon, 18 Sep 2006, Michael Haardt wrote:
>>    
>>
>>>On Mon, Sep 18, 2006 at 09:44:20AM -0700, Ned Freed wrote:
>>>      
>>>
>>>>There are all sorts of ways to do it. Here's an obvious one: If the
>>>>octet-value extension is enabled, any occurances of ${x} where x takes
>>>>the form of a space-separated list of decimal values is replaced with
>>>>a sequence of octets corresponding to each value.
>>>>        
>>>>
>>>What a brilliant idea! How about using the same syntax variables do,
>>>and embedding arbitrary data written as a function?
>>>      
>>>
>>You mean something like option (3) from my message of March 2005?
>>    http://www.imc.org/ietf-mta-filters/mail-archive/msg02018.html
>>    
>>
>Didn't we end up zapping namespaces from the variables spec?
>  
>
No, an extension can define a namespace for variables.



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8ILe9tl078183; Mon, 18 Sep 2006 14:40:09 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8ILe9C6078182; Mon, 18 Sep 2006 14:40:09 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from mail.serendipity.cx (IDENT:[e-mail] [66.92.2.87]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8ILe8jl078175 for <[e-mail]>; Mon, 18 Sep 2006 14:40:09 -0700 (MST) (envelope-from [e-mail])
Received: from [192.168.1.185] (localhost [127.0.0.1]) by mail.serendipity.cx (Postfix) with ESMTP id E39E56022631; Mon, 18 Sep 2006 14:40:05 -0700 (PDT)
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
From: Aaron Stone <[e-mail]>
To: Philip Guenther <[e-mail]>
Cc: [e-mail]
In-Reply-To: <[e-mail]>
References: <[e-mail]> <[e-mail]> <[e-mail]> <[e-mail]> <[e-mail]> <[e-mail]>
Content-Type: text/plain
Date: Mon, 18 Sep 2006 14:44:25 -0700
Message-Id: <1158615865.15870.27.camel@localhost>
Mime-Version: 1.0
X-Mailer: Evolution 2.6.3 
Content-Transfer-Encoding: 7bit
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

On Mon, 2006-09-18 at 15:12 -0600, Philip Guenther wrote:
> On Mon, 18 Sep 2006, Michael Haardt wrote:
> > On Mon, Sep 18, 2006 at 09:44:20AM -0700, Ned Freed wrote:
> >> There are all sorts of ways to do it. Here's an obvious one: If the
> >> octet-value extension is enabled, any occurances of ${x} where x takes
> >> the form of a space-separated list of decimal values is replaced with
> >> a sequence of octets corresponding to each value.
> >
> > What a brilliant idea! How about using the same syntax variables do,
> > and embedding arbitrary data written as a function?
> 
> You mean something like option (3) from my message of March 2005?
>     http://www.imc.org/ietf-mta-filters/mail-archive/msg02018.html

Didn't we end up zapping namespaces from the variables spec?

Aaron



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8ILDLVl076186; Mon, 18 Sep 2006 14:13:21 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8ILDKJL076185; Mon, 18 Sep 2006 14:13:20 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from smtp-out.sendmail.com (smtp-out.sendmail.com [209.246.26.45]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8ILDINe076179 for <[e-mail]>; Mon, 18 Sep 2006 14:13:19 -0700 (MST) (envelope-from [e-mail])
Received: from [10.201.0.58] (adsl-64-58-1-252.mho.net [64.58.1.252] (may be forged)) (authenticated bits=0) by foon.sendmail.com (Switch-3.2.4/Switch-3.2.0) with ESMTP id k8ILCnhv010450 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 18 Sep 2006 14:12:53 -0700
X-DKIM: Sendmail DKIM Filter v0.5.1 foon.sendmail.com k8ILCnhv010450
DKIM-Signature: a=rsa-sha1; c=relaxed/simple; d=sendmail.com; s=tls.dkim; t=1158613987; bh=I2D01kpyb7p8nMfyzimT39b1obE=; h=X-DomainKeys: DomainKey-Signature:Date:From:X-X-Sender:To:cc:Subject:In-Reply-To: Message-ID:References:MIME-Version:Content-Type; b=YPkS4B0vjfCsjIpK e5dyOeBKq3tb1mXORdqrsVRA4iw011Z2n923rhhuEzQ3ldZAQLA43fflQahFaxa2BCO mTYbVkINyOR5hOJMBYj3nI2qby7ytTZtN4dwspgy+f0GENnPKcnqL29Nt/kkQq5Bpb5 h8gE9goLg/k8uD4b1OLrY=
X-DomainKeys: Sendmail DomainKeys Filter v0.4.1 foon.sendmail.com k8ILCnhv010450
DomainKey-Signature: a=rsa-sha1; s=tls; d=sendmail.com; c=nofws; q=dns; h=date:from:x-x-sender:to:cc:subject:in-reply-to:message-id: references:mime-version:content-type; b=Xd3jakZE6y70OdOHW3ayJvr6twwd2WEHtcM1wFnhzPGfHjfCdCnUVDQ9lX29uPIU3 TrZLVjTAu69nU7vwCoi00Tx5ZJmOjOhLAf4WODdahFBMxVyDn9KLM2e1vwxA1V8bUmm Vl6B/PyAc7h5AreLxvXAFTg0+lYojoYA9U0kKuc=
Date: Mon, 18 Sep 2006 15:12:48 -0600
From: Philip Guenther <[e-mail]>
X-X-Sender: [e-mail]
To: Michael Haardt <[e-mail]>
cc: [e-mail]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
In-Reply-To: <[e-mail]>
Message-ID: <[e-mail]>
References: <[e-mail]> <[e-mail]> <[e-mail]> <[e-mail]> <[e-mail]>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

On Mon, 18 Sep 2006, Michael Haardt wrote:
> On Mon, Sep 18, 2006 at 09:44:20AM -0700, Ned Freed wrote:
>> There are all sorts of ways to do it. Here's an obvious one: If the
>> octet-value extension is enabled, any occurances of ${x} where x takes
>> the form of a space-separated list of decimal values is replaced with
>> a sequence of octets corresponding to each value.
>
> What a brilliant idea! How about using the same syntax variables do,
> and embedding arbitrary data written as a function?

You mean something like option (3) from my message of March 2005?
    http://www.imc.org/ietf-mta-filters/mail-archive/msg02018.html


Philip Guenther

("Over the years, I've developed my sense of deja vu so acutely that now I 
can remember things that *have* happened before ...")



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IKdciN073736; Mon, 18 Sep 2006 13:39:38 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8IKdcYR073735; Mon, 18 Sep 2006 13:39:38 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from ns4.neustar.com (ns4.neustar.com [156.154.24.139]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IKdb3M073727 for <[e-mail]>; Mon, 18 Sep 2006 13:39:38 -0700 (MST) (envelope-from [e-mail])
Received: from stiedprstage1.ietf.org (stiedprstage1.va.neustar.com [10.31.47.10]) by ns4.neustar.com (Postfix) with ESMTP id 0B7852AC73; Mon, 18 Sep 2006 20:39:32 +0000 (GMT)
Received: from ietf by stiedprstage1.ietf.org with local (Exim 4.43) id 1GPPtr-0000Z9-QI; Mon, 18 Sep 2006 16:39:31 -0400
X-test-idtracker: no
From: The IESG <[e-mail]>
To: IETF-Announce <[e-mail]>
Cc: Internet Architecture Board <[e-mail]>, RFC Editor <[e-mail]>, sieve mailing list <[e-mail]>, sieve chair <[e-mail]>, sieve chair <[e-mail]>
Subject: Protocol Action: 'SIEVE Email Filtering: Spamtest and  Virustest Extensions' to Proposed Standard 
Message-Id: <[e-mail]>
Date: Mon, 18 Sep 2006 16:39:31 -0400
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

The IESG has approved the following document:

- 'SIEVE Email Filtering: Spamtest and Virustest Extensions '
   <draft-ietf-sieve-spamtestbis-05.txt> as a Proposed Standard

This document is the product of the Sieve Mail Filtering Language 
Working Group. 

The IESG contact persons are Lisa Dusseault and Ted Hardie.

A URL of this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-sieve-spamtestbis-05.txt

Technical Summary
 
This is a revision of RFC 3685. This document adds 3 new tests to the
SIEVE email filtering language: "spamtest", "spamtestplus" and
"virustest". Those tests permit users to use simple, portable commands
for spam and virus tests on email messages.  

This document addresses issues raised by implementors. The document
contains straightforward clarifications, updates to references and
boilerplate. In addition to that it introduces a new extension
"spamtestplus", which is basically the same as "spamtest", but allows to
test scores in percents, while the original "spamtest" test only used the
1 to 10 scale. The introduction of new scale is the result of customer
feedback: users feel more comfortable using percents. 
 
Working Group Summary
 
The SIEVE WG has reviewed the draft and discussed it at a few meetings.
Last-call (and post last-call) reviews were performed by the following
people:
- Ned Freed
- Mark E. Mallett
- Arnt Gulbrandsen
- Aaron Stone

The WG discussed the issue of obsoleting the "spamtest" extension and
decided to keep it in the document, as currently there are several
deployments of "spamtest" and no deployments of "spamtestplus".
The WG will reevaluate this decision when the time comes to move the
document to Draft.

 
Protocol Quality
 
Lisa Dusseault reviewed this document for the IESG.

Note to RFC Editor
 
1.  Note that this draft obsoletes RFC3685.

2.  The reference to "I-D.newman-i18n-comparator" should be moved to the
"Normative" reference section.

3.  Please remove the version-by-version changes and add the summary of
changes here:

ADD:
   Appendix B. Important changes since RFC3685.

   Listed below are the major changes from the previous specification
   [RFC3685], which this one supersedes.

   1. A ":percent" argument has been added to the "spamtest" test adding
a new 0-100 numerical range for test results.

   2. A "spamtestplus" requires item has been added to indicate the
   presence of this extension in scripts.

   3. The "count" match type from [I-D.ietf-sieve-3431bis] can now be
used to determine whether a message was tested or not.

   4. Clarified that "test not done" also means "SIEVE system could not
   determine if a test was done".



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IKKrHf072314; Mon, 18 Sep 2006 13:20:53 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8IKKrBj072313; Mon, 18 Sep 2006 13:20:53 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from mout0.freenet.de (mout0.freenet.de [194.97.50.131]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IKKqj2072307 for <[e-mail]>; Mon, 18 Sep 2006 13:20:52 -0700 (MST) (envelope-from [e-mail])
Received: from [194.97.55.148] (helo=mx5.freenet.de) by mout0.freenet.de with esmtpa (Exim 4.62) (envelope-from <[e-mail]>) id 1GPPbn-00016Q-75 for [e-mail]; Mon, 18 Sep 2006 22:20:51 +0200
Received: from nostromo.freenet-ag.de ([194.97.7.6]) by mx5.freenet.de with esmtps (TLSv1:AES256-SHA:256) (Exim 4.62 #12) id 1GPPbn-0000BC-46 for [e-mail]; Mon, 18 Sep 2006 22:20:51 +0200
Received: from michael by nostromo.freenet-ag.de with local (ID michael) (Exim 4.62 #12) id 1GPPbm-0003DQ-FQ for [e-mail]; Mon, 18 Sep 2006 22:20:50 +0200
Date: Mon, 18 Sep 2006 22:20:50 +0200
From: Michael Haardt <[e-mail]>
To: [e-mail]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
Message-ID: <[e-mail]>
References: <[e-mail]> <[e-mail]> <[e-mail]> <[e-mail]>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <[e-mail]>
User-Agent: Mutt/1.5.6i
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

On Mon, Sep 18, 2006 at 09:44:20AM -0700, Ned Freed wrote:
> There are all sorts of ways to do it. Here's an obvious one: If the
> octet-value extension is enabled, any occurances of ${x} where x takes
> the form of a space-separated list of decimal values is replaced with
> a sequence of octets corresponding to each value.

What a brilliant idea! How about using the same syntax variables do,
and embedding arbitrary data written as a function?

  "${hex:deadbeef}"
  "${dec:222 173 190 239}"

Exim string operators work that way.  I didn't follow the variables
extension too close, because I consider it to be too expensive for my
needs.  If there are string operators already, adapt the above as needed,
but it would be both extensible and easier to read than backslash quoting.
If needed, someone could add "${from_base64:}".

If used without variables, the function would not expand its arguments
first, thus evaluate in linear time.  If used with variables, its argument
would be expanded first, like strings are expanded.

Michael



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IJvsWl070382; Mon, 18 Sep 2006 12:57:54 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8IJvsOS070381; Mon, 18 Sep 2006 12:57:54 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from mout2.freenet.de (mout2.freenet.de [194.97.50.155]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IJvr0v070375 for <[e-mail]>; Mon, 18 Sep 2006 12:57:54 -0700 (MST) (envelope-from [e-mail])
Received: from [194.97.55.242] (helo=mx9.freenet.de) by mout2.freenet.de with esmtpa (Exim 4.62) (envelope-from <[e-mail]>) id 1GPPFY-0002kB-4z for [e-mail]; Mon, 18 Sep 2006 21:57:52 +0200
Received: from nostromo.freenet-ag.de ([194.97.7.6]) by mx9.freenet.de with esmtps (TLSv1:AES256-SHA:256) (Exim 4.62 #12) id 1GPPFY-0005MQ-3m for [e-mail]; Mon, 18 Sep 2006 21:57:52 +0200
Received: from michael by nostromo.freenet-ag.de with local (ID michael) (Exim 4.62 #12) id 1GPPFX-0003CI-FQ for [e-mail]; Mon, 18 Sep 2006 21:57:51 +0200
Date: Mon, 18 Sep 2006 21:57:51 +0200
From: Michael Haardt <[e-mail]>
To: [e-mail]
Subject: Re: non-UTF-8 sequences in Sieve scripts
Message-ID: <[e-mail]>
References: <[e-mail]> <[e-mail]> <[e-mail]> <[e-mail]> <[e-mail]>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <[e-mail]>
User-Agent: Mutt/1.5.6i
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

On Mon, Sep 18, 2006 at 11:07:27AM -0400, Cyrus Daboo wrote:
> >I don't see the ambiguity.  MIME encodes the data in US-ASCII, and the
> >script only contains the MIME-encoded data.  Sieve knows nothing about
> >the MIME-encoding of content passed e.g. to vacation: Its _type_ may be
> >a Klingon sound file, not even text data, but MIME-encoded Sieve only
> >sees US-ASCII.
> 
> I cannot see any restriction that would prevent use of 
> Content-Transfer-Encoding: 8-bit, in which case the MIME parts are not 
> necessarily US-ASCII and not necessarily UTF-8, e.g. a text/plain part with 
> charset=iso8859-1.

The restriction is that the script is written in UTF-8.  MIME is
specifically made to transport arbitrary typed data through an environment
that allows only 7-bit US-ASCII, so no problem there.

With regard to Sieve, think of UTF-8 as your transport.  You can
encode arbitrary data, but you can not encode it arbitrarily, but
like 7-bit-only mail gateways, that's no problem for MIME.

Michael



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IIQS5U061887; Mon, 18 Sep 2006 11:26:28 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8IIQS6Q061885; Mon, 18 Sep 2006 11:26:28 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from mauve.mrochek.com (206.117.180.234.brandx.net [206.117.180.234] (may be forged)) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IIQR9l061879 for <[e-mail]>; Mon, 18 Sep 2006 11:26:27 -0700 (MST) (envelope-from [e-mail])
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <[e-mail]> for [e-mail]; Mon, 18 Sep 2006 11:26:26 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <[e-mail]>; Mon, 18 Sep 2006 11:26:24 -0700 (PDT)
Cc: [e-mail], Alexey Melnikov <[e-mail]>
To: Arnt Gulbrandsen <[e-mail]>
Message-id: <[e-mail]>
Date: Mon, 18 Sep 2006 11:18:15 -0700 (PDT)
From: Ned Freed <[e-mail]>
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
In-reply-to: "Your message dated Mon, 18 Sep 2006 11:31:07 +0200" <[e-mail]>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed
References: <[e-mail]> <[e-mail]>
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

> Alexey Melnikov writes:
> > Here is a strawman syntax for new quoted strings that would only allow
> > for valid UTF-8 sequences, but would also allow for escaped non-UTF-8
> > sequences:
> > ...

> The capability name needs changing.

> >   quoted-string      = old-quoted-string / new-quoted-string
> >                          ; "new-quoted-string" is not allowed as
> >                          ; a parameter to "require" action.
> >                          ; "new-quoted-string" is only allowed after
> >                          ; 'require "utf8-strings";'

> You're actually talking about non-utf8-compliant-strings. "utf8-strings"
> is a poor name for that.

> I find it very difficult to summon any enthusiasm for this extension.
> Please include at least three realistic examples in the draft to show
> that it has value.

The obvious one is that I, and I suspect lots of other people, have to deal
with unencoded 8bit in various charsets in message headers. I've tried the
"reject all mail containing raw 8bit in the headers" approach and find it
catches far too much legitimate (but malformed) mail - when I had such a policy
in place here at home my wife found it impossible to correspond with several
people she works with.

Interestingy, I actually find it's better to use this as a whilelisting
mechanism than a blacklisting mechanism, that is, I sometimes set up rules
that look for bogus 8bit sequences found only in legitimate mail and use
that to keep it out of the spam folder.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IGs1X4053559; Mon, 18 Sep 2006 09:54:01 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8IGs1RH053558; Mon, 18 Sep 2006 09:54:01 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from mauve.mrochek.com (206.117.180.234.brandx.net [206.117.180.234] (may be forged)) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IGs0sX053548 for <[e-mail]>; Mon, 18 Sep 2006 09:54:01 -0700 (MST) (envelope-from [e-mail])
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <[e-mail]> for [e-mail]; Mon, 18 Sep 2006 09:53:57 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <[e-mail]>; Mon, 18 Sep 2006 09:53:55 -0700 (PDT)
Cc: Michael Haardt <[e-mail]>, [e-mail]
To: Alexey Melnikov <[e-mail]>
Message-id: <[e-mail]>
Date: Mon, 18 Sep 2006 09:44:20 -0700 (PDT)
From: Ned Freed <[e-mail]>
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
In-reply-to: "Your message dated Mon, 18 Sep 2006 16:08:55 +0100" <[e-mail]>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed
References: <[e-mail]> <[e-mail]> <[e-mail]>
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

> Michael Haardt wrote:

> >On Sun, Sep 17, 2006 at 11:39:53PM +0100, Alexey Melnikov wrote:
> >
> >
> >>Here is a strawman syntax for new quoted strings that would only allow
> >>for valid UTF-8 sequences, but would also allow for escaped non-UTF-8
> >>sequences:
> >>
> >>  new-quoted-string  = "~" DQUOTE new-quoted-text DQUOTE
> >>
> >>
> >You introduce a new lexical token to Sieve, thus changing the Sieve
> >syntax.
> >
> Yes.

I have to agree with Michael - changing the basic syntax of sieve is
unacceptable.

> >I don't think the base spec allows that.
> >
> IMHO, if it explicitly prohibit such change, we need to change the base
> spec.

One of the key strengths of sieve is that the core syntax is both simple
and immutable. This makes it possible to perform syntax checks even if
you don't understand all the extensions a given script uses. Breaking 
this would IMO be hugely damaging.

> >Sieve has a very
> >small syntax, moving most stuff to the semantical layer for being very
> >extensible, pretty much like LISP does.
> >
> >The variables extension, for example, introduces the concept of expanding
> >strings, but only at the semantic level.  Syntactically, strings still
> >look the same.
> >
> >
> The problem is that RFC 3028 said that '\ <octet>' for any <octet> other
> than <\> or <"> SHOULD be interpreted as <octet>.
> This effectively means that we can't fix quoted strings. So I had to
> introduced a new quoted string.

First of all, having an extension that changes the interpretation of, say, \x
followed by two hex digits would IMO be far less damaging than an extension
that changes the core syntax.

> Do you have any better suggestions?

There are all sorts of ways to do it. Here's an obvious one: If the
octet-value extension is enabled, any occurances of ${x} where x takes
the form of a space-separated list of decimal values is replaced with
a sequence of octets corresponding to each value. Hex is a bit more
difficult since the sequences could be confused with a variable names, so
if you want values in hex the simplest thing would be to invent a different
escape convention, e.g. something like $%x%.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IFBV2R044099; Mon, 18 Sep 2006 08:11:31 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8IFBVgh044098; Mon, 18 Sep 2006 08:11:31 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from rufus.isode.com (rufus.isode.com [62.3.217.251]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IFBOTv044044 for <[e-mail]>; Mon, 18 Sep 2006 08:11:28 -0700 (MST) (envelope-from [e-mail])
Received: from [172.16.1.99] (shiny.isode.com [62.3.217.250])  by rufus.isode.com (submission channel) via TCP with ESMTPA  id <[e-mail]>; Mon, 18 Sep 2006 16:11:14 +0100
Message-ID: <[e-mail]>
Date: Mon, 18 Sep 2006 16:11:05 +0100
From: Alexey Melnikov <[e-mail]>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: Michael Haardt <[e-mail]>
CC: [e-mail]
Subject: Re: non-UTF-8 sequences in Sieve scripts
References: <[e-mail]> <[e-mail]> <[e-mail]> <[e-mail]>
In-Reply-To: <[e-mail]>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

Michael Haardt wrote:

>On Mon, Sep 18, 2006 at 10:05:19AM -0400, Cyrus Daboo wrote:
>  
>
>>But it also says:
>>
>>2.4.2.4. MIME Parts
>>
>>  In a few places, [MIME] body parts are represented as strings.  These
>>  parts include MIME headers and the body.  This provides a way of
>>  embedding typed data within a Sieve script so that, among other
>>  things, character sets other than UTF-8 can be used for output
>>  messages.
>>
>>i.e. there is ambiguity here, but the bottom line is non-utf-8 data can be 
>>used in current sieve scripts without violating the spec. That is the 
>>problem we want to solve, but we cannot outlaw non-utf-8 now without making 
>>many implementations non-compliant. What we want to do in the long run is 
>>deprecate use of non-utf-8 strings in favor of escaped utf-8 strings, which 
>>is what Alexey's strawman is leading to.
>>    
>>
>
>I don't see the ambiguity.  MIME encodes the data in US-ASCII, and the
>script only contains the MIME-encoded data.
>
What about 8bit and binary content transfer encodings?

>Sieve knows nothing about
>the MIME-encoding of content passed e.g. to vacation: Its _type_ may be
>a Klingon sound file, not even text data, but MIME-encoded Sieve only
>sees US-ASCII.
>
>We are looking at non-UTF-8 data that Sieve sees as such.
>
>I don't mean to outlaw non-UTF-8 in Sieve scripts, because I see no
>purpose in doing so.  But I don't want to legalise it either.  If many
>(how many?) implementations decided to implement a superset of RFC 3028,
>that's their decision.  I prefer to stay silent about that, and use a
>regular extension to offer that functionality.
>  
>
You opinion is noted. So basically you want a variant of option (2) from 
my poll.



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IF9Wtt043695; Mon, 18 Sep 2006 08:09:32 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8IF9WI5043694; Mon, 18 Sep 2006 08:09:32 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from rufus.isode.com (rufus.isode.com [62.3.217.251]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IF9IUJ043665 for <[e-mail]>; Mon, 18 Sep 2006 08:09:28 -0700 (MST) (envelope-from [e-mail])
Received: from [172.16.1.99] (shiny.isode.com [62.3.217.250])  by rufus.isode.com (submission channel) via TCP with ESMTPA  id <[e-mail]>; Mon, 18 Sep 2006 16:09:06 +0100
Message-ID: <[e-mail]>
Date: Mon, 18 Sep 2006 16:08:55 +0100
From: Alexey Melnikov <[e-mail]>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: Michael Haardt <[e-mail]>
CC: [e-mail]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
References: <[e-mail]> <[e-mail]>
In-Reply-To: <[e-mail]>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

Michael Haardt wrote:

>On Sun, Sep 17, 2006 at 11:39:53PM +0100, Alexey Melnikov wrote:
>  
>
>>Here is a strawman syntax for new quoted strings that would only allow
>>for valid UTF-8 sequences, but would also allow for escaped non-UTF-8
>>sequences:
>>
>>  new-quoted-string  = "~" DQUOTE new-quoted-text DQUOTE
>>    
>>
>You introduce a new lexical token to Sieve, thus changing the Sieve
>syntax.
>
Yes.

>I don't think the base spec allows that.
>
IMHO, if it explicitly prohibit such change, we need to change the base 
spec.

>Sieve has a very
>small syntax, moving most stuff to the semantical layer for being very
>extensible, pretty much like LISP does.
>
>The variables extension, for example, introduces the concept of expanding
>strings, but only at the semantic level.  Syntactically, strings still
>look the same.
>  
>
The problem is that RFC 3028 said that '\ <octet>' for any <octet> other 
than <\> or <"> SHOULD be interpreted as <octet>.
This effectively means that we can't fix quoted strings. So I had to 
introduced a new quoted string.

Do you have any better suggestions?



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IF7g6c043420; Mon, 18 Sep 2006 08:07:42 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8IF7gli043419; Mon, 18 Sep 2006 08:07:42 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from mulberrymail.com (static-71-240-120-213.pitt.east.verizon.net [71.240.120.213]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IF7eFv043393 for <[e-mail]>; Mon, 18 Sep 2006 08:07:40 -0700 (MST) (envelope-from [e-mail])
Received: from [17.101.34.128] ([17.101.34.128]) (authenticated bits=0) by mulberrymail.com (8.13.6/8.13.6) with ESMTP id k8IF7WBr027517 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 18 Sep 2006 11:07:34 -0400
Date: Mon, 18 Sep 2006 11:07:27 -0400
From: Cyrus Daboo <[e-mail]>
To: Michael Haardt <[e-mail]>, [e-mail]
Subject: Re: non-UTF-8 sequences in Sieve scripts
Message-ID: <[e-mail]>
In-Reply-To: <[e-mail]>
References: <[e-mail]> <[e-mail]> <[e-mail]> <[e-mail]>
X-Mailer: Mulberry/4.0.5 (Mac OS X)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=disabled  version=3.1.1
X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on  piper.mulberrymail.com
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

Hi Michael,

--On September 18, 2006 4:33:03 PM +0200 Michael Haardt 
<[e-mail]> wrote:

> I don't see the ambiguity.  MIME encodes the data in US-ASCII, and the
> script only contains the MIME-encoded data.  Sieve knows nothing about
> the MIME-encoding of content passed e.g. to vacation: Its _type_ may be
> a Klingon sound file, not even text data, but MIME-encoded Sieve only
> sees US-ASCII.

I cannot see any restriction that would prevent use of 
Content-Transfer-Encoding: 8-bit, in which case the MIME parts are not 
necessarily US-ASCII and not necessarily UTF-8, e.g. a text/plain part with 
charset=iso8859-1.

-- 
Cyrus Daboo



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IF6nns043340; Mon, 18 Sep 2006 08:06:49 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8IF6nH5043339; Mon, 18 Sep 2006 08:06:49 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from mulberrymail.com (static-71-240-120-213.pitt.east.verizon.net [71.240.120.213]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IF6ido043330 for <[e-mail]>; Mon, 18 Sep 2006 08:06:48 -0700 (MST) (envelope-from [e-mail])
Received: from [17.101.34.128] ([17.101.34.128]) (authenticated bits=0) by mulberrymail.com (8.13.6/8.13.6) with ESMTP id k8IF6ZYg027510 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 18 Sep 2006 11:06:38 -0400
Date: Mon, 18 Sep 2006 11:06:30 -0400
From: Cyrus Daboo <[e-mail]>
To: Michael Haardt <[e-mail]>, [e-mail]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
Message-ID: <[e-mail]>
In-Reply-To: <[e-mail]>
References: <[e-mail]> <[e-mail]>
X-Mailer: Mulberry/4.0.5 (Mac OS X)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=disabled  version=3.1.1
X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on  piper.mulberrymail.com
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

Hi Michael,

--On September 18, 2006 4:51:49 PM +0200 Michael Haardt 
<[e-mail]> wrote:

>> Here is a strawman syntax for new quoted strings that would only allow
>> for valid UTF-8 sequences, but would also allow for escaped non-UTF-8
>> sequences:
>>
>>   new-quoted-string  = "~" DQUOTE new-quoted-text DQUOTE
>
> You introduce a new lexical token to Sieve, thus changing the Sieve
> syntax.  I don't think the base spec allows that.  Sieve has a very
> small syntax, moving most stuff to the semantical layer for being very
> extensible, pretty much like LISP does.
>
> The variables extension, for example, introduces the concept of expanding
> strings, but only at the semantic level.  Syntactically, strings still
> look the same.

How about if 'require string-escape' is present in the script, then ALL 
strings following that will be of the new 'escaped' form. i.e. a global 
switch covering all strings used in the script.

-- 
Cyrus Daboo



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IEppuf042208; Mon, 18 Sep 2006 07:51:51 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8IEppxI042207; Mon, 18 Sep 2006 07:51:51 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from mout1.freenet.de (mout1.freenet.de [194.97.50.132]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IEpo2m042200 for <[e-mail]>; Mon, 18 Sep 2006 07:51:51 -0700 (MST) (envelope-from [e-mail])
Received: from [194.97.50.136] (helo=mx3.freenet.de) by mout1.freenet.de with esmtpa (Exim 4.62) (envelope-from <[e-mail]>) id 1GPKTN-0003ch-VH for [e-mail]; Mon, 18 Sep 2006 16:51:49 +0200
Received: from nostromo.freenet-ag.de ([194.97.7.6]) by mx3.freenet.de with esmtps (TLSv1:AES256-SHA:256) (Exim 4.62 #12) id 1GPKTN-0001Yd-Sm for [e-mail]; Mon, 18 Sep 2006 16:51:49 +0200
Received: from michael by nostromo.freenet-ag.de with local (ID michael) (Exim 4.62 #12) id 1GPKTN-00034g-4V for [e-mail]; Mon, 18 Sep 2006 16:51:49 +0200
Date: Mon, 18 Sep 2006 16:51:49 +0200
From: Michael Haardt <[e-mail]>
To: [e-mail]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
Message-ID: <[e-mail]>
References: <[e-mail]>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <[e-mail]>
User-Agent: Mutt/1.5.6i
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

On Sun, Sep 17, 2006 at 11:39:53PM +0100, Alexey Melnikov wrote:
> Here is a strawman syntax for new quoted strings that would only allow
> for valid UTF-8 sequences, but would also allow for escaped non-UTF-8
> sequences:
> 
>   new-quoted-string  = "~" DQUOTE new-quoted-text DQUOTE

You introduce a new lexical token to Sieve, thus changing the Sieve
syntax.  I don't think the base spec allows that.  Sieve has a very
small syntax, moving most stuff to the semantical layer for being very
extensible, pretty much like LISP does.

The variables extension, for example, introduces the concept of expanding
strings, but only at the semantic level.  Syntactically, strings still
look the same.

Michael



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IEX7IG040572; Mon, 18 Sep 2006 07:33:07 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8IEX7ai040571; Mon, 18 Sep 2006 07:33:07 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from mout1.freenet.de (mout1.freenet.de [194.97.50.132]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IEX6Ov040565 for <[e-mail]>; Mon, 18 Sep 2006 07:33:06 -0700 (MST) (envelope-from [e-mail])
Received: from [194.97.50.138] (helo=mx0.freenet.de) by mout1.freenet.de with esmtpa (Exim 4.62) (envelope-from <[e-mail]>) id 1GPKBF-0007Hq-63 for [e-mail]; Mon, 18 Sep 2006 16:33:05 +0200
Received: from nostromo.freenet-ag.de ([194.97.7.6]) by mx0.freenet.de with esmtps (TLSv1:AES256-SHA:256) (Exim 4.62 #12) id 1GPKBF-0001fx-4Z for [e-mail]; Mon, 18 Sep 2006 16:33:05 +0200
Received: from michael by nostromo.freenet-ag.de with local (ID michael) (Exim 4.62 #12) id 1GPKBD-000344-Q7 for [e-mail]; Mon, 18 Sep 2006 16:33:03 +0200
Date: Mon, 18 Sep 2006 16:33:03 +0200
From: Michael Haardt <[e-mail]>
To: [e-mail]
Subject: Re: non-UTF-8 sequences in Sieve scripts
Message-ID: <[e-mail]>
References: <[e-mail]> <[e-mail]> <[e-mail]>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <[e-mail]>
User-Agent: Mutt/1.5.6i
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

On Mon, Sep 18, 2006 at 10:05:19AM -0400, Cyrus Daboo wrote:
> But it also says:
> 
> 2.4.2.4. MIME Parts
> 
>   In a few places, [MIME] body parts are represented as strings.  These
>   parts include MIME headers and the body.  This provides a way of
>   embedding typed data within a Sieve script so that, among other
>   things, character sets other than UTF-8 can be used for output
>   messages.
> 
> i.e. there is ambiguity here, but the bottom line is non-utf-8 data can be 
> used in current sieve scripts without violating the spec. That is the 
> problem we want to solve, but we cannot outlaw non-utf-8 now without making 
> many implementations non-compliant. What we want to do in the long run is 
> deprecate use of non-utf-8 strings in favor of escaped utf-8 strings, which 
> is what Alexey's strawman is leading to.

I don't see the ambiguity.  MIME encodes the data in US-ASCII, and the
script only contains the MIME-encoded data.  Sieve knows nothing about
the MIME-encoding of content passed e.g. to vacation: Its _type_ may be
a Klingon sound file, not even text data, but MIME-encoded Sieve only
sees US-ASCII.

We are looking at non-UTF-8 data that Sieve sees as such.

I don't mean to outlaw non-UTF-8 in Sieve scripts, because I see no
purpose in doing so.  But I don't want to legalise it either.  If many
(how many?) implementations decided to implement a superset of RFC 3028,
that's their decision.  I prefer to stay silent about that, and use a
regular extension to offer that functionality.

Michael



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IEFiSG038169; Mon, 18 Sep 2006 07:15:44 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8IEFhqt038168; Mon, 18 Sep 2006 07:15:43 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from mulberrymail.com (static-71-240-120-213.pitt.east.verizon.net [71.240.120.213]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IEFgPw038157; Mon, 18 Sep 2006 07:15:43 -0700 (MST) (envelope-from [e-mail])
Received: from [17.101.34.128] ([17.101.34.128]) (authenticated bits=0) by mulberrymail.com (8.13.6/8.13.6) with ESMTP id k8IEFbpQ027394 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 18 Sep 2006 10:15:39 -0400
Date: Mon, 18 Sep 2006 10:15:32 -0400
From: Cyrus Daboo <[e-mail]>
To: IMAP Extensions WG <[e-mail]>, SIEVE <[e-mail]>
Subject: CalDAV uses collations - please review
Message-ID: <[e-mail]>
X-Mailer: Mulberry/4.0.5 (Mac OS X)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=disabled  version=3.1.1
X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on  piper.mulberrymail.com
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

Hi folks,
Following IESG review, we got push-back on the text comparison operations 
used in CalDAV (Calendaring Extensions to WebDAV). As a result we have 
added support for collations as defined in 
<http://www.ietf.org/internet-drafts/draft-newman-i18n-comparator-14.txt>. 
We would appreciate it if the collation experts in the IMAP and SIEVE 
communities could review the changes and use of collations in this 
specification to ensure we are doing the right thing.

The link to the new draft is here:

<http://www.ietf.org/internet-drafts/draft-dusseault-caldav-15.txt>

An HTML diff from the previous version is here:

<http://ietf.osafoundation.org/caldav/draft-dusseault-caldav-15-from-14.diff.html>

The key section is 7.5 that describes use of collations in CalDAV.

Discussion of CalDAV is taking place at 
<mailto:[e-mail]> so please post any comments there or 
send directly to me or the other co-authors, thanks.

-- 
Cyrus Daboo



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IE5aNX037037; Mon, 18 Sep 2006 07:05:36 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8IE5aSO037036; Mon, 18 Sep 2006 07:05:36 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from mulberrymail.com (static-71-240-120-213.pitt.east.verizon.net [71.240.120.213]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IE5Z5W037029 for <[e-mail]>; Mon, 18 Sep 2006 07:05:35 -0700 (MST) (envelope-from [e-mail])
Received: from [17.101.34.128] ([17.101.34.128]) (authenticated bits=0) by mulberrymail.com (8.13.6/8.13.6) with ESMTP id k8IE5SIS027359 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 18 Sep 2006 10:05:30 -0400
Date: Mon, 18 Sep 2006 10:05:19 -0400
From: Cyrus Daboo <[e-mail]>
To: Michael Haardt <[e-mail]>, [e-mail]
Subject: Re: non-UTF-8 sequences in Sieve scripts
Message-ID: <[e-mail]>
In-Reply-To: <[e-mail]>
References: <[e-mail]> <[e-mail]>
X-Mailer: Mulberry/4.0.5 (Mac OS X)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=disabled  version=3.1.1
X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on  piper.mulberrymail.com
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

Hi Michael,

--On September 18, 2006 12:06:43 PM +0200 Michael Haardt 
<[e-mail]> wrote:

> I prefer that the base spec continues to only describe UTF-8, staying
> silent on implementations that take the freedom to violate RFC 3028,
> which says in 2.1:
>
>    The language is represented in UTF-8, as specified in [UTF-8].

But it also says:

2.4.2.4. MIME Parts

   In a few places, [MIME] body parts are represented as strings.  These
   parts include MIME headers and the body.  This provides a way of
   embedding typed data within a Sieve script so that, among other
   things, character sets other than UTF-8 can be used for output
   messages.

i.e. there is ambiguity here, but the bottom line is non-utf-8 data can be 
used in current sieve scripts without violating the spec. That is the 
problem we want to solve, but we cannot outlaw non-utf-8 now without making 
many implementations non-compliant. What we want to do in the long run is 
deprecate use of non-utf-8 strings in favor of escaped utf-8 strings, which 
is what Alexey's strawman is leading to.

-- 
Cyrus Daboo



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IDxfQv036666; Mon, 18 Sep 2006 06:59:41 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8IDxfFU036665; Mon, 18 Sep 2006 06:59:41 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from mulberrymail.com (static-71-240-120-213.pitt.east.verizon.net [71.240.120.213]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IDxdBT036659 for <[e-mail]>; Mon, 18 Sep 2006 06:59:40 -0700 (MST) (envelope-from [e-mail])
Received: from [17.101.34.128] ([17.101.34.128]) (authenticated bits=0) by mulberrymail.com (8.13.6/8.13.6) with ESMTP id k8IDxT7i027333 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 18 Sep 2006 09:59:31 -0400
Date: Mon, 18 Sep 2006 09:59:20 -0400
From: Cyrus Daboo <[e-mail]>
To: Arnt Gulbrandsen <[e-mail]>, [e-mail]
cc: Alexey Melnikov <[e-mail]>
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
Message-ID: <[e-mail]>
In-Reply-To: <[e-mail]>
References: <[e-mail]> <[e-mail]>
X-Mailer: Mulberry/4.0.5 (Mac OS X)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=disabled  version=3.1.1
X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on  piper.mulberrymail.com
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

Hi Arnt,

--On September 18, 2006 11:31:07 AM +0200 Arnt Gulbrandsen 
<[e-mail]> wrote:

>>   quoted-string      = old-quoted-string / new-quoted-string
>>                          ; "new-quoted-string" is not allowed as
>>                          ; a parameter to "require" action.
>>                          ; "new-quoted-string" is only allowed after
>>                          ; 'require "utf8-strings";'
>
> You're actually talking about non-utf8-compliant-strings. "utf8-strings"
> is a poor name for that.

Not quite - the new mechanism allows for non-utf8 strings to be 'encoded' 
as a utf-8 string via an escaping mechanism. As such strings (and indeed 
the entire script) is still utf-8 (except where current implementations 
already violate that). Perhaps a better better capability name is 
'string-escape'.

-- 
Cyrus Daboo



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IA6lPR011718; Mon, 18 Sep 2006 03:06:47 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8IA6lhN011716; Mon, 18 Sep 2006 03:06:47 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from mout2.freenet.de (mout2.freenet.de [194.97.50.155]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8IA6jQj011598 for <[e-mail]>; Mon, 18 Sep 2006 03:06:45 -0700 (MST) (envelope-from [e-mail])
Received: from [194.97.55.147] (helo=mx4.freenet.de) by mout2.freenet.de with esmtpa (Exim 4.62) (envelope-from <[e-mail]>) id 1GPG1T-0007Ad-V0 for [e-mail]; Mon, 18 Sep 2006 12:06:43 +0200
Received: from nostromo.freenet-ag.de ([194.97.7.6]) by mx4.freenet.de with esmtps (TLSv1:AES256-SHA:256) (Exim 4.62 #12) id 1GPG1T-0005ey-Tk for [e-mail]; Mon, 18 Sep 2006 12:06:43 +0200
Received: from michael by nostromo.freenet-ag.de with local (ID michael) (Exim 4.62 #12) id 1GPG1T-0002r6-BV for [e-mail]; Mon, 18 Sep 2006 12:06:43 +0200
Date: Mon, 18 Sep 2006 12:06:43 +0200
From: Michael Haardt <[e-mail]>
To: [e-mail]
Subject: Re: non-UTF-8 sequences in Sieve scripts
Message-ID: <[e-mail]>
References: <[e-mail]>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <[e-mail]>
User-Agent: Mutt/1.5.6i
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

On Fri, Sep 15, 2006 at 12:34:16AM +0100, Alexey Melnikov wrote:
> 1). draft-ietf-sieve-3028bis stays as it is now, i.e. it allows for 
> non-UTF-8 sequences, but recommends UTF-8. Eventually implementations 
> will implement new strings with proper escaping mechanism, but existing 
> strings will stay as described till the end of time.

I dislike allowing non-UTF-8 sequences very strongly, because it forbids
to convert a Sieve filter to other character sets without scanning it
for strings.

> 2). draft-ietf-sieve-3028bis gets updated to only describe UTF-8 
> sequences with no escaping (*). The draft can describe existing 
> situation with non-UTF-8 sequences, or it can remain completely silent 
> on the issue. This wouldn't affect existing implementations, as long as 
> non-UTF-8 sequences in scripts are not explicitly prohibited. The draft 
> will also explicitly state that future revisions of this document would 
> not allow for non-UTF-8 sequences.

I prefer that the base spec continues to only describe UTF-8, staying
silent on implementations that take the freedom to violate RFC 3028,
which says in 2.1:

   The language is represented in UTF-8, as specified in [UTF-8].

Whoever decides to leave UTF-8 knows the risks of doing so.
We decided "envelope-auth" should be introduced as extension, although
implementations simply offered it before.  I see a similar situation now:
Some implementations offer a super set of RFC 3028.  To me, that is not a
reason to extend the base spec, but a reason to introduce an extension.
That way, we can be sure all Sieve implementations will conform to the
new spec.

On the other side, I don't think we gain something by saying: "You MUST
not use anything but UTF-8, or you MUST NOT stick the Sieve logo on your
implementation." ;-)

An extension to introduce arbitrary octet sequences encoded as UTF-8
(or ASCII) character sequences is fine with me, but I doubt its use
for the base specification, as it is most useful when working with
raw data, which the base spec does not offer.

Michael



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8I9du8Y007895; Mon, 18 Sep 2006 02:39:56 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8I9duUm007894; Mon, 18 Sep 2006 02:39:56 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from rufus.isode.com (rufus.isode.com [62.3.217.251]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8I9dtZ2007887 for <[e-mail]>; Mon, 18 Sep 2006 02:39:55 -0700 (MST) (envelope-from [e-mail])
Received: from [172.16.1.99] (shiny.isode.com [62.3.217.250])  by rufus.isode.com (submission channel) via TCP with ESMTPA  id <RQ5pagBQGH=[e-mail]>; Mon, 18 Sep 2006 10:39:54 +0100
Message-ID: <[e-mail]>
Date: Mon, 18 Sep 2006 10:39:45 +0100
From: Alexey Melnikov <[e-mail]>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915
X-Accept-Language: en-us, en
To: Arnt Gulbrandsen <[e-mail]>
CC: [e-mail]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
References: <[e-mail]> <[e-mail]>
In-Reply-To: <[e-mail]>
MIME-version: 1.0
Content-type: text/plain; charset="ISO-8859-1"; format="flowed"
Content-transfer-encoding: 7bit
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

Arnt Gulbrandsen wrote:

> Alexey Melnikov writes:
>
>> Here is a strawman syntax for new quoted strings that would only allow
>> for valid UTF-8 sequences, but would also allow for escaped non-UTF-8
>> sequences:
>> ...
>
> The capability name needs changing.
>
>>   quoted-string      = old-quoted-string / new-quoted-string
>>                          ; "new-quoted-string" is not allowed as
>>                          ; a parameter to "require" action.
>>                          ; "new-quoted-string" is only allowed after
>>                          ; 'require "utf8-strings";'
>
> You're actually talking about non-utf8-compliant-strings. 
> "utf8-strings" is a poor name for that.

Ok.

> I find it very difficult to summon any enthusiasm for this extension. 
> Please include at least three realistic examples in the draft to show 
> that it has value.

I will refer you to the following messages in the archive:
<http://www.imc.org/ietf-mta-filters/mail-archive/msg03105.html>
<http://www.imc.org/ietf-mta-filters/mail-archive/msg03147.html>



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8I9bKOt007718; Mon, 18 Sep 2006 02:37:20 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8I9bKpB007717; Mon, 18 Sep 2006 02:37:20 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from kalyani.oryx.com (kalyani.oryx.com [195.30.37.30]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8I9bJuK007711 for <[e-mail]>; Mon, 18 Sep 2006 02:37:20 -0700 (MST) (envelope-from [e-mail])
Received: from libertango.oryx.com (libertango.oryx.com [195.30.37.9]) by kalyani.oryx.com (Postfix) with ESMTP id A5FDB4ACBD for <[e-mail]>; Mon, 18 Sep 2006 11:37:18 +0200 (CEST)
Message-Id: <[e-mail]>
Date: Mon, 18 Sep 2006 11:38:28 +0200
From: Arnt Gulbrandsen <[e-mail]>
To: [e-mail]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
References: <[e-mail]>
In-Reply-To: <[e-mail]>
Content-Type: text/plain; format=flowed
MIME-Version: 1.0
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

Alexey reminded me of some archived posts I'd forgotten. Please ignore 
my previous message.



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8I9U01X006987; Mon, 18 Sep 2006 02:30:00 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8I9U0iK006986; Mon, 18 Sep 2006 02:30:00 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from kalyani.oryx.com (kalyani.oryx.com [195.30.37.30]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8I9TwjK006979 for <[e-mail]>; Mon, 18 Sep 2006 02:29:59 -0700 (MST) (envelope-from [e-mail])
Received: from libertango.oryx.com (libertango.oryx.com [195.30.37.9]) by kalyani.oryx.com (Postfix) with ESMTP id ABB4F4ACBD; Mon, 18 Sep 2006 11:29:57 +0200 (CEST)
Message-Id: <[e-mail]>
Date: Mon, 18 Sep 2006 11:31:07 +0200
From: Arnt Gulbrandsen <[e-mail]>
To: [e-mail]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
Cc: Alexey Melnikov <[e-mail]>
References: <[e-mail]>
In-Reply-To: <[e-mail]>
Content-Type: text/plain; format=flowed
MIME-Version: 1.0
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

Alexey Melnikov writes:
> Here is a strawman syntax for new quoted strings that would only allow
> for valid UTF-8 sequences, but would also allow for escaped non-UTF-8
> sequences:
> ...

The capability name needs changing.

>   quoted-string      = old-quoted-string / new-quoted-string
>                          ; "new-quoted-string" is not allowed as
>                          ; a parameter to "require" action.
>                          ; "new-quoted-string" is only allowed after
>                          ; 'require "utf8-strings";'

You're actually talking about non-utf8-compliant-strings. "utf8-strings" 
is a poor name for that.

I find it very difficult to summon any enthusiasm for this extension. 
Please include at least three realistic examples in the draft to show 
that it has value.

Arnt



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8I94Ztu004146; Mon, 18 Sep 2006 02:04:35 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8I94ZUs004145; Mon, 18 Sep 2006 02:04:35 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from rufus.isode.com (rufus.isode.com [62.3.217.251]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8I94Ynv004139 for <[e-mail]>; Mon, 18 Sep 2006 02:04:34 -0700 (MST) (envelope-from [e-mail])
Received: from [172.16.1.99] (shiny.isode.com [62.3.217.250])  by rufus.isode.com (submission channel) via TCP with ESMTPA  id <[e-mail]>; Mon, 18 Sep 2006 10:04:33 +0100
Message-ID: <[e-mail]>
Date: Sun, 17 Sep 2006 23:39:53 +0100
From: Alexey Melnikov <[e-mail]>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: MTA filtering mailing list <[e-mail]>
Subject: Proposal for escaping on non-UTF-8 sequences in Sieve
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

Here is a strawman syntax for new quoted strings that would only allow
for valid UTF-8 sequences, but would also allow for escaped non-UTF-8
sequences:

   new-quoted-string  = "~" DQUOTE new-quoted-text DQUOTE

   new-quoted-text    = *(utf8-quoted-safe / quoted-special /
                          quoted-non-utf8)

   not-qspecial       = %x01-09 / %x0B-0C / %x0E-21 / %x23-5B / %x5D-7F /
                        UTF8-2 / UTF8-3 / UTF8-4
                          ; a single Unicode character other than NUL,
                          ; CR, or LF, represented in UTF-8

   old-quoted-string  = DQUOTE quoted-text DQUOTE

   quoted-non-utf8    = "\" "x" 2*HEXDIG
                          ; represents a hex encoded octet

   quoted-string      = old-quoted-string / new-quoted-string
                          ; "new-quoted-string" is not allowed as
                          ; a parameter to "require" action.
                          ; "new-quoted-string" is only allowed after
                          ; 'require "utf8-strings";'

   utf8-quoted-safe   = CRLF / not-qspecial
                          ; either a CRLF pair, OR a single octet other
                          ; than NUL, CR, LF, double-quote, or backslash

As quoted strings can already contain multiple lines, I don't think we
need to introduce a new variant of the "multi-line" string. Please let
me know if you disagree.




Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8FFDagS085801; Fri, 15 Sep 2006 08:13:36 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8FFDZAU085794; Fri, 15 Sep 2006 08:13:35 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from mauve.mrochek.com (206.117.180.234.brandx.net [206.117.180.234] (may be forged)) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8FFDUXI085782 for <[e-mail]>; Fri, 15 Sep 2006 08:13:31 -0700 (MST) (envelope-from [e-mail])
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <[e-mail]> for [e-mail]; Fri, 15 Sep 2006 08:13:24 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <[e-mail]>; Fri, 15 Sep 2006 08:13:22 -0700 (PDT)
Cc: Ned Freed <[e-mail]>, MTA filtering mailing list <[e-mail]>
To: Alexey Melnikov <[e-mail]>
Message-id: <[e-mail]>
Date: Fri, 15 Sep 2006 08:12:57 -0700 (PDT)
From: Ned Freed <[e-mail]>
Subject: Re: non-UTF-8 sequences in Sieve scripts
In-reply-to: "Your message dated Fri, 15 Sep 2006 15:22:06 +0100" <[e-mail]>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed
References: <[e-mail]> <[e-mail]> <[e-mail]>
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

> Ned Freed wrote:

> >
> >> Hi everyone,
> >
> >
> >> I think there is rough consensus to have an extension that would
> >> introduce new strings with proper escaping mechanism. The new strings
> >> will allow for UTF-8 (as is) + non-UTF-8 can be escaped. As nobody
> >> volunteered to do this, I will write a short proposal on this.
> >
> >
> >> But I would like to get general feeling of the Working Group about
> >> non-UTF-8 sequences in existing 3028bis strings.
> >> draft-ietf-sieve-3028bis-09.txt doesn't prohibit them. However, what
> >> should our long term solution be? I see two choices:
> >
> >
> >> 1). draft-ietf-sieve-3028bis stays as it is now, i.e. it allows for
> >> non-UTF-8 sequences, but recommends UTF-8. Eventually implementations
> >> will implement new strings with proper escaping mechanism, but existing
> >> strings will stay as described till the end of time.
> >
> >
> >> 2). draft-ietf-sieve-3028bis gets updated to only describe UTF-8
> >> sequences with no escaping (*). The draft can describe existing
> >> situation with non-UTF-8 sequences, or it can remain completely silent
> >> on the issue. This wouldn't affect existing implementations, as long as
> >> non-UTF-8 sequences in scripts are not explicitly prohibited. The draft
> >> will also explicitly state that future revisions of this document would
> >> not allow for non-UTF-8 sequences.
> >
> >> =====
> >> So basically 2) would state the desire to deprecate use of unescaped
> >> non-UTF-8 sequences in scripts, while 1) would allow for them
> >> indefinitely.
> >
> >> Do people see other choices?
> >
> > First of all, I think we need to describe whatever escaping mechanism
> > we decide
> > on, preferably in the base document. I don't think it is reasonable to
> > suggest
> > abandoning a capability, let alone forbidding it, unless we have the
> > alternative in place.

> Right. I will post a proposal next week.
> I am not sure I want to have it in the base spec, as it will mean
> delaying its publications, but we can argue this point separately later.

> > Once we have the alternative defined I think the right approach is
> > somewhere
> > between (1) and (2). (2) goes too far IMO by saying that deprecation will
> > happen in the future. I think the correct course is to say that the
> > ability to use non-UTF-8 _may_ be abandoned in the future and to plan
> > accordingly.

> Right. In general case it is not possible to put requirements on future
> specs.

> Ok, if (2) is replaced with what you've proposed, which choice would you
> prefer?

Definitely (2). People need to be warned that this may change in the future.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8FEMO4O078011; Fri, 15 Sep 2006 07:22:24 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8FEMOXY078010; Fri, 15 Sep 2006 07:22:24 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from rufus.isode.com (rufus.isode.com [62.3.217.251]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8FEMMJV078003 for <[e-mail]>; Fri, 15 Sep 2006 07:22:23 -0700 (MST) (envelope-from [e-mail])
Received: from [172.16.1.99] (shiny.isode.com [62.3.217.250])  by rufus.isode.com (submission channel) via TCP with ESMTPA  id <[e-mail]>; Fri, 15 Sep 2006 15:22:21 +0100
Message-ID: <[e-mail]>
Date: Fri, 15 Sep 2006 15:22:06 +0100
From: Alexey Melnikov <[e-mail]>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: Ned Freed <[e-mail]>
CC: MTA filtering mailing list <[e-mail]>
Subject: Re: non-UTF-8 sequences in Sieve scripts
References: <[e-mail]> <[e-mail]>
In-Reply-To: <[e-mail]>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

Ned Freed wrote:

>
>> Hi everyone,
>
>
>> I think there is rough consensus to have an extension that would
>> introduce new strings with proper escaping mechanism. The new strings
>> will allow for UTF-8 (as is) + non-UTF-8 can be escaped. As nobody
>> volunteered to do this, I will write a short proposal on this.
>
>
>> But I would like to get general feeling of the Working Group about
>> non-UTF-8 sequences in existing 3028bis strings.
>> draft-ietf-sieve-3028bis-09.txt doesn't prohibit them. However, what
>> should our long term solution be? I see two choices:
>
>
>> 1). draft-ietf-sieve-3028bis stays as it is now, i.e. it allows for
>> non-UTF-8 sequences, but recommends UTF-8. Eventually implementations
>> will implement new strings with proper escaping mechanism, but existing
>> strings will stay as described till the end of time.
>
>
>> 2). draft-ietf-sieve-3028bis gets updated to only describe UTF-8
>> sequences with no escaping (*). The draft can describe existing
>> situation with non-UTF-8 sequences, or it can remain completely silent
>> on the issue. This wouldn't affect existing implementations, as long as
>> non-UTF-8 sequences in scripts are not explicitly prohibited. The draft
>> will also explicitly state that future revisions of this document would
>> not allow for non-UTF-8 sequences.
>
>> =====
>> So basically 2) would state the desire to deprecate use of unescaped
>> non-UTF-8 sequences in scripts, while 1) would allow for them 
>> indefinitely.
>
>> Do people see other choices?
>
> First of all, I think we need to describe whatever escaping mechanism 
> we decide
> on, preferably in the base document. I don't think it is reasonable to 
> suggest
> abandoning a capability, let alone forbidding it, unless we have the
> alternative in place.

Right. I will post a proposal next week.
I am not sure I want to have it in the base spec, as it will mean 
delaying its publications, but we can argue this point separately later.

> Once we have the alternative defined I think the right approach is 
> somewhere
> between (1) and (2). (2) goes too far IMO by saying that deprecation will
> happen in the future. I think the correct course is to say that the
> ability to use non-UTF-8 _may_ be abandoned in the future and to plan
> accordingly.

Right. In general case it is not possible to put requirements on future 
specs.

Ok, if (2) is replaced with what you've proposed, which choice would you 
prefer?



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8FE8coZ075818; Fri, 15 Sep 2006 07:08:38 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8FE8c4o075816; Fri, 15 Sep 2006 07:08:38 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from mauve.mrochek.com (206.117.180.234.brandx.net [206.117.180.234] (may be forged)) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8FE8Yow075797 for <[e-mail]>; Fri, 15 Sep 2006 07:08:35 -0700 (MST) (envelope-from [e-mail])
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <[e-mail]> for [e-mail]; Fri, 15 Sep 2006 07:08:32 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <[e-mail]>; Fri, 15 Sep 2006 07:08:31 -0700 (PDT)
Cc: MTA filtering mailing list <[e-mail]>
To: Alexey Melnikov <[e-mail]>
Message-id: <[e-mail]>
Date: Fri, 15 Sep 2006 07:01:40 -0700 (PDT)
From: Ned Freed <[e-mail]>
Subject: Re: non-UTF-8 sequences in Sieve scripts
In-reply-to: "Your message dated Fri, 15 Sep 2006 00:34:16 +0100" <[e-mail]>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format=flowed
References: <[e-mail]>
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

> Hi everyone,

> I think there is rough consensus to have an extension that would
> introduce new strings with proper escaping mechanism. The new strings
> will allow for UTF-8 (as is) + non-UTF-8 can be escaped. As nobody
> volunteered to do this, I will write a short proposal on this.

> But I would like to get general feeling of the Working Group about
> non-UTF-8 sequences in existing 3028bis strings.
> draft-ietf-sieve-3028bis-09.txt doesn't prohibit them. However, what
> should our long term solution be? I see two choices:

> 1). draft-ietf-sieve-3028bis stays as it is now, i.e. it allows for
> non-UTF-8 sequences, but recommends UTF-8. Eventually implementations
> will implement new strings with proper escaping mechanism, but existing
> strings will stay as described till the end of time.

> 2). draft-ietf-sieve-3028bis gets updated to only describe UTF-8
> sequences with no escaping (*). The draft can describe existing
> situation with non-UTF-8 sequences, or it can remain completely silent
> on the issue. This wouldn't affect existing implementations, as long as
> non-UTF-8 sequences in scripts are not explicitly prohibited. The draft
> will also explicitly state that future revisions of this document would
> not allow for non-UTF-8 sequences.

> =====
> So basically 2) would state the desire to deprecate use of unescaped
> non-UTF-8 sequences in scripts, while 1) would allow for them indefinitely.

> Do people see other choices?

First of all, I think we need to describe whatever escaping mechanism we decide
on, preferably in the base document. I don't think it is reasonable to suggest
abandoning a capability, let alone forbidding it, unless we have the
alternative in place.

Once we have the alternative defined I think the right approach is somewhere
between (1) and (2). (2) goes too far IMO by saying that deprecation will
happen in the future. I think the correct course is to say that the
ability to use non-UTF-8 _may_ be abandoned in the future and to plan
accordingly.

				Ned



Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8F9owjt035201; Fri, 15 Sep 2006 02:50:58 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8F9owVW035200; Fri, 15 Sep 2006 02:50:58 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from rufus.isode.com (rufus.isode.com [62.3.217.251]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8F9opBO035180 for <[e-mail]>; Fri, 15 Sep 2006 02:50:55 -0700 (MST) (envelope-from [e-mail])
Received: from [172.16.1.99] (shiny.isode.com [62.3.217.250])  by rufus.isode.com (submission channel) via TCP with ESMTPA  id <[e-mail]>; Fri, 15 Sep 2006 10:50:42 +0100
Message-ID: <[e-mail]>
Date: Fri, 15 Sep 2006 00:34:16 +0100
From: Alexey Melnikov <[e-mail]>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: MTA filtering mailing list <[e-mail]>
Subject: non-UTF-8 sequences in Sieve scripts
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

Hi everyone,

I think there is rough consensus to have an extension that would 
introduce new strings with proper escaping mechanism. The new strings 
will allow for UTF-8 (as is) + non-UTF-8 can be escaped. As nobody 
volunteered to do this, I will write a short proposal on this.

But I would like to get general feeling of the Working Group about 
non-UTF-8 sequences in existing 3028bis strings.
draft-ietf-sieve-3028bis-09.txt doesn't prohibit them. However, what 
should our long term solution be? I see two choices:

1). draft-ietf-sieve-3028bis stays as it is now, i.e. it allows for 
non-UTF-8 sequences, but recommends UTF-8. Eventually implementations 
will implement new strings with proper escaping mechanism, but existing 
strings will stay as described till the end of time.

2). draft-ietf-sieve-3028bis gets updated to only describe UTF-8 
sequences with no escaping (*). The draft can describe existing 
situation with non-UTF-8 sequences, or it can remain completely silent 
on the issue. This wouldn't affect existing implementations, as long as 
non-UTF-8 sequences in scripts are not explicitly prohibited. The draft 
will also explicitly state that future revisions of this document would 
not allow for non-UTF-8 sequences.

=====
So basically 2) would state the desire to deprecate use of unescaped 
non-UTF-8 sequences in scripts, while 1) would allow for them indefinitely.

Do people see other choices?

Please reply with your preferences till September 29th.

Thanks!
Alexey

(*) - "MUST accept UTF-8 sequences, MAY accept non-UTF-8 sequences"




Received: from balder-227.proper.com (localhost [127.0.0.1]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8EK2UmX028853; Thu, 14 Sep 2006 13:02:30 -0700 (MST) (envelope-from [e-mail])
Received: (from majordom@localhost) by balder-227.proper.com (8.13.5/8.13.5/Submit) id k8EK2Uvu028852; Thu, 14 Sep 2006 13:02:30 -0700 (MST) (envelope-from [e-mail])
X-Authentication-Warning: balder-227.proper.com: majordom set sender to [e-mail] using -f
Received: from rufus.isode.com (rufus.isode.com [62.3.217.251]) by balder-227.proper.com (8.13.5/8.13.5) with ESMTP id k8EK2SJg028844 for <[e-mail]>; Thu, 14 Sep 2006 13:02:29 -0700 (MST) (envelope-from [e-mail])
Received: from [172.16.1.99] (shiny.isode.com [62.3.217.250])  by rufus.isode.com (submission channel) via TCP with ESMTPA  id <[e-mail]>; Thu, 14 Sep 2006 21:02:26 +0100
Message-ID: <[e-mail]>
Date: Thu, 14 Sep 2006 21:02:21 +0100
From: Alexey Melnikov <[e-mail]>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: [e-mail]
Subject: Working Group Last Call on draft-ietf-sieve-3028bis-09.txt
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Sender: [e-mail]
Precedence: bulk
List-Archive: <http://www.imc.org/ietf-mta-filters/mail-archive/>
List-ID: <ietf-mta-filters.imc.org>
List-Unsubscribe: <mailto:[e-mail]?body=unsubscribe>

Hi everybody,
It has been a year since Working Group Last Call on 
draft-ietf-sieve-3028bis. There was a significant number of changes to 
the document since then, so Cyrus and I decided that the current version 
(draft-ietf-sieve-3028bis-09.txt) has to be last called again.

A two week working group last call of this document starts tomorrow on 
September 15th 2006 and ends on September 29th 2006 at 6 pm EST.

Please review this document and send issues to the list or directly to 
the editors (CCing chairs).

NB If you do review this document, but have no issues or comments, 
please send both co-chairs (or the list) an email to indicate you have 
looked at it. This will allow the chairs to measure the scope of reviews 
of this document to aid in our determination on whether to send it to 
the IESG. Thanks.

--
Alexey Melnikov
On behalf of SIEVE WG Chairs