regarding your comments on proposed media type text/troff' to Informational RFC

[moore at cs.utk.edu (Keith Moore)]

> On Thu April 14 2005 22:49, Keith Moore wrote:
> > [cc'ed to ietf-types list, which per RFC 2048 is where this document
> > should be reviewed.]
> 
> Pre-draft discussions and earlier drafts have been discussed on
> ietf-types; 

And you dismissed the objections that other people raised about this
very issue on that list.

> > I raise this concern both because this was something that was
> > discussed  extensively in the ietf-822 working group that created
> > MIME, and also  because we've seen far too many security holes
> > result from giving the  sender a way to choose what program to run
> > on the recipient's system 
> 
> The "process" parameter is explicitly *NOT* "giving the sender a way
> to choose what program to run on the recipient's system".  It is a way
> for the sender to suggest a recommended processing sequence to the
> human recipient.  N.B. "suggest", "recommended", and "human".

It's a distinction without a practical difference.


> > RFC 2046 (section 4.5.1) even includes the following language, which
> > 
> > also appeared in RFCs 1341 and 1521:
> > 
> > >    To reduce the danger of transmitting rogue programs, it is
> > >    strongly recommended that implementations NOT implement a
> > >    path-search mechanism whereby an arbitrary program named in the
> > >    Content-Type parameter (e.g., an "interpreter=" parameter) is
> > >    found and executed using the message body as input.
> 
> I fully agree that implementations should take suitable precautions. I
> would (and have in the draft) go farther and state that
> implementations (w.r.t. text/troff) should not automatically process
> content other than possibly stripping directives for presentation. 
> And there are some specific additional recommendations for safe
> presentation whether or not directives are stripped (e.g. precautions
> against control characters). 

There have been similar recommendations in every version of the MIME
spec. Guess what?  They were consistently ignored by major vendors. 
Running code and hundreds of billions of dollars worth of damage due to
viruses  aptly demonstrates that even though the spec says "don't
present executable content" implementations will do it anyway.  The
process parameter is executable content.

> > and to allow the sender to  
> > specify arbitrary preprocessing programs and arbitrary parameters to
> > those programs.
> 
> The intent is to specify the preprocessors and parameters which will
> produce the output that the author intended, not arbitrary progams/
> parameters.  

If the author wants to enclose a Makefile or build script as a separate
text/plain attachment, he's free to do so.  

> The nature of the text media type is that subtypes of that type are
> (supposed to be) presentable to a user w/o processing of any type (as
> discussed in some detail in the draft).  As with some other "rich"
> text subtypes, the proposed text/troff may also be processed for
> improved presentation (e.g. to produce hard copy).  The "process"
> parameter merely provides hints for the recipient to perform such
> processing *IF* the recipient decides to do so.

It's a lousy way to do that.  Parameters are typically not displayed to
the recipient.  They're either interpreted my the MUA or ignored.

> > Nor am I persuaded by the "sender must specify the order of 
> > preprocessing" argument, as groff has options to specify which of 
> > several preprocessors to use, and it seems to work fine.
> 
> At the risk of being repetitive, no, groff's limited facility is
> inadequate:
> 
> o groff is not installed everywhere
> 
> o there are different versions of groff; some have no support for the
>   grap preprocessor
> 
> o no version of groff up to the present one has support for either
>   chem or dformat preprocessors, or for preprocessors such as the 
>   one for handling ABNF
> 
> o groff uses fixed order of preprocessing, with eqn last.  That fails
>   miserably for documents with equations in graphs (CSTR 114) or with
>   equations in tables (CSTR 49) or both
> 
> o groff's fixed order does not accommodate tables in diagrams, which
>   is certainly possible.

Let me get this straight.  First you argue that the process parameter
isn't intended to be executed, and then you argue that a different kind
of parameter that only specified the preprocessors to be used would 
not be appropriate because _it_ couldn't be executed?

And then you're arguing that the purpose of the process parameter
isn't to specify arbitrary code to be executed, but you also admit
that there's no clearly defined set of preprocessors for troff?
If there's no clearly defined set, it's arbitrary.

> > The Version 7  
> > UNIX versions of these tools (circa 1979) described in the "CSTR 49"
> > document you cite had several limitations (many related to having to
> > run in 64k of code space on a pdp 11/70), that don't apply to modern
> > troff implementations.
> 
> How is that relevant?  The CSTR documents have been updated.  

Some of the problems with ordering of preprocessors had to do with some
preprocessors running out of memory if earlier preprocessors generated
lots of output.

> Moreover,
> the primary consideration is the order which is necessary to produce
> proper formatted output; the order that works for tables in diagrams
> is different from the order that works for diagrams in tables, in a
> mutually exclusive relationship (i.e. there is no order that will
> cover both cases).

So design a parameter that specifies ordering without specifying the
command-line.

The reality is that in practice troff isn't a single data format, it's 
a lot of fairlly arbitrary variations on a format.  For this reason, 
it doesn't lend itself to easy MIME typing.  So specifying a sane 
MIME type for troff is a harder problem than it initially seems, 
and you're running into that.

Keith
>From [e-mail]  Fri Apr 15 16:36:54 2005
From: moore at cs.utk.edu (Keith Moore)
Date: Fri Apr 15 16:37:59 2005
Subject: regarding your comments on proposed media type text/troff' to
 Informational RFC
In-Reply-To: <[e-mail]>
References: <[e-mail]>
	<[e-mail]>
	<[e-mail]>
	<[e-mail]>
Message-ID: <[e-mail]>

> On Thu April 14 2005 22:49, Keith Moore wrote:
> > [cc'ed to ietf-types list, which per RFC 2048 is where this document
> > should be reviewed.]
> 
> Pre-draft discussions and earlier drafts have been discussed on
> ietf-types; 

And you dismissed the objections that other people raised about this
very issue on that list.

> > I raise this concern both because this was something that was
> > discussed  extensively in the ietf-822 working group that created
> > MIME, and also  because we've seen far too many security holes
> > result from giving the  sender a way to choose what program to run
> > on the recipient's system 
> 
> The "process" parameter is explicitly *NOT* "giving the sender a way
> to choose what program to run on the recipient's system".  It is a way
> for the sender to suggest a recommended processing sequence to the
> human recipient.  N.B. "suggest", "recommended", and "human".

It's a distinction without a practical difference.


> > RFC 2046 (section 4.5.1) even includes the following language, which
> > 
> > also appeared in RFCs 1341 and 1521:
> > 
> > >    To reduce the danger of transmitting rogue programs, it is
> > >    strongly recommended that implementations NOT implement a
> > >    path-search mechanism whereby an arbitrary program named in the
> > >    Content-Type parameter (e.g., an "interpreter=" parameter) is
> > >    found and executed using the message body as input.
> 
> I fully agree that implementations should take suitable precautions. I
> would (and have in the draft) go farther and state that
> implementations (w.r.t. text/troff) should not automatically process
> content other than possibly stripping directives for presentation. 
> And there are some specific additional recommendations for safe
> presentation whether or not directives are stripped (e.g. precautions
> against control characters). 

There have been similar recommendations in every version of the MIME
spec. Guess what?  They were consistently ignored by major vendors. 
Running code and hundreds of billions of dollars worth of damage due to
viruses  aptly demonstrates that even though the spec says "don't
present executable content" implementations will do it anyway.  The
process parameter is executable content.

> > and to allow the sender to  
> > specify arbitrary preprocessing programs and arbitrary parameters to
> > those programs.
> 
> The intent is to specify the preprocessors and parameters which will
> produce the output that the author intended, not arbitrary progams/
> parameters.  

If the author wants to enclose a Makefile or build script as a separate
text/plain attachment, he's free to do so.  

> The nature of the text media type is that subtypes of that type are
> (supposed to be) presentable to a user w/o processing of any type (as
> discussed in some detail in the draft).  As with some other "rich"
> text subtypes, the proposed text/troff may also be processed for
> improved presentation (e.g. to produce hard copy).  The "process"
> parameter merely provides hints for the recipient to perform such
> processing *IF* the recipient decides to do so.

It's a lousy way to do that.  Parameters are typically not displayed to
the recipient.  They're either interpreted my the MUA or ignored.

> > Nor am I persuaded by the "sender must specify the order of 
> > preprocessing" argument, as groff has options to specify which of 
> > several preprocessors to use, and it seems to work fine.
> 
> At the risk of being repetitive, no, groff's limited facility is
> inadequate:
> 
> o groff is not installed everywhere
> 
> o there are different versions of groff; some have no support for the
>   grap preprocessor
> 
> o no version of groff up to the present one has support for either
>   chem or dformat preprocessors, or for preprocessors such as the 
>   one for handling ABNF
> 
> o groff uses fixed order of preprocessing, with eqn last.  That fails
>   miserably for documents with equations in graphs (CSTR 114) or with
>   equations in tables (CSTR 49) or both
> 
> o groff's fixed order does not accommodate tables in diagrams, which
>   is certainly possible.

Let me get this straight.  First you argue that the process parameter
isn't intended to be executed, and then you argue that a different kind
of parameter that only specified the preprocessors to be used would 
not be appropriate because _it_ couldn't be executed?

And then you're arguing that the purpose of the process parameter
isn't to specify arbitrary code to be executed, but you also admit
that there's no clearly defined set of preprocessors for troff?
If there's no clearly defined set, it's arbitrary.

> > The Version 7  
> > UNIX versions of these tools (circa 1979) described in the "CSTR 49"
> > document you cite had several limitations (many related to having to
> > run in 64k of code space on a pdp 11/70), that don't apply to modern
> > troff implementations.
> 
> How is that relevant?  The CSTR documents have been updated.  

Some of the problems with ordering of preprocessors had to do with some
preprocessors running out of memory if earlier preprocessors generated
lots of output.

> Moreover,
> the primary consideration is the order which is necessary to produce
> proper formatted output; the order that works for tables in diagrams
> is different from the order that works for diagrams in tables, in a
> mutually exclusive relationship (i.e. there is no order that will
> cover both cases).

So design a parameter that specifies ordering without specifying the
command-line.

The reality is that in practice troff isn't a single data format, it's 
a lot of fairlly arbitrary variations on a format.  For this reason, 
it doesn't lend itself to easy MIME typing.  So specifying a sane 
MIME type for troff is a harder problem than it initially seems, 
and you're running into that.

Keith
>From [e-mail]  Sat Apr 16 17:12:35 2005
From: blilly at erols.com (Bruce Lilly)
Date: Sat Apr 16 17:12:52 2005
Subject: regarding your comments on proposed media type text/troff' to
	Informational RFC
In-Reply-To: <[e-mail]>
References: <[e-mail]>
	<[e-mail]>
	<[e-mail]>
Message-ID: <[e-mail]>

On Fri April 15 2005 10:36, Keith Moore wrote:

> And you dismissed the objections that other people raised about this
> very issue on that list.

No, wording of the draft was changed as a result.

> > The "process" parameter is explicitly *NOT* "giving the sender a way
> > to choose what program to run on the recipient's system".  It is a way
> > for the sender to suggest a recommended processing sequence to the
> > human recipient.  N.B. "suggest", "recommended", and "human".
> 
> It's a distinction without a practical difference.

No, if some wayward implementer were to automatically execute programs
contrary to the specification, said implementer can be informed that he
is out of line.  That is a very real "practical difference" between the
actual draft and your mischaracterization of what the draft says.  You
seem to:
a) want to remove any standardized mechanism for the author to convey
   relevant information to the recipient
b) believe that groff can magically determine preprocessors, an
   appropriate ordering of preprocessors with appropriate arguments,
   etc.
Removal of the "process" parameter will not prevent a wayward implementer
from simply automatically invoking groff; removal of the parameter
therefore will not address inappropriate use of the media type and
metadata, and removal of discussion of inappropriate use of said parameter
along with its specification would leave no stick with which to educate
such implementers.  It will, however, impair legitimate uses of the media
type and metadata.

> There have been similar recommendations in every version of the MIME
> spec. Guess what?  They were consistently ignored by major vendors. 
> Running code and hundreds of billions of dollars worth of damage due to
> viruses  aptly demonstrates that even though the spec says "don't
> present executable content" implementations will do it anyway.

Don't blame me for inappropriate behavior of some unrelated vendor. If
multiple recommendations haven't prevented inappropriate implementations,
what leads you to the conclusion that failing to make recommendations
will improve matters?

> The 
> process parameter is executable content.

No, it's a human-writable, human-readable hint in a well-understood,
precise, concise, standardized (e.g. POSIX) format.
 
> > The intent is to specify the preprocessors and parameters which will
> > produce the output that the author intended, not arbitrary progams/
> > parameters.  
> 
> If the author wants to enclose a Makefile or build script as a separate
> text/plain attachment, he's free to do so.  

You seem to be fixated on a niche application of media types, which are
being separated from the mail environment.  In general, there is nothing
in which to "enclose" separate content, or to which to make an
"attachment".  Consider, as an example of possible use outside of a mail
environment, a means of recording such metadata in some sort of database
with entries for files (e.g. similar to a MacOS "resource fork", or the
comments field which can be set for files in recent versions of MS
Windows).

Moreover, a "build script" would fit into the category of "languages for
'active' (computational) material", which is the application media type,
not text, and certainly not text/plain.  See
draft-freed-media-type-reg-04.txt.
 
> It's a lousy way to do that.  Parameters are typically not displayed to
> the recipient.  They're either interpreted my the MUA or ignored.

Again, you're fixated on a mail environment, which is not the only
application for media types.
 
> Let me get this straight.

OK, I'll try to clarify the situation.

> First you argue that the process parameter 
> isn't intended to be executed,

Not intended to be automatically executed.  If the human recipient,
after exercising due caution, wishes to follow the suggestion supplied
by the content author, he is of course free to do so -- or he can
execute some other, possibly similar set of processes, or he can choose
not to format the content.

> and then you argue that a different kind 
> of parameter that only specified the preprocessors to be used would 
> not be appropriate because _it_ couldn't be executed?

No, the objections to a mere unordered list of preprocessors includes:

1. preprocessor order is critically important
2. preprocessor arguments are critically important
3. environment variables (e.g. GROFF_NO_SGR in recent groff versions)
   may be critically important
4. formatter arguments are critically important
5. postprocessing may be required

Your suggestion addresses none of those issues, all of which are
accommodated by command-line syntax.  Moreover:

I. command-line syntax is easy for an author to indicate, with precision
   and conciseness
II. command-line syntax can be read, understood, modified if desired,
   and used by recipients.

A mere list lacks those characteristics.  Obfuscating processing steps
by some sort of mechanical translation of easily-generated, easily-read
syntax adds extra manual, error-prone steps for both author and recipient,
but will not slow down a wayward implementer who is determined to
automatically pass content to a program; he'll simply automate the
translation.  That would have precisely the opposite effect of the
current wording of the draft; it would make legitimate use difficult
while making automated misuse straightforward.

> And then you're arguing that the purpose of the process parameter
> isn't to specify arbitrary code to be executed,

Correct.

> but you also admit 
> that there's no clearly defined set of preprocessors for troff?

No, at any given time, one can enumerate a set of preprocessors, as has
been done in the registration template in the draft under discussion --
but that set may change over time as new preprocessors are developed.
One can clearly define a set of preprocessors as programs with the
following characteristics:

i. the program operates as a filter, accepting input and producing
   output (as opposed to programs which do not do both, such as "ls",
   etc.)
ii. The input and output formats conform to troff format conventions as
   specified in CSTR 54, e.g. directives intermixed with text, directives
   begin with dot, etc.
iii. input is human-readable content describing some abstract format
   which is translated into formatting directives or to another human-
   readable format for further processing with the ultimate goal being
   production of formatted output
iv. the input to be preprocessed is delimited by specified directives,
   and other input is passed to the output unaltered (so the programs
   in the graphviz suite, which can generate pic output, do not
   qualify as preprocessors because they do not accept/pass other
   input).

> If there's no clearly defined set, it's arbitrary.

There is a clearly defined set; see above for a definition.
 
> > > The Version 7  
> > > UNIX versions of these tools (circa 1979) described in the "CSTR 49"
> > > document you cite had several limitations (many related to having to
> > > run in 64k of code space on a pdp 11/70), that don't apply to modern
> > > troff implementations.
> > 
> > How is that relevant?  The CSTR documents have been updated.  
> 
> Some of the problems with ordering of preprocessors had to do with some
> preprocessors running out of memory if earlier preprocessors generated
> lots of output.

While that might be relevant to a recipient with a particular legacy
implementation, it is not relevant for the purpose of describing the
preprocessors, their order, their arguments, and environment settings
necessary for successful formatting of particular content.
 
> So design a parameter that specifies ordering without specifying the
> command-line.

See above reasons why that would be harmful to the intended use for
end-to-end, human-to-human communications intended for the parameter,
and why it would not deter determined but wayward implementers.
 
> The reality is that in practice troff isn't a single data format, it's 
> a lot of fairlly arbitrary variations on a format.

Not arbitrary.

> For this reason,  
> it doesn't lend itself to easy MIME typing.

As mentioned in the draft, some specific instances of content might be
more appropriately categorized under the image or application media
types.  The draft explicitly does not address those cases.

> So specifying a sane  
> MIME type for troff is a harder problem than it initially seems, 
> and you're running into that.

No, there's no particular difficulty in specifying the format for
text use.  Your complaints seem to boil down to implementers who
ignore existing recommendations which are reinforced in the draft
under discussion.  Nothing that can be said in this particular draft
is likely to change that for this media type or any other -- idiotic
implementers are what they are.  Making media types more difficult for
legitimate, conscientious users to use on the groundless assumption
that that will cause determined wayward implementers to see the light
smacks of a "nanny" or "Big Brother is Watching You" approach.
>From [e-mail]  Sat Apr 16 17:12:35 2005
From: blilly at erols.com (Bruce Lilly)
Date: Sat Apr 16 17:12:54 2005
Subject: regarding your comments on proposed media type text/troff' to
	Informational RFC
In-Reply-To: <[e-mail]>
References: <[e-mail]>
	<[e-mail]>
	<[e-mail]>
Message-ID: <[e-mail]>

On Fri April 15 2005 10:36, Keith Moore wrote:

> And you dismissed the objections that other people raised about this
> very issue on that list.

No, wording of the draft was changed as a result.

> > The "process" parameter is explicitly *NOT* "giving the sender a way
> > to choose what program to run on the recipient's system".  It is a way
> > for the sender to suggest a recommended processing sequence to the
> > human recipient.  N.B. "suggest", "recommended", and "human".
> 
> It's a distinction without a practical difference.

No, if some wayward implementer were to automatically execute programs
contrary to the specification, said implementer can be informed that he
is out of line.  That is a very real "practical difference" between the
actual draft and your mischaracterization of what the draft says.  You
seem to:
a) want to remove any standardized mechanism for the author to convey
   relevant information to the recipient
b) believe that groff can magically determine preprocessors, an
   appropriate ordering of preprocessors with appropriate arguments,
   etc.
Removal of the "process" parameter will not prevent a wayward implementer
from simply automatically invoking groff; removal of the parameter
therefore will not address inappropriate use of the media type and
metadata, and removal of discussion of inappropriate use of said parameter
along with its specification would leave no stick with which to educate
such implementers.  It will, however, impair legitimate uses of the media
type and metadata.

> There have been similar recommendations in every version of the MIME
> spec. Guess what?  They were consistently ignored by major vendors. 
> Running code and hundreds of billions of dollars worth of damage due to
> viruses  aptly demonstrates that even though the spec says "don't
> present executable content" implementations will do it anyway.

Don't blame me for inappropriate behavior of some unrelated vendor. If
multiple recommendations haven't prevented inappropriate implementations,
what leads you to the conclusion that failing to make recommendations
will improve matters?

> The 
> process parameter is executable content.

No, it's a human-writable, human-readable hint in a well-understood,
precise, concise, standardized (e.g. POSIX) format.
 
> > The intent is to specify the preprocessors and parameters which will
> > produce the output that the author intended, not arbitrary progams/
> > parameters.  
> 
> If the author wants to enclose a Makefile or build script as a separate
> text/plain attachment, he's free to do so.  

You seem to be fixated on a niche application of media types, which are
being separated from the mail environment.  In general, there is nothing
in which to "enclose" separate content, or to which to make an
"attachment".  Consider, as an example of possible use outside of a mail
environment, a means of recording such metadata in some sort of database
with entries for files (e.g. similar to a MacOS "resource fork", or the
comments field which can be set for files in recent versions of MS
Windows).

Moreover, a "build script" would fit into the category of "languages for
'active' (computational) material", which is the application media type,
not text, and certainly not text/plain.  See
draft-freed-media-type-reg-04.txt.
 
> It's a lousy way to do that.  Parameters are typically not displayed to
> the recipient.  They're either interpreted my the MUA or ignored.

Again, you're fixated on a mail environment, which is not the only
application for media types.
 
> Let me get this straight.

OK, I'll try to clarify the situation.

> First you argue that the process parameter 
> isn't intended to be executed,

Not intended to be automatically executed.  If the human recipient,
after exercising due caution, wishes to follow the suggestion supplied
by the content author, he is of course free to do so -- or he can
execute some other, possibly similar set of processes, or he can choose
not to format the content.

> and then you argue that a different kind 
> of parameter that only specified the preprocessors to be used would 
> not be appropriate because _it_ couldn't be executed?

No, the objections to a mere unordered list of preprocessors includes:

1. preprocessor order is critically important
2. preprocessor arguments are critically important
3. environment variables (e.g. GROFF_NO_SGR in recent groff versions)
   may be critically important
4. formatter arguments are critically important
5. postprocessing may be required

Your suggestion addresses none of those issues, all of which are
accommodated by command-line syntax.  Moreover:

I. command-line syntax is easy for an author to indicate, with precision
   and conciseness
II. command-line syntax can be read, understood, modified if desired,
   and used by recipients.

A mere list lacks those characteristics.  Obfuscating processing steps
by some sort of mechanical translation of easily-generated, easily-read
syntax adds extra manual, error-prone steps for both author and recipient,
but will not slow down a wayward implementer who is determined to
automatically pass content to a program; he'll simply automate the
translation.  That would have precisely the opposite effect of the
current wording of the draft; it would make legitimate use difficult
while making automated misuse straightforward.

> And then you're arguing that the purpose of the process parameter
> isn't to specify arbitrary code to be executed,

Correct.

> but you also admit 
> that there's no clearly defined set of preprocessors for troff?

No, at any given time, one can enumerate a set of preprocessors, as has
been done in the registration template in the draft under discussion --
but that set may change over time as new preprocessors are developed.
One can clearly define a set of preprocessors as programs with the
following characteristics:

i. the program operates as a filter, accepting input and producing
   output (as opposed to programs which do not do both, such as "ls",
   etc.)
ii. The input and output formats conform to troff format conventions as
   specified in CSTR 54, e.g. directives intermixed with text, directives
   begin with dot, etc.
iii. input is human-readable content describing some abstract format
   which is translated into formatting directives or to another human-
   readable format for further processing with the ultimate goal being
   production of formatted output
iv. the input to be preprocessed is delimited by specified directives,
   and other input is passed to the output unaltered (so the programs
   in the graphviz suite, which can generate pic output, do not
   qualify as preprocessors because they do not accept/pass other
   input).

> If there's no clearly defined set, it's arbitrary.

There is a clearly defined set; see above for a definition.
 
> > > The Version 7  
> > > UNIX versions of these tools (circa 1979) described in the "CSTR 49"
> > > document you cite had several limitations (many related to having to
> > > run in 64k of code space on a pdp 11/70), that don't apply to modern
> > > troff implementations.
> > 
> > How is that relevant?  The CSTR documents have been updated.  
> 
> Some of the problems with ordering of preprocessors had to do with some
> preprocessors running out of memory if earlier preprocessors generated
> lots of output.

While that might be relevant to a recipient with a particular legacy
implementation, it is not relevant for the purpose of describing the
preprocessors, their order, their arguments, and environment settings
necessary for successful formatting of particular content.
 
> So design a parameter that specifies ordering without specifying the
> command-line.

See above reasons why that would be harmful to the intended use for
end-to-end, human-to-human communications intended for the parameter,
and why it would not deter determined but wayward implementers.
 
> The reality is that in practice troff isn't a single data format, it's 
> a lot of fairlly arbitrary variations on a format.

Not arbitrary.

> For this reason,  
> it doesn't lend itself to easy MIME typing.

As mentioned in the draft, some specific instances of content might be
more appropriately categorized under the image or application media
types.  The draft explicitly does not address those cases.

> So specifying a sane  
> MIME type for troff is a harder problem than it initially seems, 
> and you're running into that.

No, there's no particular difficulty in specifying the format for
text use.  Your complaints seem to boil down to implementers who
ignore existing recommendations which are reinforced in the draft
under discussion.  Nothing that can be said in this particular draft
is likely to change that for this media type or any other -- idiotic
implementers are what they are.  Making media types more difficult for
legitimate, conscientious users to use on the groundless assumption
that that will cause determined wayward implementers to see the light
smacks of a "nanny" or "Big Brother is Watching You" approach.
>From [e-mail]  Sun Apr 17 00:46:16 2005
From: moore at cs.utk.edu (Keith Moore)
Date: Sun Apr 17 00:46:45 2005
Subject: regarding your comments on proposed media type text/troff' to
	Informational RFC
In-Reply-To: <[e-mail]>
References: <[e-mail]>
	<[e-mail]>
	<[e-mail]>
	<[e-mail]>
Message-ID: <[e-mail]>

Bruce,

bottom line - this parameter violates both the intent and the wording 
of the MIME spec.  it's simply unacceptable.
solve the problem in a different way.

Keith


On Apr 16, 2005, at 11:12 AM, Bruce Lilly wrote:

> On Fri April 15 2005 10:36, Keith Moore wrote:
>
>> And you dismissed the objections that other people raised about this
>> very issue on that list.
>
> No, wording of the draft was changed as a result.
>
>>> The "process" parameter is explicitly *NOT* "giving the sender a way
>>> to choose what program to run on the recipient's system".  It is a 
>>> way
>>> for the sender to suggest a recommended processing sequence to the
>>> human recipient.  N.B. "suggest", "recommended", and "human".
>>
>> It's a distinction without a practical difference.
>
> No, if some wayward implementer were to automatically execute programs
> contrary to the specification, said implementer can be informed that he
> is out of line.  That is a very real "practical difference" between the
> actual draft and your mischaracterization of what the draft says.  You
> seem to:
> a) want to remove any standardized mechanism for the author to convey
>    relevant information to the recipient
> b) believe that groff can magically determine preprocessors, an
>    appropriate ordering of preprocessors with appropriate arguments,
>    etc.
> Removal of the "process" parameter will not prevent a wayward 
> implementer
> from simply automatically invoking groff; removal of the parameter
> therefore will not address inappropriate use of the media type and
> metadata, and removal of discussion of inappropriate use of said 
> parameter
> along with its specification would leave no stick with which to educate
> such implementers.  It will, however, impair legitimate uses of the 
> media
> type and metadata.
>
>> There have been similar recommendations in every version of the MIME
>> spec. Guess what?  They were consistently ignored by major vendors.
>> Running code and hundreds of billions of dollars worth of damage due 
>> to
>> viruses  aptly demonstrates that even though the spec says "don't
>> present executable content" implementations will do it anyway.
>
> Don't blame me for inappropriate behavior of some unrelated vendor. If
> multiple recommendations haven't prevented inappropriate 
> implementations,
> what leads you to the conclusion that failing to make recommendations
> will improve matters?
>
>> The
>> process parameter is executable content.
>
> No, it's a human-writable, human-readable hint in a well-understood,
> precise, concise, standardized (e.g. POSIX) format.
>
>>> The intent is to specify the preprocessors and parameters which will
>>> produce the output that the author intended, not arbitrary progams/
>>> parameters.
>>
>> If the author wants to enclose a Makefile or build script as a 
>> separate
>> text/plain attachment, he's free to do so.
>
> You seem to be fixated on a niche application of media types, which are
> being separated from the mail environment.  In general, there is 
> nothing
> in which to "enclose" separate content, or to which to make an
> "attachment".  Consider, as an example of possible use outside of a 
> mail
> environment, a means of recording such metadata in some sort of 
> database
> with entries for files (e.g. similar to a MacOS "resource fork", or the
> comments field which can be set for files in recent versions of MS
> Windows).
>
> Moreover, a "build script" would fit into the category of "languages 
> for
> 'active' (computational) material", which is the application media 
> type,
> not text, and certainly not text/plain.  See
> draft-freed-media-type-reg-04.txt.
>
>> It's a lousy way to do that.  Parameters are typically not displayed 
>> to
>> the recipient.  They're either interpreted my the MUA or ignored.
>
> Again, you're fixated on a mail environment, which is not the only
> application for media types.
>
>> Let me get this straight.
>
> OK, I'll try to clarify the situation.
>
>> First you argue that the process parameter
>> isn't intended to be executed,
>
> Not intended to be automatically executed.  If the human recipient,
> after exercising due caution, wishes to follow the suggestion supplied
> by the content author, he is of course free to do so -- or he can
> execute some other, possibly similar set of processes, or he can choose
> not to format the content.
>
>> and then you argue that a different kind
>> of parameter that only specified the preprocessors to be used would
>> not be appropriate because _it_ couldn't be executed?
>
> No, the objections to a mere unordered list of preprocessors includes:
>
> 1. preprocessor order is critically important
> 2. preprocessor arguments are critically important
> 3. environment variables (e.g. GROFF_NO_SGR in recent groff versions)
>    may be critically important
> 4. formatter arguments are critically important
> 5. postprocessing may be required
>
> Your suggestion addresses none of those issues, all of which are
> accommodated by command-line syntax.  Moreover:
>
> I. command-line syntax is easy for an author to indicate, with 
> precision
>    and conciseness
> II. command-line syntax can be read, understood, modified if desired,
>    and used by recipients.
>
> A mere list lacks those characteristics.  Obfuscating processing steps
> by some sort of mechanical translation of easily-generated, easily-read
> syntax adds extra manual, error-prone steps for both author and 
> recipient,
> but will not slow down a wayward implementer who is determined to
> automatically pass content to a program; he'll simply automate the
> translation.  That would have precisely the opposite effect of the
> current wording of the draft; it would make legitimate use difficult
> while making automated misuse straightforward.
>
>> And then you're arguing that the purpose of the process parameter
>> isn't to specify arbitrary code to be executed,
>
> Correct.
>
>> but you also admit
>> that there's no clearly defined set of preprocessors for troff?
>
> No, at any given time, one can enumerate a set of preprocessors, as has
> been done in the registration template in the draft under discussion --
> but that set may change over time as new preprocessors are developed.
> One can clearly define a set of preprocessors as programs with the
> following characteristics:
>
> i. the program operates as a filter, accepting input and producing
>    output (as opposed to programs which do not do both, such as "ls",
>    etc.)
> ii. The input and output formats conform to troff format conventions as
>    specified in CSTR 54, e.g. directives intermixed with text, 
> directives
>    begin with dot, etc.
> iii. input is human-readable content describing some abstract format
>    which is translated into formatting directives or to another human-
>    readable format for further processing with the ultimate goal being
>    production of formatted output
> iv. the input to be preprocessed is delimited by specified directives,
>    and other input is passed to the output unaltered (so the programs
>    in the graphviz suite, which can generate pic output, do not
>    qualify as preprocessors because they do not accept/pass other
>    input).
>
>> If there's no clearly defined set, it's arbitrary.
>
> There is a clearly defined set; see above for a definition.
>
>>>> The Version 7
>>>> UNIX versions of these tools (circa 1979) described in the "CSTR 49"
>>>> document you cite had several limitations (many related to having to
>>>> run in 64k of code space on a pdp 11/70), that don't apply to modern
>>>> troff implementations.
>>>
>>> How is that relevant?  The CSTR documents have been updated.
>>
>> Some of the problems with ordering of preprocessors had to do with 
>> some
>> preprocessors running out of memory if earlier preprocessors generated
>> lots of output.
>
> While that might be relevant to a recipient with a particular legacy
> implementation, it is not relevant for the purpose of describing the
> preprocessors, their order, their arguments, and environment settings
> necessary for successful formatting of particular content.
>
>> So design a parameter that specifies ordering without specifying the
>> command-line.
>
> See above reasons why that would be harmful to the intended use for
> end-to-end, human-to-human communications intended for the parameter,
> and why it would not deter determined but wayward implementers.
>
>> The reality is that in practice troff isn't a single data format, it's
>> a lot of fairlly arbitrary variations on a format.
>
> Not arbitrary.
>
>> For this reason,
>> it doesn't lend itself to easy MIME typing.
>
> As mentioned in the draft, some specific instances of content might be
> more appropriately categorized under the image or application media
> types.  The draft explicitly does not address those cases.
>
>> So specifying a sane
>> MIME type for troff is a harder problem than it initially seems,
>> and you're running into that.
>
> No, there's no particular difficulty in specifying the format for
> text use.  Your complaints seem to boil down to implementers who
> ignore existing recommendations which are reinforced in the draft
> under discussion.  Nothing that can be said in this particular draft
> is likely to change that for this media type or any other -- idiotic
> implementers are what they are.  Making media types more difficult for
> legitimate, conscientious users to use on the groundless assumption
> that that will cause determined wayward implementers to see the light
> smacks of a "nanny" or "Big Brother is Watching You" approach.