Re: [websec] Issue 17: Registry for magic numbers

[Adam Barth <[e-mail]>]

You've posed a large number of questions.  I'll do my best to answer them.

On Tue, Oct 25, 2011 at 11:31 PM, Larry Masinter <[e-mail]> wrote:
> This gets back to the question of the scope of the document. Does it, or does it not, handle sniffing of arbitrary blobs of data that come in without any content-type,

User agents that implement sniffing are expected to sniff HTTP
responses that lack a Content-Type header, yes.

> blobs of data labeled application/octet-stream,

User agents that implement sniffing are not expected to sniff HTTP
responses that contain Content-Type header with the value
application/octet-stream.

> and data coming via ftp (through ftp URIs)

Yes.

> or thumb drives

Yes.

>or mounted NFS file systems or whatever?

Yes.

You can answer these questions by reading the document.  For example,
the document explicitly states the set of Content-Type header values
that trigger sniffing.  The document also explicitly calls out FTP as
an example.

> Does it, or does it not, handle sniffing rules inside ZIP packaged web applications?

Not as described by this document.  However, I've been told that
another document has re-used the algorithm for that purpose.

> If it does, then sniffing should cover everything that is sniffable, including almost all MIME types

Why is that?  The document describes what is essentially the minimal
amount of sniffing a user agent needs to perform in order to be
competitive in the browser market.  I don't think we should be
encouraging sniffing beyond that.

> -- you say "most MIME types that get registered don't need sniffing rules", I don't know what the percentage is,

With the possible exception of fonts, I believe the document describes
all the sniffing rules that are necessary today.  You can compare with
list of MIME types in the document with the list of registered MIME
types if you wish to get a sense of what I mean when I say that "most"
don't need sniffing rules.

> but after all, don't you want to be able to discover file types??

I'm not sure what you mean by "discover file types".  There's no
discovery going on here.

> Of course, maybe that broad applicability of sniffing isn't appropriate, but then ... where's are the boundaries?

The boundaries are exactly what's described in the document.  There's
been a great deal of research and implementation experience poured
into the document to determine precisely where to draw the boundaries.
 As far as I can tell, the document describes the optimal point.  If
you have data that shows otherwise, I'd like to see it.

> Which situations are in scope vs. not?

The criteria I would use is the following one:

"Given a diverse market of browser vendors, is this a sniffing
algorithm that all browser vendors are mutually interested in
converging upon."

If the answer is "yes", then you've identified the correct scope and
rules.  If "no", then the spec needs to be improved.  If there is no
such set of rules, then this endeavor is a waste of time and any spec
we create will be dead letter.

> And don't some of the "in-scope" situations need almost all MIME types to be sniffable?

No.

Adam


> -----Original Message-----
> From: [e-mail] [mailto:[e-mail]] On Behalf Of Adam Barth
> Sent: Tuesday, October 25, 2011 9:00 PM
> To: Tobias Gondrom
> Cc: [e-mail]
> Subject: Re: [websec] Issue 17: Registry for magic numbers
>
> Yeah, I think we're much better off creating a new registry rather than using the MIME registry.  The truth is that most MIME types that get registered don't need sniffing rules.  The only ones that need it are the legacy ones and the ones browser vendor cause to need it because of the prisoner's dilemma in the browser market.
>
> Adam
>
>
> On Tue, Oct 25, 2011 at 8:52 PM, Tobias Gondrom <[e-mail]> wrote:
>> <hat="individual">
>> For me the point is, currently we have a table in the document, which
>> inside an RFC is rather static and hard to extend.
>> So it looks like a good case for a registry to allow for extendibility
>> for new mime-types. (e.g. we keep the table in the document, create an
>> IANA registry, copy the values to the registry and allow for future
>> entries by expert review) That can either be added to the current
>> Mime-type registry, or we create a new one (e.g. within the websec
>> namespace) with only these elements.
>>
>> Just my 5cents.
>>
>> Tobias
>>
>>
>>
>> On 25/10/11 05:23, Adam Barth wrote:
>>>
>>> On Mon, Oct 24, 2011 at 9:07 PM, "Martin J. Dürst"
>>> <[e-mail]>  wrote:
>>>>
>>>> On 2011/10/25 11:21, Adam Barth wrote:
>>>>>
>>>>> http://trac.tools.ietf.org/wg/websec/trac/ticket/17 refers to an
>>>>> IANA registry with magic numbers for various media types.  I wanted
>>>>> to compare them to what's in the draft, but I couldn't find it.  I
>>>>> found the media type registry, e.g., for images:
>>>>>
>>>>> http://www.iana.org/assignments/media-types/image/index.html
>>>>>
>>>>> but I don't see any magic numbers.  Would someone be willing to
>>>>> point me in the right direction?
>>>>
>>>> They are in the templates. To get the template for a registration,
>>>> start at the overview page
>>>> (http://www.iana.org/assignments/media-types/index.html).
>>>>
>>>> Then go to the page that lists all the registration for a give top
>>>> level, e.g.
>>>> http://www.iana.org/assignments/media-types/image/index.html for images.
>>>>
>>>> Then look at each registration template (click on the link in the
>>>> left column, or in the right column if the left one doesn't have a
>>>> link and the right one is to an RFC). You may then find a magic
>>>> number in the registration template. As an example, for image/jp2,
>>>> the template is at
>>>> http://www.iana.org/assignments/media-types/image/jp2.
>>>>
>>>> But it looks like earlier templates didn't have a field for a magic
>>>> number, and this and the reasons Anne gave make this information
>>>> helpful for cross-checking, but not much more.
>>>
>>> == Images ==
>>>
>>> PNG has a registration template
>>> <http://www.iana.org/assignments/media-types/image/png>, but lacks a
>>> signature.
>>> JPEG doesn't have a template.
>>> GIF doesn't have a template.
>>> BMP isn't even registered.
>>> WEBP isn't even registered.
>>> ICO has a registration template
>>> <http://www.iana.org/assignments/media-types/image/vnd.microsoft.icon
>>> >
>>> and has the correct signature.  Yay!
>>>
>>> == Text ==
>>>
>>> HTML lacks a registration template.
>>>
>>> == Application ==
>>>
>>> PDF doesn't have a template.
>>> Postscript doesn't have a template.
>>> OGG doesn't have a template.
>>> RAR isn't even registered.
>>> ZIP has a registration template
>>> <http://www.iana.org/assignments/media-types/application/zip>, but
>>> lacks a signature.
>>> GZIP isn't even registered.
>>> RSS isn't even registered.
>>> Atom lacks a registration template.
>>>
>>> == Audio ==
>>>
>>> WAV isn't even registered.
>>>
>>> == Video ==
>>>
>>> MP4 lacks a registration template.
>>> WebM isn't even registered.
>>>
>>> This does not look like a promising approach.  Note: I haven't even
>>> looked through all the registrations to see how many have signatures
>>> that we shouldn't be using.
>>>
>>> Adam
>>> _______________________________________________
>>> websec mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/websec
>>
>> _______________________________________________
>> websec mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/websec
>>
> _______________________________________________
> websec mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/websec
>