[Ssh] Re: SSHM charter - next steps

[Mouse <[e-mail]>]

(In passing, it would help readability if you would avoid using
paragraph-length lines.  If you want readers to reflow your text,
format=flowed would be your friend.)

>> Well, the current spec does not support handling non-UTF-8 octet
>> sequences in these places.  Environment variables, even more than
>> usernames, have always been octet sequences rather than character
>> sequences.  I count that as an encoding-related problem, though a
>> problem with the spec, not with any particular implementation.

Actually, I've reread that part of the spec.  The name and value in env
requests are "string"s, which means there is no particular encoding
requirement at that level; I haven't found any statement that the name
and/or value strings have any additional restrictions on them.  So,
unless I've missed something (hardly impossible), for environment
variables the incompatibility is nonexistent at the ssh protocol level.
If that means just pushing it off to upper levels, well, see xkcd 927.

> I think the intent of mandating an encoding is to improve interoperability i$

Presumably.  But it ends up making the spec unimplementable in some
cases, because (for example) some systems don't store usernames as
character strings, only octet strings.  You write of "whatever local
encoding that system is using" - this assumes those octet sequences
represent characters (which may not be true; see below), and,
furthermore, that the encoding is somewhere accessible to the
implementation.  This is false often enough to be problematic.  Suppose
my /etc/passwd has a username field consisting of the three octets (in
hex) c5, b2, 30.  Is that A-ring, superscript-2, zero (8859-1)?  Is it
A-ring, G-with-dot-above, zero (8859-14)?  Is it U-ogonek, zero
(UTF-8)?  Is it something else (KOI-8 maybe)?  Without consulting with
the user who chose the username, I can't tell; software has no hope of
telling.

It is plausible to claim that those octets represent characters at the
human level.  Even that, though, is debatable; octet sequences, even
sometimes octet sequences in places typically used for character
strings, do sometimes get used to store (possibly encoded) binary data,
and there exist systems where usernames are for mechanized consumption
rather than human consumption.

Furthermore, even if those octets _do_ represent characters at the
human level, how is software supposed to get at that information?  None
of my computers have mindreading hardware; would you say I need to run
shell startup for each username on the system to find out what its
locale settings are (and what if someone doesn't have any?) when my ssh
server receives a username?  Or do I have to mandate that everyone's
username uses the same encoding in order for my system to have the
privilege of using ssh (and where do I store that encoding)?  Or maybe
I need to redesign the OS's username database to add character set tags
to usernames?  And, on top of all that, this would mandate that every
ssh implementation contain all the hair that is Unicode normalization.

Eh, no.  Personally, I consider usefulness more important than
conformance to some fantasy world in which all those octet sequences
are somehow marked with character set and encoding tags.  That's why
moussh treats those as opaque octet sequences.

> I havenâ??t checked to see if OpenSSH converts to/from UTF-8 or not if you a$

I'd be interested to know what you find.  If it recodes, I'd also be
interested to know where it gets its idea of the "local encoding".

For that matter, you were writing (upthread) as if you are resposible
for AsyncSSH.  Assuming that's true, what does _it_ do to recode (say)
the octets representing the local username into UTF-8 for transmission?
Error if not already UTF-8?  If not, where does it get the relevant
encoding from?

>> There also are places where it's not clear whether there's an
>> encoding issue.  For example, in a pty-req channel request, there is
>> a "string" which is documented as "TERM environment variable value".
>> It is not clear whether this use of "string" is text in the sense
>> described on 4251 page 9.
> There is already a terminal type field in the SSH 'pty-reqâ?? channel messag$

There is.  That's exactly the field I'm talking about.  4254 documents
it as

      string    TERM environment variable value (e.g., vt100)

which (a) assumes that terminal types are strings, (b) that the system
has environment variables, (c) that the terminal type is kept in one,
and (d) that variable's name is TERM.  It basically amounts to assuming
the whole world is UNIX variants.

/~\ The ASCII				  Mouse
\ / Ribbon Campaign
 X  Against HTML		[e-mail]
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B