Revised 'how to do anony ftp' DRAFT

[April Marine <[e-mail]>]

Look for problems/comments either introduced with > in the margin
or between <<>>s.

April



ALMOST AN INTERNET DRAFT (BUT NOT YET)                       P. Deutsch
IAFA FTP USER DOC                                             A. Emtage
                                                                 Bunyip
                                                              A. Marine
                                                                    SRI
                                                          November 1992
                                                      Expires: May 1993


                 How to Do Anonymous File Transfers 
 

Status of This Memo

This document is an Internet Draft.  Internet Drafts are working
documents of the Internet Engineering Task Force (IETF), its Areas,
and its Working Groups. Note that other groups may also distribute
working documents as Internet Drafts).

Internet Drafts are draft documents valid for a maximum of six months.
Internet Drafts may be updated, replaced, or obsoleted by other
documents at any time.  It is not appropriate is use Internet Drafts
as reference material or to cite them other than as a "working draft"
or "work in progress."

Please check the Internet Draft abstract listing contained in the IETF
Shadow Directories (cd internet-drafts) to learn the current status of
this or any other Internet Draft.

This INTERNET-DRAFT provides information for the Internet community.
It does not specify an Internet standard. 


Expiration Date

This Internet Draft expires May 18, 1993.

Abstract

This document provides information for the novice Internet
user about using the File Transfer Protocol (FTP).  It
explains what FTP is, what anonymous FTP is, and what an
anonymous FTP archive site is.  It shows an example
anonymous FTP session.  It also discusses common ways
files are packaged for efficient storage and transmission.

Acknowledgements

This document is the result of work done in the Internet Anonymous
FTP Archives (IAFA) working group of the IETF.  Special thanks are
due to John Curran (BBN), Aydin Edguer (CWRU), Marsha Perrot
(PREPnet), Bob Peterson (Texas Instruments), and Stephen Tihor
(NYU) for excellent comments.


What is an Archive Site?

An archive site is a host that acts as a repository of
information, much like a conventional library.  Information
is stored on specific computers on the network, and is made
available for users to transfer to their local sites.  Users
can run software to identify this information and transfer
it to their own hosts.  Such a transfer is done with
a program that implements the File Protocol (FTP).


What is FTP?

FTP refers to one of the protocols within the TCP/IP
protocol suite used on the Internet.  The File Transfer
Protocol makes it possible to transfer files from one
Internet computer (or "host") to another.  There are many
ftp programs built on the specification of the FTP protocol.
A user of an ftp program must log in to both hosts in
order to transfer a file from one to the other.

It is common for a user with files on more than one host to
use the ftp program to regularly transfer files from one
host to another.  In this case, the user has an account on
both hosts involved, so he has passwords for both hosts.

However, Internet users may also take advantage of a wealth
of information available from archive sites by using a
general purpose account called "anonymous FTP".




What is Anonymous FTP?

Anonymous FTP is a means by which archive sites allow
general access to their archives of information.  These
sites create a special account called "anonymous."  User
"anonymous" has limited access rights to the archive host,
as well as some operating restrictions.  In fact, basically
the only operations allowed are logging in using FTP, listing the
contents of a limited set of directories, and retrieving
files.  Note that usually "anonymous" users are not allowed
to transfer files TO the archive site, merely retrieve files
from such a site.

Traditionally, this special anonymous user account will
accept any string as a password, although it is common to
use either the password "guest" or one's electronic mail
(e-mail) address.  Some archive sites now explicitly ask for
the user's e-mail address and will not allow login with the
"guest" password.  Providing an e-mail address is a courtesy
that allows archive site operators to get some idea of who
is using their services.


What Information Do You Need to Know?

To retrieve a specific file, a user needs to know what host
it is on, and the pathname of the file.  A pathname tells
the directory (and possibly subdirectories) that house the
file, and the name of the file.  Often discussions of
available files will not specifically say, "This file is
available for anonymous ftp from X host with Y pathname."
However, if a file is publicly announced as available and
referred to as something like pub/good-stuff on
nisc.sri.com, it is a good assumption that you can try to
transfer it.




A Sample Session

To start an ftp session on a UNIX or VMS host, you type "ftp" and
the host name or host IP address.  As an example, if you
wish to access the DDN Newtork Information Center (NIC)
archive site, you would normally execute one of the
following commands at the UNIX prompt:

        ftp nic.ddn.mil
or
        ftp 192.112.36.5

Observe that the first form uses the fully-qualified domain
name and the second uses the Internet address for the same
host.


The following is an example of connecting to the nic.ddn.mil
host to retrieve FYI 4, "FYI on Questions and Answers:
Answers to Commonly Asked 'New Internet User' Questions."

Note several things about the session.

 1. Every response the ftp program at the archive site gives
    is preceded by a number.  These numbers are called
    Reply Codes and are defined in the FTP specification,
    RFC 959.  The text that accompanies these reply codes
    can vary at different sites, and usually does.  Note
    that the nic.ddn.mil administrator has chosen to provide
    a list of directories to users when they log in.  This
    is unusual; normally users interested in knowing the
    list of accessible directories must give a command to
    list them.

 2. The password you type is never shown on your screen.

 3. It is possible to "browse" in archives, but often users
    already know the pathname of the file they want.  The
    pathname for FYI 4 is rfc/fyi4.txt.  In the example, we
    first connect to the 'rfc' directory (cd rfc), then get
    the specific file we know we want.  If you do not know 
    the name of the file you want, a file called README is
    probably the one to retrieve first.
    

paris% ftp nic.ddn.mil
Connected to nic.ddn.mil.
220-*****Welcome to the Network Information Center*****
    *****Login with username "anonymous" and password "guest"
    *****You may change directories to the following:
      ddn-news          - DDN Management Bulletins
      domain            - Root Domain Zone Files
      iesg              - IETF Steering Group
      ietf              - Internet Engineering Task Force
      internet-drafts   - Internet Drafts
      netinfo           - NIC Information Files
      netprog           - Guest Software (ex. whois.c)
      protocols         - TCP-IP & OSI Documents
      rfc               - RFC Repository
      scc               - DDN Security Bulletins
      std               - Internet Protocol Standards
220 And more!
Name (nic.ddn.mil:april):anonymous
331 Guest login ok, send "guest" as password.
Password:
230 Guest login ok, access restrictions apply.
ftp>cd rfc
250 CWD command successful.
ftp>get fyi4.txt
200 PORT command successful.
150 Opening ASCII mode data connection for fyi4.txt (91885 bytes).
226 Transfer complete.
local: fyi4.txt remote: fyi4.txt
94240 bytes received in 6 seconds (15 Kbytes/s)
ftp>quit
221 Goodbye.
paris%



Variations

The above example is of the ftp program available on UNIX
systems.  Other operating systems also make ftp programs
available.  The actual commands you type may vary somewhat
with other programs.  However, in general, you will do the
following with every ftp program:

  - Log in to your local host, and invoke the FTP program.

  - Provide the hostname or host address for the archive site.

  - Once connected to the remote host, log in with username
    "anonymous." 

  - Provide either the password "guest" or whatever the password the
    site requests.

  - Issue whatever ftp commands you require, such as those to
    change directories or to retrieve a file.

  - When finished, exit the FTP program, which will close your
    connection to the archive host.


Other FTP Commands

We have demonstrated some of the commands available with ftp
programs.  Many others are possible.  For example, once you
have logged in to a remote host:

  - You may view the contents of the directory to which you are
    connected.  Type "dir" or "ls" to do so.

  - You may ask the ftp program to display a list of available
    commands, typically by invoking the ftp program without
    arguments and typing "help."

    
  - You may rename a file by using the "get" command's optional
    local file name, which follows the remote file name on the
    command line.  You might rename a file when the remote file
    name exceeds the local file system's naming constraints, e.g.
    if the remote file name is too long.  An example of using the
    "get" command to rename a file when transferring it might be
    "get really-long-named-file.txt short.txt".

  - You may set BINARY mode to transfer executable programs or files
    of data.  Type "binary" to do so.  Usually
    ftp programs assume files use only 7 bits per byte, the norm for
>    standard ASCII-encoded files.  The BINARY command allows you to
>    transfer files that use the full 8 bits per byte without error,
>    but this may have implications on how the file is transferred
>    to your local system.
    

>    If you are not sure what format a file is in, it does not hurt
>    to use BINARY mode as a default.

  - You may transfer multiple files at the same time.  To set this
    mode, type "mget."  You then supply a file name pattern that
    the remote system understands and it tries to transfer each
    file in turn.  If your local FTP user agent cannot transform
    the remote file names into legal local file names, or if there
    are some files that must be transfered in ASCII mode and others
    that must be transfered in BINARY mode, you may not be able to
    take advantage of this facility.

  - <<others?>>

Full details on the commands and options available
are in the ftp documentation that comes with your system.
You can also type "help" at the ftp command prompt for
more a list of command options.

A copy of the UNIX version of the ftp documentation is
available from the online manual.  If your UNIX site has the
manuals installed, type the following at the UNIX prompt:

	% man ftp



The Packaging and Naming of Files

Several widely used conventions allow the efficient storage and
transmission of information stored at archive sites.

Information stored on archive sites is often "transformed" in three
common ways.  "Compressing" (reducing the size of) the stored
information makes more space available on the archive, and reduces the
amount of data actually transferred across the network.  "Bundling"
several files into one larger file maintains the internal directory
structure of the components, and allows users to transfer only one
larger object rather than several (sometimes hundreds) of smaller
files.  

In addition, binary data is often converted into an ASCII format for
transmission.  Traditionally, Internet RFC 822-based electronic mail
and USENET protocols did not allow the transmission of "binary"
(8-bit) data; therefore, files in binary format had to be transformed
into printable 7-bit ASCII before being transmission.

On many systems various file naming conventions are used to enable the
remote user to determine the format of the stored information without
first having to retrieve the files.  Below we list the more common
compression, bundling, and transformation conventions used on the
Internet.  This list is not intended to be exhaustive.  In all cases
public domain or freely-available implementations of the programs
associated with these mechanisms are currently available on the
network.

  1) compress/uncompress

  Filenames terminating in ".Z" normally signify files that have been
  compressed by the standard UNIX Lempel-Ziv "compress" utility.
  There is an equivalent program called "uncompress" to reverse the
  process and return the file to its original state.  No bundling
  mechanism is provided, and the resulting files are always in binary
  format, regardless of the original format of the input data.

  2) atob/btoa

  Performs a transformation of ASCII to binary (atob) and the reverse
  (btoa) in a standard format.  Files so transformed often have
  filenames terminated with ".atob".  No bundling or compression
  mechanisms are used.

  3) atox/xtoa

  A data transformation standard used to convert binary files to
  transferable ASCII format.  Sometimes used in preference to other
  similar mechanisms because it is more space efficient;  however, it is
  not a compression mechanism per se.  It is just more efficient in the
  transformation from one format to the other.  Filenames of files in
  this format often have the ".atox" extension.

  4) uuencode/uudecode

  Transforms ASCII to binary ("uuencode") and the reverse ("uudecode")
  transformation in a standard manner.  Originally used in the UUCP
  ("Unix to Unix CoPy") mail/USENET system.  No bundling or
  compression mechanisms are used.  Naming conventions often add
  a .uu at the end of the file name.

  5) tar/untar

  Originally a UNIX based utility for bundling (and unbundling)
  several files and directories into (and from) a single file (the
  acronym stands for "Tape ARchive").  Standard format provides no
  compression mechanism.  The resulting bundled file is always in
  binary format regardless of whether the constituent files are binary
  or not.  Naming conventions usually hold that the filename of a
  "tarfile" contain the sequence ".tar" or "-tar".

  6) zip/unzip

  Often used in IBM PC environments, these complementary programs
  provide both bundling and compression mechanisms.  The resulting
  files are always in binary format.  Files resulting from the "zip"
  program are by convention terminated with the ".zip" filename
  extension.

  7) arc/unarc

  Often used in IBM PC environments, these complementary programs
  provide both bundling and compression mechanisms.  The resulting
  files are always in binary format.  Files stored in this format
  often have a ".arc" filename extension.

  8) binhex

  Used in the Apple MacIntosh environment, the binhex process provides
  bundling as well as binary to ASCII data transformations.  Files in
  this format by convention have a filename extension of ".hqx".
 
  9) shar

  Bourse shell archives package text or binary files into a single longer
  file which, when executed, will create the component files.  Because this 
  format is vulnerable to misuse, most users use a special tool called unshar
  to decode these archives.  By convention, files in this format have a 
  filename extension of ".shar".

  10) VMS_SHARE

  DCL archives package text or binary files into a single longer
  file which, when executed, will created the component files.  Because this 
  format is vulnerable to misuse, care must be take to examine such an
  archive before executing it.  By convention, files in this format have a 
  filename extension of ".shar".

  11) Multipart shar/vms_share files

  Sometimes these shell archive files are broken into multiple
  small parts to simplify their transfer over other forms of
  fileservers that share the same archive tree.  In such cases, the
  parts of the files are usually suffixed with a part number (e.g.
  xyz.01 xyz.02 xyz.03 ... or even .01-of-05).  Collect all the
  parts, concatenate them on your local system, and then apply the
>  normal tool to the result.
<<"apply the normal tool to the result"???  Lost me>>

   
In some cases, a series of the above processes are performed to
produce the final file as stored on the archive.  In cases where
multiple transformation processes have been used, tradition holds that
the original (base) filename be changed to reflect these processes,
and that the associated filename extensions be added in the order in
which the processes was performed.  For example, a common procedure is
first to bundle the original files and directories using the "tar"
process, then to "compress" the bundled file.  Starting with a base
file name of "foobar", the file name in the archive would become
"foobar.tar.Z".  As this is a binary file, it would require a further
transformation into printable ASCII by a program such as "uuencode" in
order to be transmitted over traditional email or USENET facilities,
so it might finally be called "foobar.tar.Z.uu."

Some oeprating systems can not handle multiple periods; in such cases 
they are often replaced by hyphen ( - ), underscore ( _ ), or by
detailed instructions in the "read me" files in the directories.


Compress and Tar

Here is an example of the use of the "compress/uncompress" and 
"tar/untar" programs.

Suppose "patch" is a useful public domain program for applying program
patches and updates.  You find this file at an archive site as
"patch.tar.Z".  Now you know that the ".Z" indicates that the file was
compressed with the UNIX "compress" command, and the ".tar" indicates
that it was tar'ed using the UNIX "tar" tape archive command.

First retrieve the file onto your machine using anonymous ftp.  To
unpack this program, you would first  uncompress it by typing:

   uncompress patch.tar.Z

This will uncompress the file, and in the process, rename
it to "patch.tar".  You can then execute the "tar" command to
extract the individual files.

In the example of patch.tar, you could invoke the command
as:

	% tar xvf patch.tar

The files would be extracted (that's the 'x' argument to
tar) from the file patch.tar (that's the 'f' argument).
Because we use the 'v' (for verbose) argument, the name
of each file is printed as it is extracted.  When tar is
complete you should have all the files that make up the
"patch" program in your working directory.


Etiquette

Not every site that supports FTP permits anonymous tranfers.  It is
wrong to try to get files from systems that have not advertised the
availability of such a service. 

Remember that Internet site administrators for archive sites
have made their systems available out of a sense of
community.  Rarely are they fully compensated for the time
and effort it takes to administer such a site.  There are
some things users can do to make their jobs somewhat easier,
such as checking with local support personnel first if
problems occur before asking the archive adminstrator for help.

Most archive machines performing other functions as well.
Please respect the needs of their primary users and restrict
your ftp access to non-prime hours (generally between 1900
and 0600 hours local time for that site) whenever possible.
It is especially important to remember this for sites
located on another continent or across a significant body of
water because most such links are relatively slow and heavily loaded.

To reduce redundant storage, you should find out how to make useful
the files you fetch using FTP available to your entire
organization.  If you retrieve and test a program that turns out to
be useful, you should probably ask your administrator to consider
making the program generally available, which will reduce the
redundant effort and disk space resulting from multiple individuals
installing the same package in their personal directories.

If you find an interesting file or program on an archive
site, tell others about it.  You should not copy the
file or program to your own archive unless you are
willing to keep your copy current.

<<I stole these points from Peter, but there must be others we
can note>>

Security Considerations

Security considerations are not discussed in this document.


Authors' Addresses

  Peter Deutsch
  Bunyip Information Systems
  266 Blvd. Neptune  
  Dorval, Quebec, H9S 2L4
  Canada

  Phone: [phone]
  EMail: [e-mail]


  Alan Emtage
  Bunyip Information Systems
  266 Blvd. Neptune  
  Dorval, Quebec, H9S 2L4
  Canada

  Phone: [phone]
  EMail: [e-mail]


  April N. Marine
  SRI International
  Network Information Systems Center
  333 Ravenswood Avenue, EJ294
  Menlo Park, CA  94025-3493

  Phone: [phone]
  EMail: [e-mail]



Expiration Date

This Internet Draft expires May 18, 1993.