The IPSendfile Project
Presented at Networkshop92, Brisbane, Australia.
The internet is used to transport data from machine to machine, user to user. A very common use is for one user to send a file to another user. Unfortunately, and this is very surprising, the Internet family of services does not include a convenient method for doing this.
The most natural approach would be simply to ask, using the user interface, for the file to be sent to a destination user address consisting of the username and domain. This would treat the file as a message containing arbitrary binary data of any length and deliver the file to some part of the user file space.
The first candidate for providing this type of service is the File Transfer Protocol or FTP . FTP is an interactive service that requires an end-to-end connection with the destination host. The source user `logs in' with a username and password and then instructs the FTP program to move a file from their file space to the directory of the destination username.
The first problem with this is that the service is interactive and requires an end-to-end connection. Most users would rather send the file and get on with their work while the file is delivered to the destination. If there is a network problem, the file should be queued, and delivered when the problem is resolved. With FTP the user has to try again later manually.
The second problem is that the user must have a username and password on the destination machine. This is either inconvenient, in that the user then must have an account on every machine that a file is to be sent to, or is insecure if the destination user is to give out their username and password for the transfer.
There is a convention adopted by most systems that the user name `anonymous' can be used by anyone to retrieve files. This has been extended to the case of sending files by providing a writable directory, often called `incoming' (or was that `in.coming'?), where files can be left for a destination user. Once a file has been deposited in `incoming' you then have to send email to the destination user telling them there is a file waiting. This has the serious drawback that while the file is in `incoming' it can be viewed or deleted by other users.
FTP doesn't seem to be the answer to the file sending problem.
The messaging service that is almost universal on the Internet is electronic mail. It consists of a combination of SMTP  for message transport and RFC822  for message format. Unfortunately, this service is strongly oriented towards reletively short electronic mail messages containing 7-bit US-ASCII characters and so is unsuitable as it stands for transporting binary files of any length. The RFC822 message format standard has recently been extended (MIME ) to handle arbitrary message types in different encodings, including arbitrary binary. However, this is an unnatural layering with a binary transport protocol (TCP) carrying a 7-bit US-ASCII electronic mail service which is being used to carry binary messages.
It is far more natural to implement a messaging service that carries arbitrary length binary messages and use it to carry higher level message services such as a sender oriented file transfer service.
The `Sydney Unix Network' software and its commercial successor `MHSnet' that formed the basis for the ACSnet network does have a suitable architecture. It also has a sender initiated file transfer system. To send a file to another person a command like:
netfile email@example.com myfile
is used. The file is transported across the network by the MHSnet software and kept in a holding area on the destination machine. A message is then sent (via email) to the destination user notifying them that a file has arrived. The destination user then invokes the command:
to retrieve the file.
MHSnet provides many more facilities than are required to build an Internet message service. It has a complete routing table management system, support for any virtual circuit with a range of delay and error characteristics and a session layer protocol for establishing connections. It is also a proprietary package and is not widely available on the Internet.
To provide a sender initiated file transfer service for the internet it was decided to build a basic message transfer service.
The Message Transfer Service carries messages across a TCP connection. Messages may contain an arbitrary number of 8-bit bytes and no interpretation is made of the message by the Message Transfer Service. Carried with each message are the addresses of the source and destination users and a message type and associated parameters. The Message Transfer Service will process the received message in a way appropriate to the message type. This is expected to be implemented by the Message Transfer Service passing the message to another process.
The Message Transfer Service has the ability to start a transfer at an arbitrary point in the message. This would normally occur when a connection has been re-established after an unexpected disconnect and is in response to a receiver asking that the message transfer begin at a nominated point.
The service consists of three programs:
A user invokes sendmsg with a destination user address, a
message type and associated parameters and the data to send.
The message is placed on a queue which is periodically
scanned by sender which attempts to open a connection to
tcpsmd on the destination machine and then, using the
protocol described below, transfers the message. At the
destination machine, tcpsmd receives the file and then
invokes a program to process the particular message type
The message transfer protocol is designed to be extremely simple and easy to implement. It may be possible to implement this service using a language like Perl or with the Bourne shell with a small number of support programs. The description of the protocol that follows is informal and designed to convey the flavour rather than the exact detail. For a more detailed description see the draft RFC.
A message transfer is initiated by a sender opening a full duplex TCP connection  to a receiver. All commands are transmitted as 7-bit US-ASCII characters in 8-bit bytes with the high-order bit set to zero. Each command is terminated by an ASCII newline character (012). All data is transmitted as 8-bit bytes.
The receiver begins the session by sending an ``ACCEPT'' string with zero or more options.
The sender then sends the sequence:
ID unique-id TYPE name RECEIVER local-part [@ domain ] SENDER local-part [ FLAG flag ] [ PARAMETER param ] [ TIMEOUT number ] SIZE number
Items in square brackets are optional.
The unique identifier is a 14 character string of printable characters. The `name' is a message type, in the case of a file transfer it is ``file''. The `param' and `flag' are character strings consisting of fields separated by spaces.
The receiver responds by sending either:
The `address' is a number giving the point in the file that the transmission begins. This would normally be zero but in that case of a restart after a connection is broken would be the address of the last received data. If the receiver cannot continue with the transfer, it sends NAK and a `reason' string.
If the receiver has responded with START, the sender continues with the line:
In this command, `start' refers to the start address chosen by the sender which must be less than of equal to that chosen by the receiver.
This is followed by the data, exactly size-start bytes, and then
The `crc' field is a 32 bit CRC, calculated according to the Ethernet polynomial, over the size-start data bytes.
This transfer is now concluded by the receiver responding with either
ACK or NAK reason
At this point the receiver can close the connection to indicate it is unwilling to accept any more messages or if the connection remains open the process can repeat with the sender sending the initial sequence.
Using the message transfer service we can easily provide a sender oriented file transfer service.
This consists of three programs:
The user invokes sendfile with the destination user address
and the name of the file or files to be sent. This uses the
sendmsg function to send a message of type file containing
the files to the destination user. At the destination
machine, TCPsmd receives the message and invokes filer to
process it. Filer moves the file into a holding directory
and notifies the destination user (via email) that a file
has arrived. At some later time the destination user invokes
getfile to retrieve the file.
A sender initiated file transfer system has been developed based on a more general message transfer system.
The software is freely available via ftp from ftp.cs.usyd.edu.au in the TCPmsg directory. It is designed to be portable across all Unix and Unix-like systems.
The authors wish to thank Chris Rusbridge of the University of South Australia for suggesting that the Internet could benefit from a sendfile facility similar to that available on ACSnet.
We also wish to thank the AARNet board for providing a grant to allow the work to be carried out.
 RFC822 - D.H.Crocker. Standard for the Format of ARPA Internet Text Messages.
 RFC793 - J. Postel (ed.). Transmission Control Protocol - DARPA Internet Program Protocol Specification.
 RFC821 - J. Postel, Simple Mail Transfer Protocol.
 RFC959 - J. Postel, J. Reynolds. File Transfer Protocol.
 RFC1341 - N.Borenstein, N.Freed, MIME (Multipurpose Internet Mail Extension).