3. Using EFS

Once installed, efs operates largely transparently. All files normally accessible to you on the internet, become part of a large virtual file system. These files are accessed using an extended file name syntax. To access file <path> on remote host <host> by logging in as user <user>, you simply specify the full path of the file as /<user>@<host>:<path>. Nearly all Emacs file handling functions work for remote files. It is not possible to access remote files using shell commands in an emacs *shell* buffer, as such commands are passed directly to the shell, and not handled by emacs.

FTP is the underlying utility that efs uses to operate on remote files.

For example, if find-file is given a filename of:

/ange@anorman:/tmp/notes

then EFS will spawn an FTP process, connect to the host 'anorman' as user 'ange', get the file `/tmp/notes' and pop up a buffer containing the contents of that file as if it were on the local file system. If efs needed a password to connect then it would prompt the user in the minibuffer. For further discussion of the EFS path syntax, see the paragraph on extended file name syntax 3.2 Extended filename syntax.

Full file-name completion is supported on every type of remote host. To do filename completion, EFS needs a listing from the remote host. Therefore, for very slow connections, it might not save any time. However, the listing is cached, so subsequent uses of file-name completion will be just as fast as for local file names.

3.1 Using nonstandard ports
3.2 Extended filename syntax		The EFS extended filename syntax.
3.3 Passwords
3.4 Using Dired		Browsing directories.
3.5 Using a .netrc file		Preventing password pestering.
3.6 EFS commands		Interactive commands supplied by EFS.
3.7 FTP processes		How EFS does its work
3.8 Tips for using EFS		Some stuff to help you use EFS
3.9 Descriptive directory listings
3.10 Using EFS with non-Unix hosts		Some of what you want to know
3.11 File- and host-name completion		Works but has its price
3.12 Accessing the FTP process buffer		manually

3.1 Using nonstandard ports

EFS supports the use of nonstandard ports on remote hosts. To specify that port <port> should be used, give the host name as host#<port>. Host names may be given in this form anywhere that efs normally expects a host name. This includes in the `.netrc' file. Logically, EFS treats different ports to correspond to different remote hosts.

3.2 Extended filename syntax

The default full EFS path syntax is

/<user>@<host>#<port>:<path>

Both the #<port>'and <user>@ may be omitted.

If the #<port> is omitted, then the default port is taken to be 21, the usual FTP port. For most users, the port syntax will only very rarely be necessary.

If the <user>@ is omitted, then EFS will use a default user. If a login token is specified in your `.netrc' file, then this will be used as the default user for <host>. Otherwise, it is determined based on the value of the variable efs-default-user.

This EFS path syntax can be customised to a certain extent by changing a number of variables. To undertake such a customization requires some knowledge about the internal workings of EFS.

3.3 Passwords

A password is required for each host / user pair. This will be prompted for when needed, unless already set by calling efs-set-passwd, or specified in a valid `~/.netrc' file.

When EFS prompts for a password, it provides defaults from its cache of currently known passwords. The defaults are ordered such that passwords for accounts which have the same user name as the login which is currently underway have priority. You can cycle through your list of defaults with C-n to cycle forwards and C-p to cycle backwards. The list is circular.

3.3.1 Passwords for anonymous user

Passwords for the user anonymous (or ftp) are handled specially. The variable efs-generate-anonymous-password controls what \vindex efs-generate-anonymous-password happens. If the value of this variable is a string, then this is used as the password; if non-nil, then a password is created from the name of the user and the hostname of the machine on which Emacs is running; if nil (the default) then the user is prompted for a password as normal.

3.3.2 Account passwords

Some FTP servers require an additional password which is sent by the ACCOUNT command. EFS will detect this and prompt the user for an account password if the server expects one. Also, an account password can be set by calling efs-set-account, or by specifying an account token in the `.netrc' file.

Some operating systems, such as CMS, require that ACCOUNT be used to give a write access password for minidisks. efs-set-account can be used to set a write password for a specific minidisk. Also, tokens of the form

minidisk <minidisk name> <password>

may be added to host lines in your `.netrc' file. Minidisk tokens must be at the end of the host line, however there may be an arbitrary number of them for any given host.

3.3.3 Broken ftp clients

Many ftp clients are derivatives of the ftp command found in 4.2BSD. This has a bug in the implementation of the QUOTE command which causes the commands to be fed into sprintf as format strings. For passwords, this means that \% characters are misinterpreted. To work around this bug, set efs-ftp-broken-quote to non-NIL.

3.4 Using Dired

This feature of EFS is particularly useful when file transfers, as opposed to file editing, are the order of the day. Simply run find-file on a directory to get a listing of the files in that directory. For example, you might run find-file on

/anonymous@archive.site.com:pub

to see what's in the `pub' directory of your favourite archive site. This brings up a Dired buffer of all the files in that directory. The f command is useful for looking at `README' files -- if you then decide to save it C-x C-w is useful. You can also use this method to copy files, but the c command is easier. The f command can also be used to descend the directory tree by applying it to directories.

You can also use Dired to refresh EFS's internal cache. If you (or anybody else) has changed a remote directory since you first accessed it with EFS, completion is not provided on any new files that EFS does not know about. If you have (or create) a Dired buffer which contains the modified directory, executing revert-buffer with a prefix argument (C-u g in the Dired buffer) will force a refresh of both the the buffer and also EFS's internal cache. If you find that filename completion isn't working on a file that you know is there, this is how to fix the problem.

Dired provides facilities for maintaining an entire directory tree in a Dired buffer, for marking files which match a certain regexp (or you can select files interactively) and then copying all those files to your local host (or even a different remote host). Another useful feature is Virtual Dired, which allows you to save Dired buffers of remote hosts, allowing you to browse them at a later date without actually needing to connect to the host.

3.5 Using a .netrc file

Being prompted for passwords all the time can get rather annoying, but there is a way to fix the problem -- a `.netrc' and efs-netrc-filename.

if you want another filename) file in your home directory. Basically, this is a file (in the format of Unix netrc(5)) which contains the names of all the machines you regularly login to, as well as the username and password you use for that machine. You can also supply an account password, if required.

Your `.netrc' file consists of lines of the form

machine <machine-name> login <user-name> password <password>

It doesn't all have to be on the one line, though: any login or password commands in the file refer to the previous machine command. You can also have

account
<account-passwd>

commands if you need special account passwords.

For example, you might have the following line in your `.netrc':

machine Y.local.lan.edu login myname password secret

Then if you run find-file on the file `/Y.local.lan.edu:somefile' you will automatically be logged in as user myname with password secret. You can still login under another name and password, if you so desire: just include the user@ part of the filename.

You may also include a default option, as follows:

default login <user-name> password <password>

which applies to any other machines not mentioned elsewhere in your `.netrc'. A particularly useful application of this is with anonymous logins:

default login myname password myname@myhost.edu

so that accessing `/anyhost:anyfile' will automatically log you in anonymously, provided the host is not mentioned in the `.netrc'. Note also that if the value of efs-default-user is non-nil, its value will have precedence over the username supplied in the default option of the `.netrc'.

The `.netrc' file is also useful in another regard: machines included in it are provided with hostname completion. That is, for any machine in the `.netrc', you need only type a slash and the first few characters of its name and then press TAB to be logged in automatically with a username and password from the `.netrc' file. So it's a good idea to put hosts you use regularly in your `.netrc' as well:

machine archive.site.com login anonymous password myname@X.local.lan.edu

3.6 EFS commands

EFS supplies a few interactive commands to make connecting with hosts a little easier.

Command efs-set-user: Prompts for a hostname and a username. Next time access to the host is attempted, EFS will attempt to log in again with the new username.

Command efs-set-passwd: Prompts for a hostname, user and password. Future logins to that host as that user will use the given password.

Command efs-set-account: Prompts for a hostname, user and account. Future logins to that host as that user will use the given account.

Note that the effects of the above three commands only last the duration of the current Emacs session. To make their effects permanent, you may include them as lisp code in your `.emacs':

(efs-set-user HOST USER)
(efs-set-password HOST USER PASSWORD)
(efs-set-account HOST USER ACCOUNT)

This is an alternative to using a `.netrc'; See section 3.5 Using a .netrc file.

Command efs-kill-ftp-process: kill the FTP process associated with a given buffer's filename (by default the current buffer). This is an easy way to achieve a resynch: any future accesses to the remote host will cause the FTP process to be recreated.

3.7 FTP processes

When EFS starts up an FTP process, it leaves it running for speed purposes. Some FTP servers will close the connection after a period of time, but EFS should be able to quietly reconnect the next time that the process is needed.

The FTP process will be killed should the associated `*ftp user@host*' buffer be deleted. This should not cause efs any grief.

3.7.1 Showing background FTP activity on the mode-line

After EFS is loaded, the command efs-display-ftp-activity will cause background FTP activity to be displayed on the mode line. The variable efs-mode-line-format is used to determine how this data is displayed. efs does not continuously track the number of active sessions, as this would cause the display to change too rapidly. Rather, it uses a heuristic algorithm to determine when there is a significant change in FTP activity.

3.7.2 File types

By default EFS will assume that all files are ASCII. If a file being transferred matches the value of efs-binary-file-name-regexp then the file will be assumed to be a binary file, and EFS will transfer it using "type image". ASCII files will be transferred using a transfer type which efs computes to be correct according to its knowledge of the file system of the remote host. The command efs-prompt-for-transfer-type toggles the variable efs-prompt-for-transfer-type. When this variable is non-nil, EFS will prompt the user for the transfer type to use for every FTP transfer. Having this set all the time is annoying, but it is useful to give special treatment to a small set of files. There is also a variable efs-text-file-name-regexp. This is tested before efs-binary-file-name-regexp, so if you set efs-text-file-name-regexp to a non-trivial regular expression, and efs-binary-file-name-regexp to `".*"', the result will to make image the default transfer type.

Also, if you set efs-treat-crlf-as-nl, then EFS will use type image to transfer files between hosts whose file system differ only in that one specifies end of line as CR-LF, and the other as NL. This is useful if you are transferring files between UNIX and DOS machines, and have a package such as `dos-mode.el', that handles the extra ^M's.

3.7.3 Status reports

Most EFS commands that talk to the FTP process output a status message on what they are doing. In addition, efs can take advantage of the FTP client's HASH command to display the status of transferring files and listing directories. See the documentation for the variables efs-hash-mark-size, efs-send-hash and efs-verbose for more details.

3.7.4 Caching of directory information

EFS keeps an internal cache of file listings from remote hosts. If this cache gets out of synch, it can be renewed by reverting a dired buffer for the appropriate directory (dired-revert is usually bound to g).

Alternatively, you can add the following two lines to your `.emacs' file if you want C-r to refresh EFS's cache whilst doing filename completion.

(define-key minibuffer-local-completion-map "\C-r" 'efs-re-read-dir)
(define-key minibuffer-local-must-match-map "\C-r" 'efs-re-read-dir)

3.8 Tips for using EFS

Beware of compressing files on non-UNIX hosts. EFS will do it by copying the file to the local machine, compressing it there, and then sending it back. Binary file transfers between machines of different architectures can be a risky business. Test things out first on some test files. Look at See section 5. Bugs and Wish List. Also, note that EFS sometimes copies files by moving them through the local machine. Again, be careful when doing this with binary files on non-Unix machines.
Beware that dired over ftp will use your setting of dired-no-confirm (list of dired commands for which confirmation is not asked). You might want to reconsider your setting of this variable, because you might want confirmation for more commands on remote direds than on local direds. For example, I strongly recommend that you not include compress in this list. If there is enough demand it might be a good idea to have an alist efs-dired-no-confirm of pairs ( TYPE . LIST ), where TYPE is an operating system type and LIST is a list of commands for which confirmation would be suppressed. Then remote dired listings would take their (buffer-local) value of dired-no-confirm from this alist. Who votes for this?
Some combinations of FTP clients and servers break and get out of sync when asked to list a non-existent directory. Some of the ai.mit.edu machines cause this problem for some FTP clients. Using efs-kill-ftp-process can be used to restart the ftp process, which should get things back in synch.
Some ftp servers impose a length limit on the password that can be sent. If this limit is exceeded they may bomb in an incomprehensible way. This sort of behaviour is common with MVS servers. Therefore, you should beware of this possibility if you are generating a long password (like an email address) with efs-generate-anonymous-password.
Some antiquated FTP servers hang when asked for an RNFR command. EFS sometimes uses this to test whether its local cache is stale. If your server for HOST hangs when asked for this command, put
(efs-set-host-property HOST 'rnfr-failed t)
in your efs-ftp-startup-function-alist entry for HOST.
The FTP servers on some Unix machines have problems if the ls command is used. EFS will try to correct for this automatically, and send the dir command instead. If it fails, you can call the function efs-add-host, and give the host type as dumb-unix. Note that this change will take effect for the current Emacs session only. To make this specification for future emacs sessions, put
(efs-add-host 'dumb-unix "hostname")
in your `.emacs' file. Also, please report any failure to automatically recognize dumb unix to the "bugs" address given below, so that we can fix the auto recognition code.

3.9 Descriptive directory listings

Some hosts (such as cs.uwp.edu) now use descriptive directory listings (which in fact contain less information than the standard listing!) when issued the ls command, and EFS has been modified to cope with this. EFS can detect such listings, but if you regularly use a remote host which uses this extended listing format you should set the variable efs-dl-dir-regexp to a regular expression which matches directories using the extended listing format. You shouldn't anchor the regexp with `$' -- that way the regexp will match subdirectories as well. Alternatively, you can use the interactive command efs-add-dl-dir to temporarily add a remote directory for this Emacs session only.

Dired has been modified to work with such descriptive listings.

3.10 Using EFS with non-Unix hosts

EFS also works with some non-Unix hosts, although not necessarily with all the features available with Unix hosts. VMS, CMS, and MTS systems will all now work with EFS and Dired. It also works with a whole bunch of others, but documentation for that has not been written yet. This section was taken straight from the ange-ftp manual, and is therefore in all likelihood out-of-date.

EFS should be able to automatically detect which type of host you are using (VMS, CMS or MTS), but if it is unable to do so you can fix the problem by setting the appropriate efs-TYPE-host-regexp variable (where TYPE is one of `vms', `cms' or `mts') -- see below. If EFS is unable to automatically detect any VMS, CMS or MTS host, please report this as a bug: See section 5. Bugs and Wish List.

In all cases the file-name conventions of the remote host are converted to a UNIX-ish format, and this is the format you should use to find files on such hosts.

3.10.1 VMS support		Using EFS with VMS systems
3.10.2 CMS support		Using EFS with CMS systems
3.10.3 MTS support		Using EFS with MTS systems

3.10.1 VMS support

VMS filenames are of the form FILE.TYPE;##, where both FILE and TYPE can be up to 39 characters long, and ## is an integer version number between 1 and 32,767. Valid characters in filenames are `A'-`Z', `0'-`9', `_', `-' and `$', however `$' cannot begin a filename and `-' cannot be used as the first or last character.

Directories in VMS are converted to the standard UNIX `/' notation. For example, the VMS filename

PUB$:[ANONYMOUS.SDSCPUB.NEXT]README.TXT;1

would be entered as

/PUB$$:/ANONYMOUS/SDSCPUB/NEXT/README.TXT;1

(The double `$' is required to prevent Emacs from attempting to expand an environment variable.) Similarly, to anonymously FTP the file `[.CSV.POLICY]RULES.MEM;1' from ymir.claremont.edu you would type C-x C-f /anonymous@ymir.claremont.edu:CSV/POLICY/RULES.MEM;1. You can always drop off the `;##' part at the end of the filename to get the latest version.

Sandy Rutherford provides some tips for using VMS hosts:

Although VMS is not case sensitive, EMACS running under UNIX is. Therefore, to access a VMS file, you must enter the filename with upper case letters.
To access the latest version of file under VMS, you use the filename without the `;' and version number. You should always edit the latest version of a file. If you want to edit an earlier version, copy it to a new file first. This has nothing to do with EFS, but is simply good VMS operating practice. Therefore, to edit `FILE.TXT;3' (say 3 is latest version), do C-x C-f /ymir.claremont.edu:FILE.TXT. If you inadvertently do
C-x C-f /ymir.claremont.edu:FILE.TXT;3
you will find that VMS will not allow you to save the file because it will refuse to overwrite `FILE.TXT;3', but instead will want to create `FILE.TXT;4', and attach the buffer to this file. To get out of this situation, M-x write-file /ymir.claremont.edu:FILE.TXT will attach the buffer to latest version of the file. For this reason, in Dired f (dired-find-file), always loads the file sans version, whereas v, (dired-view-file), always loads the explicit version number. The reasoning being that it reasonable to view old versions of a file, but not to edit them.
VMS filenames often contain `$' characters: make sure you always quote these as `$$' and watch out for the Emacs bug which fails to quote `$''s when defaults are presented in the minibuffer: see See section 5. Bugs and Wish List.

EFS should automatically detect that you are using a VMS host. If it fails to do so (which should be reported as a bug) you can use the command efs-add-vms-host to inform EFS manually. For a more permanent effect, or if you use a VMS host regularly, it's a good idea to set efs-vms-host-regexp to a regular expression matching that host's name. For instance, if use use ymir.claremont.edu a lot, place the following in your .emacs:

(setq efs-vms-host-regexp "^ymir.claremont.edu$")

3.10.2 CMS support

EFS has full support, including Dired support, for hosts running CMS.

CMS filenames are entered in a UNIX-y way. Minidisks are treated as UNIX directories; for example to access the file `READ.ME' in minidisk `*.311' on `cuvmb.cc.columbia.edu', you would enter

/anonymous@cuvmb.cc.columbia.edu:/*.311/READ.ME

If `*.301' is the default minidisk for this account, you could access `FOO.BAR' on this minidisk as

/anonymous@cuvmb.cc.columbia.edu:FOO.BAR

CMS filenames are of the form `FILE.TYPE', where both `FILE' and `TYPE' can be up to 8 characters. Again, beware that CMS filenames are always upper case, and hence must be entered as such.

Sandy Rutherford provides some tips on using CMS hosts:

CMS machines, with the exception of anonymous accounts, nearly always need an account password. To have EFS send an account password, you can either include it in your `.netrc' (see See section 3.5 Using a .netrc file.), or use efs-set-account.
EFS cannot send "write passwords" for a minidisk. Hopefully, we can fix this.

EFS should automatically detect that you are using a CMS host. If it fails to do so (which should be reported as a bug) you can use the command efs-add-cms-host to inform EFS manually. For a more permanent effect, or if you use a CMS host regularly, it's a good idea to set efs-cms-host-regexp to a regular expression matching that host's name.

3.10.3 MTS support

EFS has full support, including Dired support, for hosts running the Michigan terminal system, and should be able to automatically recognise any MTS machine.

MTS filenames are entered in a UNIX-y way. For example, if your account was `YYYY', the file `FILE' in the account `XXXX:' on `mtsg.ubc.ca' would be entered as

/YYYY@mtsg.ubc.ca:/XXXX:/FILE

In other words, MTS accounts are treated as UNIX directories. Of course, to access a file in another account, you must have access permission for it. If `FILE' were in your own account, then you could enter it in a relative path fashion as

/YYYY@mtsg.ubc.ca:FILE

MTS filenames can be up to 12 characters. Like UNIX, the structure of the filename does not contain a type (i.e. it can have as many `.''s as you like.) MTS filenames are always in upper case, and hence be sure to enter them as such! MTS is not case sensitive, but an EMACS running under UNIX is.

EFS should automatically detect that you are using an MTS host. If it fails to do so (which should be reported as a bug) you can use the command efs-add-mts-host to inform EFS manually. For a more permanent effect, or if you use an MTS host regularly, it's a good idea to set efs-mts-host-regexp to a regular expression matching that host's name.

3.11 File- and host-name completion

Full filename completion is supported on all remote UNIX hosts and some non-Unix hosts. Hostnames also have completion if they are mentioned in the `.netrc' and no username is specified. However using the filename completion feature can be a bit of a two edged sword.

To understand why, we need to discuss how EFS works. Whenever EFS is asked to find a remote file (or directory) an ls command is sent to the FTP process to list all the files in the directory. This list is maintained in an internal cache, to provide filename completion for later requests on that directory. EFS keeps this cache up-to-date by monitoring Emacs commands which affect files and directories, but if a process outside Emacs (such as another user) changes a directory (e.g. a new file is added) completion won't work on that file since EFS doesn't know about it yet. The solution if to force EFS to reread the directory and update it's cache, and the easiest way to do that is with Dired--see See section 3.4 Using Dired.

Another problem is that the ls command can take a long time, especially when dealing with distant hosts over slow links. So if you're after a file in the `pub/images' directory but nothing else, it's a better idea to type pub/images/file TAB than pub/im TAB which will force a read of the `pub' directory (since EFS needs to know how to complete im). A little extra typing can often save a lot of waiting. Don't be afraid to use the TAB key once the directory is cached, though.

3.12 Accessing the FTP process buffer

The FTP process used to access the remote files is available for access if you wish. It will be in a buffer called `"*ftp remote-file-name*"', i.e. if you found the file

/anonymous@archive.site.com:pub/README

there will be a buffer

*ftp anonymous@archive.site.com*

where all the transfers are taking place. You can have a look at the buffer using C-x b as usual, and even type in commands to the FTP process under an interface very much like `shell-mode'. There are two instances when doing this can be very useful: one is accessing non-UNIX hosts, where Dired and filename completion may not work (if EFS even works at all). If you are going to use mget or mput, make sure you type glob first: EFS turns globbing off by default. Don't be afraid of changing directories, either -- EFS always uses absolute pathnames when communicating with the FTP process.

You can kill the FTP process at any time simply by killing this buffer. You can also call efs-kill-ftp-process. This won't cause EFS any grief whatsoever -- if you later make another request to that host, EFS will simply fire up another process and create a new buffer to hold it.

This document was generated by XEmacs shared group account on December, 19 2009 using texi2html 1.65.