Autodir HOWTO

Introduction

Autodir and difficult to reach nest
Autodir offers a simple and effective means to create directories like home directories in a transparent manner. It relies on the autofs protocol for its operation.
This expert level document explains how to create directories on demand using Autodir in a transparent way to the applications.

How it works

Autodir uses modules to get functionality. The core Autodir implements generic functionality on which modules add specific functionality of their own.
If there are two modules, for example autohome, autogroup, two processes of Autodir should be started.

Note

For further explanation I chose autohome module to handle home directories.
When Autodir is started, autofs file system is mounted on /home directory. And this is informed to the Linux kernel that /home is managed by Autodir from now on.
Whenever an user or daemon needs user's home directory, for example /home/userhome1, they enter into /home/userhome1 to access it even it is not visible.
Kernel which notices this, informs to Autodir.

Important

Autodir rejects this request if it is not valid request.
If the request is accepted, home directory userhome1 is created in root owned restrictive /autohome directory.

Note

Home directory is not created in /home but in root owned /autohome
Finally, /autohome/userhome1 is mapped to /home/userhome1 with proper permissions.

Note

Whole operation is transparent to the user.

Some definitions

Before going further it is better to understand the following terms to simplify explanation.
Virtual directories
These directories do not exist on disk. Instead these are created and deleted on demand in memory. If system reboots all these directories vanish. In the previous explanation, all directories under /home are virtual directories.
Virtual base directory
This is the directory that holds all Virtual directories. In the previous explanation /home is virtual base directory.
Real directories
These are the directories that actually reside on the disk. Even after reboot, these remain intact. In the previous explanation all directories created under /autohome are real directories.
Real base directory
This is the directory that holds all real directories. In the above explanation /autohome is real base directory.
Each virtual directory is mapped to real directory.

Note

In the above explanation, /home/userhome1 is mapped to /autohome/userhome1
On reboot of the system real directories and their content remain intact. Virtual directories are created on demand.
Virtual directories are removed if these are not used and created again if necessary. When Virtual directory is removed backup program is started on corresponding real directory if backup is configured.

Important

Applications access virtual directories. Real directories are hidden from applications except for root. But there is one exception. Backup programs always access real directories.

Virtual directory removal

When an application tries to access virtual directory, Autodir maps virtual directory to real directory.
If this virtual directory is not accessed for a specified time, it is removed and corresponding real directory is marked for backup.
The time period to wait for removal can be given through command line option to Autodir.

Backup support

Autodir supports backup program launching when a virtual directory is removed after a period of inactivity. Removal of virtual directory is itself is an assurance that no other application can access the content and modify it.
By design, backup programs are expected to operate on real directory but not on virtual directory. If backup program try to access virtual directory Autodir assumes some regular application is in need of that directory and backup program is killed even if the virtual directory accessing process is backup program itself.
A separate backup process for each real directory is used.
Backup program can be given arguments.

Note

Backup support is independent of any particular module being used. It is applicable to all modules with Autodir.

Important

Backup programs should never access virtual directory or virtual base directory.

Note

Backup feature is not much useful if virtual directories are accessed all the time by applications.

More on backup

Autodir demands some extra requirements from backup program being used.
When backup is working on real directory and that corresponding virtual directory is requested by an application while backup is running, backup is killed.
First SIGTERM is sent to gracefully stop it. But if it does not shutdown in time SIGKILL will be sent which is guaranteed to stop the backup.

Note

When and only when backup stopped, application is given access to the virtual directory requested.

Important

Whatever backup is used, it should be able to recover from this signal gracefully, not causing unrecoverable side effects.
One more important issue is that the environment under which it is run.
All backup programs are run as root user.
But at the same time all unnecessary root privileges are taken away using POSIX capabilities. In other words these backup programs can read any file or directory that belongs to any user on the system. Other than that it is like ordinary user level process.

Few requirements

  • Linux kernel equal to or later version of 2.4. These kernel versions support mounting one directory on another directory.
  • Autodir requires autofs4 kernel module based on protocol version 4. Autofs kernel module is pretty standard and almost all distributions include it.

Introduction to Autodir options

There are two kinds of options that can be passed to Autodir.
In the first type, options are for Autodir and do not change irrespective of which module is used.
There are other type of options which are specific to the module being used. These options called suboptions and are passed to the module being used differently with Autodir option -o. This is similar to mount command suboptions.
For example, suboptions to the example module autohome can be passed as, 
-o 'realpath=/tmp/autohome,level=2,noskel'
Here realpath, level, noskel are suboptions for autohome module.

Load kernel module

Kernel module autofs must be loaded before even starting autodir for its proper operation.
This can be done as root user and using modprobe command as follows, 
# modprobe autofs4

Importing user and group accounts

If user and group accounts reside in centralized database these must be imported before starting Autodir. How to do this is out of scope of this HOWTO. But there are number of documents which explain how to do this in clear manner.

From Source

If source is downloaded, follow these simple steps to install it.
  • Unpack the source.
    $ tar zxvf <tar file name>
  • Move to the expanded directory and execute the following.
    $ ./configure
    $ make
    # make install

Managing Home directories

This section will explain how to configure Autodir to manage home directories with help from autohome module.
To load autohome module with Autodir, use option -m. For example, -m /usr/lib/autodir/autohome.so.

Important

Autodir expects user name and home directory are related to each other. For example, for user user1 the home directory should be /home/user1 or /some/directory/name/user1 but not /some/directory/name/userhome1.

Note

If the existing user password database is such that user home directories are distributed under different base directories, for example /home/class1/user1, /home/class2/user2332, then autohome configuration becomes complicated.

Base directories for autohome

Where should be virtual base directory and real base directory for autohome module?
It all depends on how user accounts are created. If an user account created for user name user1 with home directory /home/user1 then /home is virtual base directory.
Autodir option -d is used to specify virtual base directory. For example autodir -d /home assuming /home is virtual base directory.
Then whare should be real base directory? It can be any directory in any partition. But there should be enough space as all actual files are stored here instead of in virtual base directory.
If the real base directory decided to be in /var/autohome then it is specified with option -o as -o realpath=/var/autohome.

Misc suboptions for autohome

Suboption skel can be used if skeleton path is not default value /etc/skel like -o skel=/some/other/dir.
Suboption noskel can be used with -o to indicate not to copy skeleton files to home directories when created.

Summing up with an example

First, import user accounts from centralized database mysql, ldap, NIS.
Next, autofs4 kernel module must be loaded into the Linux kernel. This can be done as described in autofs kernel module section.
If /home is to be used for home directories then /home will become virtual directory and specified to autodir with -d /home option.
Assuming autohome module is located at /usr/lib/autodir/autohome.so, this module can be loaded with autodir as -m /usr/lib/autodir/autohome.so. Note that full path for module is given.
Where actually real home directories reside is given with realpath suboption. If it is /autohome, it can be given as realpath=/autohome.
With all these options autodir can be started as, 
# autodir -d /home 						\
		-m /usr/lib/autodir/autohome.so 		\
		-o 'realpath=/autohome'			 	\
Once Autodir is started, /home directory will be blank in the beginning. Whether Autodir working properly or not can be tested by changing directory to one of the home directories as root user or as the owner of the home directory.

Autodir options

In this section some of the options to Autodir are explained.
-d
For specifying virtual base directory. If this path does not exist, it will be created. Absolute path is expected for this option.
-t
Expiration timeout for virtual directories. For more details refer to virtual directory expiration.
-m
Module to be used with Autodir. Currently autohome and autogroup are available. Full path to the module expected.
-o
All suboptions that are to be passed to module are given here. This option passing syntax is similar to mount command with its -o option. See specific module sections for more info.
-f
Stay foreground and log all messages to the console. For debugging purpose and to see how Autodir works.
-l
This option expects path name to filename to which Autodir will write its process id.
-h
Help about all options supported by Autodir.
-v
Version information about Autodir.

Backup options

These options are passed to Autodir to request backup support.
-b
This is the main option to specify backup program path and arguments to it. The path given should be absolute path otherwise Autodir does not accept it.
-w
Whenever a virtual directory is not used for a period of time, it is assumed inactive and it is unmounted. After unmounting directory, whether to launch backup immediately or to wait some more time is decided with this option. It takes arguments in seconds. It is the minimum time to wait before starting backup after virtual directory expiration. It should not exceed more then one day.
-p
This is the priority to be given to backup process. This is in the range of 1 to 40 inclusive. Lower value mean higher priority and vice versa. Default value is 30.
-c
This restricts maximum number of backup process at any given time. Default is 150.

Important

Argument for -b is inclusive of absolute backup program path as well as its own arguments. Therefore it is recommended to use single quotes around this argument
Option -b takes path to executable file as well as arguments to it. But the arguments to it are interpreted for %x character sequences and replaced with predefined strings as follows.
%N
Replaced with virtual directory name.
%L
Replaced with absolute path to real directory.
%K
Replaced with host name.
Others
Others are fed to strftime. See man page for strftime for more information.

Examples

# autodir -d /home 						\
		-m /usr/lib/autodir/autohome.so 		\
		-t 1000						\
		-f 						\
		-o 'realpath=/autohome,level=1,skel=/etc/skel' 	\
		-l /var/run/autodir.pid

# autodir -d /home						\
		-m /usr/lib/autodir/autohome.so			\
		-t 300						\
		-b '/bin/tar cf /tmp/%N%F.tar %L'		\
		-w 600						\
		-o 'realpath=/tmp/autohome,level=2,noskel'	\
		-l /var/run/autodir.pid

# autodir -d /var/abase/ 					\
		-m /usr/lib/autodir/autogroup.so		\
		-t 300						\
		-b '/bin/tar cf /tmp/%N%F.tar %L'		\
		-w 86400					\
		-o 'nopriv,nosetgid,realpath=/var/realbase,level=0'

Further Information

Autodir and difficult to reach nest
Mailing list for autodir http://lists.sourceforge.net/mailman/listinfo/intraperson-autodir.
Official website is at http://www.intraperson.com/autodir/.


Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License


Get intraPerson at SourceForge.net. Fast, secure and Free Open Source software downloads