.\" Copyright 2010, 2011  Lars Wirzenius
.\" 
.\" This program is free software: you can redistribute it and/or modify
.\" it under the terms of the GNU General Public License as published by
.\" the Free Software Foundation, either version 3 of the License, or
.\" (at your option) any later version.
.\" 
.\" This program is distributed in the hope that it will be useful,
.\" but WITHOUT ANY WARRANTY; without even the implied warranty of
.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
.\" GNU General Public License for more details.
.\" 
.\" You should have received a copy of the GNU General Public License
.\" along with this program.  If not, see <http://www.gnu.org/licenses/>.
.TH OBNAM 1
.SH NAME
obnam \- make, restore, and manipulate backups
.SH SYNOPSIS
.B obnam
.RI [ options ]
.I command
.RI [ args ]...
.SH DESCRIPTION
.B obnam
makes, restores, manipulates, and otherwise deals with backups.
It can store backups on a local disk or to a server via sftp.
Every backup generation looks like a fresh snapshot,
but is really incremental:
the user does not need to worry whether it's a full backup or not.
Only changed data is backed up,
and if a chunk of data is already backed up in another file,
that data is re-used.
.PP
The place where backed up data is placed is called the
backup repository.
A repository may be, for example, a directory on an sftp server,
or a directory on a USB hard disk.
A single repository may contain backups from several clients.
Their data will intermingle as if they were using separate repositories,
but if one client backs up a file, the others may re-use the data.
.PP
.B obnam 
command line syntax consists of a 
.I command
possibly followed by arguments.
The commands are list below.
.IP \(bu
.B backup
makes a new backup.
The first time it is run, it makes a full backup,
after that an incremental one.
.IP \(bu
.B restore
is the opposite of a backup.
It copies backed up data from the backup repository to a target directory.
.IP \(bu
.B clients
lists the clients that are backed up to the repository.
.IP \(bu
.B generations
lists every backup generation for a given client, 
plus some metadata about the generation.
.IP \(bu
.B genids
lists the identifier for every backup generation for a given client.
No other information is shown.
This can be useful for scripting.
.IP \(bu
.B ls
lists the contents of a given generation, similar to 
.BR ls \-lAR .
.IP \(bu
.B verify
compares data in the backup with actual user data,
and makes sures they are identical.
It is most useful to run immediately after a backup,
to check that it actually worked.
It can be run at any time,
but if the user data has changed,
verification fails even though the backup is OK.
.IP \(bu
.B forget
removes backup generations that are no longer wanted,
so that they don't use disk space.
Note that after a backup generation is removed
the data can't be restored anymore.
You can either specify the generations to remove by listing them
on the command line,
or use the
.B \-\-keep
option to specify a policy for what to keep (everything else will
be removed).
.IP \(bu
.B fsck
checks the internal consistency of the backup repository.
It verifies that all clients, generations, directories, files, and
all file contents still exists in the backup repository.
It may take quite a long time to run.
.IP \(bu
.B force\-lock
removes a lock file for a client in the repository.
You should only force a lock if you are sure no-one is accessing that
client's data in the repository.
A dangling lock might happen, for example, if obnam loses its network
connection to the backup repository.
.IP \(bu
.B client\-keys
lists the encryption key associated with each client.
.IP \(bu
.B list\-keys
lists the keys that can access the repository,
and which toplevel directories each key can access.
Some of the toplevel directories are shared between clients,
others are specific to a client.
.IP \(bu
.B list\-toplevels
is like 
.BR list\-keys ,
but lists toplevels and which keys can access them.
.IP \(bu
.B add\-key
adds an encryption key to the repository.
By default, they key is added only to the shared toplevel directories,
but it can also be added to specific clients:
list the names of the clients on the command line.
They key is given with the
.B \-\-keyid
option.
Whoever has access to the secret key corresponding to the key id
can access the backup repository
(the shared toplevels plus specified clients).
.IP \(bu
.B remove\-key
removes a key from the shared toplevel directories,
plus any clients specified on the command line.
.SS "Making backups"
When you run a backup,
.B obnam
uploads data into the backup repository.
The data is divided into chunks,
and if a chunk already exists in the backup repository,
it is not uploaded again.
This allows 
.B obnam
to deal with files that have been changed or renamed since the previous
backup run.
It also allows several backup clients to avoid uploading the same data.
If, for example, everyone in the office has a copy of the same sales brochures,
only one copy needs to be stored in the backup repository.
.PP
Every backup run is a 
.IR generation .
In addition,
.B obnam
will make 
.I checkpoint
generations every now and then.
These are exactly like normal generations,
but are not guaranteed to be a complete snapshot of the live data.
If the backup run needs to be aborted in the middle,
the next backup run can continue from the latest checkpoint,
avoiding the need to start completely over.
.PP
If one backup run drops a backup root directory,
the older generations will still keep it:
nothing changes in the old generations just because there is a new one.
If the root was dropped by mistake,
it can be added back and the next backup run will re-use the existing
data in the backup repository,
and will only back up the file metadata (filenames, permissions, etc).
.SS "Verifying backups"
What good is a backup system you cannot rely on?
How can you rely on something you cannot test?
The
.B "obnam verify"
command checks that data in the backup repository matches actual user data.
It retrieves one or more files from the repository and compares them to
the user data.
This is essentialy the same as doing a restore,
then comparing restored files with the original files using 
.BR cmp (1),
but easier to use.
.PP
By default verification happens on all files.
You can also specify the files to be verified by listing them on the
command line.
You should specify the full paths to the files,
not relative to the current directory.
.PP
The output lists files that fail verification for some reason.
If you verify everything, it is likely that some files (e.g.,
parent directories of backup root) may have changed without it
being a problem.
Note that you will need to specify the whole path to the files
or directories to be verified, not relative to the backup root.
You still need to specify at least one of the backup roots via
the
.B \-\-root
option so that obnam will find the filesystem, in case it is
a remote one.
.SS "URL syntax"
Whenever obnam accepts a URL, it can be either a local pathname,
or an 
.B sftp
URL.
An sftp URL has the following form:
.IP
.BI sftp:// domain\fR[\fB:\fIport\fR] / path
.PP
where 
.I domain
is a normal Internetl domain name,
.I port
is an optional port number,
and
.I path
is a pathname on the server side.
Like
.BR bzr (1),
but unlike the sftp URL standard,
the pathname is absolute,
unless it starts with
.B /~/
in which case it is relative to the user's home directory on the server.
.PP
See the EXAMPLES section for examples of URLs.
.SS "Generation specifications"
When not using the latest generation,
you will need to specify which one you need.
This will be done with the
.B \-\-generation
option,
which takes a generation specification as its argument.
The specification is either the word
.IR latest ,
meaning the latest generation (also the default),
or a number.
See the
.B generations
command to see what generations are available,
and what their numbers are.
.SS "Policy for keeping and removing backup generations"
The
.B forget
command can follow a policy to automatically keep some and remove
other backup generations.
The policy is set with the
.BR \-\-keep =\fIPOLICY
option.
.PP
.I POLICY
is comma-separated list of rules.
Each rule consists of a count and a time period.
The time periods are 
.BR h ,
.BR d ,
.BR w ,
.BR m ,
and
.BR y ,
for hour, day, week, month, and year.
.PP
A policy of
.I 30d
means to keep the latest backup for each day, for thirty days.
Any backups in between will be removed,
as will any backups older than thirty days.
.PP
As an example, assume backups are taken every hour, on the hour:
at 00:00, 01:00, 02:00, and so on, until 23:00.
If the
.B forget
command is run at 23:15, with the above policy,
it will keep the backup taken at 23:00 on each day,
and remove every other backup that day.
It will also remove backups older than 30 days.
.PP
Note that obnam will only inspect timestamps in the backup repository,
and does not care what the actual current time is.
This means that if you stop making new backups,
the existing ones won't be removed automatically.
In essence, obnam pretends the current time is just after the
latest backup when 
.B forget
is run.
.PP
The rules can be given in any order,
but will be sorted to ascending order of time period before applied.
(It is an error to give two rules for the same period.)
A backup generation is kept if it matches any rule.
.PP
For example, assume the same backup frequence as above,
but a policy of
.IR 30d,52w .
This will keep the newest daily backup for each day for thirty days,
.I and
the newest weekly backup for 52 weeks.
Because the hourly backups will be removed daily,
before they have a chance to get saved by a weekly rule,
the effect is that the 23:00 o'clock backup for each day is
saved for a month,
and the 23:00 backup on Sundays is saved for a year.
.PP
If no policy is given,
.B forget
will keep everything.
.PP
A typical policy might be
.IR 72h,7d,5w,12m ,
to keep hourly backups for three days,
daily backups for a week,
weekly backups for a month,
and monthly backups for a year.
.\"
.SS "Using encryption"
.B obnam
can encrypt all the data it writes to the backup repository.
It uses 
.BR gpg (1)
to do the encryption.
You need to create a key pair using
.B "gpg --gen-key"
(or use an existing one),
and then tell
.B obnam
about it using the
.B \-\-encrypt\-with
option.
.\"---------------------------------------------------------------------
.SH OPTIONS
.\"
.TP
.BR \-h ", " \-\-help
Show a summary of options.
.\"
.TP
.BR \-\-log =\fIFILE
Write log messages to
.IR FILE .
.\"
.TP
.BR \-\-log\-level =\fILEVEL
Log messages at 
.I LEVEL
and above.
Level is one of 
.IR debug ,
.IR info ,
.IR warning ,
.IR error ,
or
.IR critical .
Default is
.IR info .
.\"
.TP
.BR \-\-repository =\fIURL
Store backups in
.IR URL ,
which may be either a pathname or an 
.B sftp
URL.
See above for a description of URL syntax.
.\"
.TP
.BR \-\-client\-name =\fICLIENT_NAME
Set name of client to use.
Default is the hostname of the machine where 
.B obnam 
runs.
This is used to determine which part of the backup repository to put 
backups in.
.\"
.TP
.BR \-\-pretend ", " \-\-dry\-run ", " \-\-no\-act
Do not actually remove anything, just pretend you do.
This currently only works for the 
.B forget
operation.
.\"
.TP
.BR \-\-node\-size =\fISIZE
Maximum size of B-tree nodes on disk.
Default is 65536 bytes.
.\"
.TP
.BR \-\-chunk\-size =\fISIZE
Maximum size of chunks of file data in backup repository.
Default is 65536 bytes.
.\"
.TP
.BR \-\-upload\-queue\-size =\fISIZE
Maximum length of upload queue for B-tree nodes.
Default is 1024 nodes.
.\"
.TP
.BR \-\-lru\-size\-size =\fISIZE
Maximum size of Least-Recently-Used cache for B-tree nodes.
Default is 10000 nodes.
.\"
.TP
.BR \-\-dump\-memory\-profile =\fIHOW
Make memory profiling dumps after each file, checkpoint, and generation.
Value is one of 
.IR none ,
.IR simple ,
.IR meliae ,
or
.IR heapy .
Default is 
.IR simple ,
which reports only the RSS value from the kernel,
and the number of live and garbage Python objects.
.\"
.TP
.BR \-\-root =\fIURL
What to back up.
Can be specified multiple times.
The
.I URL
can be specified the same way as for backup repositories.
This option is only relevant to the backup operation.
You can also give the URLs as normal command line arguments.
.\"
.TP
.BR \-\-exclude =\fIREGEXP
Exclude any files and directories whose full pathname matches the
regular expression given.
Can be given multiple times.
.\"
.TP
.BR \-\-checkpoint =\fISIZE
Make a checkpoint generation after a given size.
Default is 1 GiB.
.\"
.TP
.BR \-\-to =\fIDIR
Restore files to
.IR DIR .
This is only used with the
.B restore
command.
.\"
.TP
.BR \-\-generation =\fIGENSPEC
Use generation specified by
.IR GENSPEC .
See above for ways to specify a generation.
Default is
.IR latest .
.\"
.TP
.BR \-\-quiet
Do not report progress when running.
This is helpful when running obnam non-interactively,
for example from 
.BR cron (8).
.\"
.TP
.BR \-\-keep =\fIPOLICY
Specify which generations the
.B forget
command will keep.
Everything else will be removed.
See above for a description of
.IR POLICY .
.TP
.BR \-\-encrypt\-with =\fIKEYID
Encrypt data stored in the backup repository using
.B gpg
and the key specified with
.IR KEYID .
.TP
.BR \-\-keyid =\fIKEYID
Key identifier to be added to or removed from the repository by the
.B add\-key
and
.B remove\-key
commands.
.\" ------------------------------------------------------------------
.SH "EXIT STATUS"
.B obnam
will exit with zero if everything went well,
and non-zero otherwise.
.SH ENVIRONMENT
.B obnam
will pass on the environment it gets from its parent,
without modification.
It does not obey any unusual environment variables,
but it does obey the usual ones when running external programs,
creating temporary files, etc.
.SH FILES
There will some day be a configuration file, which will be documented here.
.SH EXAMPLE
To back up your home directory to a server:
.IP
.nf
obnam backup \-\-repository sftp://your.server/~/backups $HOME
.PP
To restore your latest backup from the server:
.IP
.nf
obnam restore \-\-repository sftp://your.server/~/backups \\
\-\-to /var/tmp/my.home.dir
.PP
To check that the backup worked:
.IP
.nf
obnam verify \-\-repository sftp://your.server/~/backups /path/to/file
.PP
To remove old backups, keeping the newest backup for each day for
ten years:
.IP
.nf
obnam forget \-\-repository sftp://your.server/~/backups \-\-keep 3650d
.PP
To verify that the backup repository is OK:
.IP
.nf
obnam fsck \-\-repository sftp://your.server/~/backups