More on doing backups with tar

Why tar?

tar is more useful that dump to backup individual files (and when you do not care about funny special files). dump is much better to backup entire partitions, and it can handle any special file (e.g. FIFOs). See the on-line FreeBSD handbook and our FreeBSD book for more details on how to use tar.

Large files

An important problem with tar is that it cannot archive large files (~2GB or more). However, the latest GNU's tar, version 1.13, can handle them. I placed a copy of the source code at /usr/dirt/src/tar-1.13. If you are using a FreeBSD 4.x machine, you can directly run the binary (/usr/dirt/bin/4.x/tar-1.13).

WARNING: If you use tar-1.13 to tar something, you have to use tar-1.13 to untar it. /usr/bin/tar does not understand the format used by tar-1.13. Therefore, clearly label any tape created with tar-1.13.

BTW, dump does handle large files.

Tape devices

The same tape drive can be address with to file names: /dev/nrsa0 and /dev/rsa0. The first one will not rewind the tape after the next operation completes (nrstands for no rewind), while the second will rewind the tape.

mt - magnetic tape manipulating program

This is a very useful program. I normally use mt rewind to rewind tapes (if using /dev/nrsa0), and mt comp off to disable compression.

IMPORTANT: When you are done using a tape, execute mt offline to prepare the tape for extraction. This is a must for the DLT8000 tape drive on oberon, I do not know about the other tapes drives.

Remote tar

It is possible to backup individual files without copying them to the machine with the tape drive. You can do this with tar or tar-1.13:

# /usr/dirt/bin/4.x/tar-1.13 cf - . | rsh oberon dd of=/dev/nrsa0 obs=20

Add yourself to the operator group in oberon, so you can use the tape drive without being root.

Efficient use of tapes: tips

• Compressed files cannot be compressed again, so disable compression with mt comp off when most of the files you want to backup are already compressed. Recompressing them will waste up to 5 GB in a 40 GB tape (tested.)
• A gigabyte is 10^9 bytes, and not 2^30 bytes. Therefore, a 40 GB tape can only contain 40,000,000,000 bytes.
• Default blocksizes do not result in maximum throughput. For instance, tar on the DLT8000 tape drive gives a throughput of less than 2 MB/s. This is way below the expected peak throughput (6 MB/s). You can improve this using the right blocksize:

  /usr/dirt/bin/4.x/tar-1.13 cf - . | dd of=/dev/nrsa0 obs=64k
  

  Using this command, you can achieve almost 6 MB/s. In other words, it will take an hour an 50 minutes to write a 40 GB tape. Using default parameters, it takes 5 hours and 30 minutes to write such tapes. Remember to use the same technique to untar:

  dd if=/dev/nrsa0 ibs=64k | /usr/dirt/bin/4.x/tar-1.13 -xf -

Remote backups frontend

I wrote a very useful frontend for doing remote backups. You can find it in ~fhernand/src/perl/back/oberon_tar.pl. It is not completely general yet, but it is a good starting point. Features include reporting (e.g. example) and several modes of operations. I will probably make it more powerful as my backup needs change. It is a good idea to print the report using a2ps, and include it in the tape case.