Comme depuis 1 an je suis confronté à des problèmes de sauvegarde et de restauration aussi bien en environnement professionnel (je ne suis pourtant pas administrateur système) que personnel( les gens qui pensent que je fais des sauvegardes toute la journée 
), je vais étudier le problème…qui est bien vaste.
Je commence par TestDisk car je viens de recevoir un courriel de la liste http://linuxfr.org qui mentionne son existence.
- http://www.cgsecurity.org/wiki/TestDisk (“ est un logiciel Open Source distribué sous licence GPL (GNU Public License). TestDisk est un logiciel performant de récupération de données! Il a été conçu à l’origine pour aider à la récupération des partitions perdues, la réparation des tables des partitions corrompues quand ces symptômes ont été causés par des logiciels défectueux, certains types de virus ou des erreurs humaines (tel que l’effacement accidentel de la Table des Partitions”).
- http://www.cgsecurity.org/wiki/Monitoring_SMART_FR (“Self-Monitoring, Analysis and Reporting Technology system). Il est possible de contrôler et surveiller les systèmes de stockage en utilisant la technologie SMART (Self-Monitoring, Analysis and Reporting Technology system) intégré dans les disques durs ATA et SCSI.Dans le but d’anticiper les pannes, la technologie SMART surveille et informe de l’état de certains indicateurs de fiabilité comme la température, le nombre de secteurs réalloués, les erreurs de localisation des secteurs et le temps cumulé d’utilisation.
…
- Utiliser un logiciel pour surveiller les informations SMART afin de détecter les problèmes physiques.
- smartmontools — logiciel opensource pour Windows, Linux, FreeBSD, NetBSD, OpenBSD, Solaris, Darwin, OS/2.
- SMARTReporter — logiciel opensource pour Apple Macintosh
- Si le disque est connecté via USB ou FireWire, les informations SMART peuvent ne pas être rapportés. Si c’est le cas, il est conseillé d’utiliser une connexion directe de type IDE, SCSI, SATA…”
- http://www.cgsecurity.org/wiki/PhotoRec (“ PhotoRec is file data recovery software designed to recover lost files including video, documents and archives from Hard Disks and CDRom and lost pictures (thus, its ‘Photo Recovery’ name) from digital camera memory. PhotoRec ignores the filesystem and goes after the underlying data, so it will still work even if your media’s filesystem has been severely damaged or re-formatted. PhotoRec is free, this open source multi-platform application is distributed under GNU Public License. PhotoRec is a companion program to TestDisk, an app for recovering lost partitions on a wide variety of filesystems and making non-bootable disks bootable again. You can download them from this link. PhotoRec searches known file header and because there is no data fragmentation (usually), it can recover the whole file. Photorec recognises numerous file format including ZIP, Office, PDF, HTML, JPEG and various graphics file formats. The whole list of file formats recovered by PhotoRec contains more than 80 file extensions.”)
- http://www.windowsdream.com/ping/download.html (“PING (Partimage Is Not Ghost) Backup and Restore Disk Partitions. PING is a live Linux ISO, based on the excellent Linux From Scratch (LFS) documentation. It can be burnt on a CD and booted, or integrated into a PXE / RIS environment. Several tools have been added and written, so to make this ISO the perfect choice to backup and restore whole partitions, an easy way. It sounds like Symantec Ghost(tm), but has even better features, and is totally free“)
- http://www.windowsdream.com/winner.html (“WINNER is a different way to set up a RIS Server. Basically, it gives you the possibility to deploy an unattended installation of Windows XP over a LAN, but has impressive specificities. Amongst them, a WINNER distribution can range from a minimalist Linux set (50 Mo) to a complete set of files (2 Go) — both of them leading to a similar, complete, Windows installation.“)
- http://fr.wikipedia.org/wiki/Remote_Installation_Services (“Les Services d’installation à distance (Remote Installation Services ou RIS) est une fonctionnalité pour serveur fournit par Microsoft qui permet à des ordinateurs dont le BIOS prend en charge le Preboot eXecution Environment (PXE) d’exécuter de façon distante des variables d’environnement de démarrage…Il est possible d’installer ou booter d’autres systèmes d’exploitation grâce au RIS, par exemple une distribution GNU/Linux au travers du Linux Terminal Server Project…Le RIS a été remplacé par “Windows Deployment Services” (WDS) dans le Service Pack 2 de Windows Server 2003 et Windows Vista.“)
- http://fr.wikipedia.org/wiki/Ghost_%28informatique%29 (“En informatique, Ghost (General Hardware Oriented System Transfer) est un logiciel, à l’origine produit par Binary Research, mais racheté par Symantec en 1998, pour cloner un disque dur (copie physique), ou plus spécifiquement, de cloner chaque partition d’un disque dur (copie logique). Le cryptage AES (Standard de chiffrement avancé) est aussi supporté pour préserver la confidentialité des sauvegardes. Ghost inclut aussi un Fdisk amélioré. L’image obtenue peut notamment être utilisée :
- comme une sauvegarde pour restaurer l’état du disque au moment de la création de l’image
- comme modèle pour réaliser un clonage de disque
Le concurrent direct de Ghost est Acronis True Image (commercial) , souvent considéré comme plus complet et simple d’utilisation.
Remarque: Depuis peu (2004) il existe PING (Partimage Is Not Ghost) qui n’est ni un produit commercial ni un logiciel open source.
Le terme “ghost” tend à se généraliser pour désigner toute image d’un support informatique ou d’un système informatique faite par un logiciel de sauvegarde. Dans le jargon informatique, le néologisme “ghoster un PC” est apparu. Il est synonyme de cloner, c’est à dire dupliquer le disque dur d’un PC. L’intérêt de ghoster un PC est de ne pas avoir besoin de réinstaller chaque logiciel un par un, ce qui est long et fastidieux pour la (ou les) personne(s) qui installe(nt) ces logiciels. Ghoster un PC est rapide et nécessite beaucoup moins de compétences; il reste cependant à personaliser la machine clonée en modifiant son nom et son adresse IP à l’issue du clonage.”
- http://en.wikipedia.org/wiki/Dd_%28Unix%29 (“dd is a common UNIX program whose primary purpose is the low-level copying and conversion of files. The title of the UNIX sixth edition manual page of May 1975 was dd – convert and copy a file, and some have suggested that since “cc” was already in use as the code for the CCompiler, “dd” was used instead; in any case, DD is an acronym for “data definition” in IBM JCL (to which the command’s syntax bears a strong resemblance) and the name and syntax of the command is generally presumed to be a bit of gallows humor.” )
To create a zipped hard drive backup image:
# dd if=/dev/hda | gzip > /mnt/hdb1/system_drive_backup.img.gz
Here dd is making an image of the first harddrive, and piping it through the gzip compression program. The compressed image is then placed in a file on a separate drive. To reverse the process:
# gzip -dc /mnt/hdb1/system_drive_backup.img.gz | dd of=/dev/hda
Here, gzip is decompressing (the -d switch) the file, sending the results to stdout (the -c switch), which are piped to dd, and then written to /dev/hda.
To create an ISO image file from a CD. Insert the source cd and unmount it first if auto CD mount is enabled, this is to improve performance by preventing random access to the mounted filesystem.
dd if=/dev/cdrom of=/tmp/image.iso bs=2k
Note: The device might also be called /dev/dvd or /dev/scd0 (for a SCSI-drive).
To create an image file named floppy.img of a floppy disk in the drive whose block-device name is /dev/fd0 (as the first floppy device is on Linux), one may invoke dd in the following way:
dd if=/dev/fd0 of=floppy.img
Or to copy the img file back to a floppy:
dd if=floppy.img of=/dev/fd0 bs=16k
To create a file with name reallylargefile with the size of 1 GB, filled with random data, do this (1G = 1073741824, 1073741824 / 512 = 2097152 (the default block size is 512)):
dd if=/dev/random of=reallylargefile count=2097152
or faster but less cryptographicly secure
dd if=/dev/urandom of=reallylargefile count=2097152
To fill the file with NULL characters, use /dev/zero instead of /dev/random. This can be used to mount a filesystem if you have a FAT32 device, but need the flexibility of EXT2 – as follows to create a 10GB image:
dd if=/dev/zero of=my.new.virtual.harddisk bs=1k count=10MBmke2fs my.new.virtual.harddisk (reply yes when it says it’s not a block device)mkdir /mnt/virtualmount -o loop my.new.virtual.harddisk /mnt/virtual/"
- http://www.linuxquestions.org/questions/showthread.php?p=1848006#post1848006 (liste impressionnante d’utilisation de la commande ‘dd’)
- http://www.softpanorama.org/Tools/dd.shtml (idem liste impressionnante d’utilisation de la commande ‘dd’:
slave% nc -l -p 9000 | dd of=/dev/sda (Replace /dev/sda with actual drive on your slave machine)
This will listen at port 9000 and whatever it gets at port 9000 will hand over to dd command and dd will simply write that to on slave harddrive (sda) bit by bit. Here I am assuming dd and netcat (nc) are available either through floppy (/mnt/floppy/nc or through /mnt/sources/mystuff/nc). In case of floppy you need to mount floppy first using command:
mount /dev/fd0 /mnt/floppy
On Master machine:
Now Login on master linux box and run following command. (It is advisable that Master Linux box should be in calm state , i.e no major jobs running on the machine). This command below will read master disk bit by bit and throw this bit stream to netcat command which is connected to netcat command at port 9000 on <slave> box.
master% dd if=/dev/sda | nc 192.168.0.254 9000
That’s it. You may have to wait for long time depending upon network speed and size of your harddrive. Typically 36GB drive may take 50 minutes over 100Mbps link. Again rather than cloning complete drive we can clone only relevant partitions and MBR only. That will make cloning much faster like we saw in above section”)
- http://www.sweetnam.eu/index.php/Using_Netcat_for_Backup (“Providing you have netcat installed, on the destination machine execute the following command:
nc -l -p 6666 > filename.tar.bz2
This creates a listening socket on port 6666 with the filename specified above. We now need to setup the source machine to dump whatever directories or filesystems you want to send to the destination machine. So on the source you can issue the following command:
tar jlcvPpf - / > /dev/tcp/192.168.0.2/6666
In the example above 192.168.0.2 is the address of the destination address and the forward slash indicates that it is the entire filesystem to be backed up. You can easily change this for example to tar jlcvPpf – /var/www > /dev/tcp/192.168.0.2/6666 to backup the /var/www directory. If there is no /dev/tcp it is possible to use netcat on both ends
tar jlcPpf - / | nc 192.168.0.2 6666")
- http://m.nu/program/util/netcat/netcat.html (“Netcat is a tool that every security professional should be aware of and possibly have in their ‘security tool box’. In May/June of 2000, insecure.org conducted a survey of 1200 Nmap users from the Nmap-hackers mailing list to determine their favorite security tools. Netcat was the second most popular tool, not including Nmap. A quick search on securityportal (www.securityportal.com) found 166 matches of netcat. Most of the matches describe or use netcat in some way. Netcat is a utility that is able to write and read data across TCP and UDP network connections. If you are responsible for network or system security it essential that you understand the capabilities of netcat.Netcat should not be installed unless you have authority to do so. Never install any executable unless you can trust the providor. If possible review the source and compile it yourself. To be safe only use netcat in a test environment.”..Nous voilà avertis ! Et voila des commandes bien puissantes: “
Remote command prompt anyone?
On a Windows NT server issue the following command in the directory that contains netcat:
nc -l -p1234 -d -e cmd.exe –L
This –l puts netcat into listen mode, the -p1234 tells netcat to use port 1234, the –d allows netcat to run detached from the console, the –e cmd.exe tells netcat to execute the cmd.exe program when a connection is made, and the –L will restart Netcat with the same command line when the connection is terminated.
On the client system issue the following command:
nc destination 1234
This command causes netcat to connect to the server named destination on port 1234. Immediately you are given a console connection to the destination server. Be careful! To exit the remote console session type:
exit
You will be returned to your own console and will be able to reconnect to the destination server because netcat was started on the destination server with the –L option.
Sur GNU/Linux: (http://linuxfr.org/forums/26/16634.html)
Coté serveur:
——————
$> nc -u -l -p -e /bin/bash
Coté client:
—————
$> echo | nc -u <serveur IP> -w1
FTP & drive mapping blocked?
To receive a file named newfile on the destination system start netcat with the following command:
nc –l –p 1234 >newfile
On the source system send a file named origfile to the destination system with the following command:
nc destination 1234 <origfile
Issue a ^C on the source system and your done. Be sure to check the file to be sure it is the same size as the original.”)
Liens:
- http://fr.wikipedia.org/wiki/Disques_durs (“Le disque dur ou Hard Disk Drive est une mémoire de masse magnétique. Il a remplacé efficacement les tambours (aujourd’hui obsolètes) et les bandes, qui sont utilisées de nos jours uniquement pour l’archivage et la sauvegarde.”)
- http://www.cgsecurity.org/wiki/Articles
- PhotoRec versus Foremost: Récupération de fichiers (Juin 2006)
- PhotoRec: Récupération de fichiers dissimulés sur un cdrom (Juin 2007)
- http://fr.wikipedia.org/wiki/Sauvegarde (“En informatique, la sauvegarde (backup en anglais) est l‘opération qui consiste à dupliquer et à mettre en sécurité les données contenues dans un système informatique. Par extension et par emprunt à l’anglais (to save), ce mot s’applique à tout enregistrement d’information sur un support permanent (par ex. l’écriture sur disque dur d’informations en mémoire vive). Dans ce sens, il est synonyme de « stockage d’information ». Mais le sens premier se rapproche de celui d’archivage, de conservation.”…”La sauvegarde s’inscrit dans une démarche plus globale qui consiste à assurer la continuité d’activité d’un système informatique ou, en cas de défaillance, son redémarrage le plus vite possible. Cette démarche est souvent formalisée dans un document qui peut porter des noms divers, par exemple le “PRA” (plan de reprise d’activité) ou le “PS” (plan de secours), et qui fait appel soit à des automatismes (ex. donner l’alerte en cas de coupure de courant ou de perte d’accès à une unité de stockage) soit à des gestes manuels (ex. remplacer des bandes magnétiques défectueuses). La tendance est à l’automatisation, réputée plus sûre dans les situations d’urgence que les opérations manuelles.”
- http://fr.wikipedia.org/wiki/Master_boot_record (“Le Master Boot Record ou MBR Zone amorce“) est le nom donné au premier secteur adressable d’un disque dur (cylindre 0, tête 0 et secteur 1, ou secteur 0 en adressage logique) dans le cadre d’un partitionnement Intel. Sa taille est de 512 octets. Le MBR contient la table des partitions (les 4 partitions primaires) du disque dur. Il contient également une routine d’amorçage dont le but est de charger le système d’exploitation (ou le boot loader/chargeur d’amorçage s’il existe) présent sur la partition active.“)
- http://en.wikipedia.org/wiki/ISO_image (“An ISO image (.iso) is a disk image of an ISO 9660 file system. More loosely, it refers to any optical disc image, even a UDF image. As is typical for disc images, in addition to the data files that are contained in the ISO image, it also contains all the filesystem metadata, (boot code, structures, and attributes). All of this information is contained in a single file. These properties make it an attractive alternative to physical media for the distribution of software that requires this additional information as it is simple to retrieve over the Internet. Some of the common uses include the distribution of operating systems, such as Linux or BSD systems, and LiveCDs.”… dd (Unix) – Linux/UNIX built-in command doing low-level copy, including generating ISO image from a CD.
-
- ISO files can be mounted directly in Linux with the single command:
mount -o loop -t iso9660 filename.iso /mnt/iso (assuming the directory /mnt/iso exists).
- ISO images can also be made in Linux with simple commands:
mkisofs -J -o filename.iso directory name. Using -J preserves the Windows Joliet) extensions, meaning the data in the directory should be readable on Windows systems.”
- http://en.wikipedia.org/wiki/ISO_Master (“ISO Master is an open source application for Linux and BSD for creating and modifying ISO9660 files (ISO images).The first version was released as a working beta in August 2006 – a GTK-based GUI for the newly developed bkisofs (a library for reading and writing ISO9960 files in user space, including Joliet, RockRidge, and El Torito extensions)”
- http://en.wikipedia.org/wiki/Isolinux (“The SYSLINUX Project covers a suite of lightweight bootloaders, for starting up computers in the Linux operating system. It is the work of H. Peter Anvin, and consists of several separate systems:
- the original SYSLINUX, used for booting from FAT filesystems (usually floppy discs);
- ISOLINUX, used for booting from CD-ROM ISO 9660 filesystems;
- PXELINUX, used for booting from a network server using the Pre-boot eXecution Environment (PXE) system;
- EXTLINUX, used to boot from Linux ext2/ext3 filesystems;
- MEMDISK, used to boot older operating systems like MS-DOS from these media;
- two separate menu systems; and
- a development environment for additional modules.
SYSLINUX is not normally used for booting full Linux installations since Linux is not normally installed on FAT filesystems. Instead, it is often used for boot or rescue floppy discs, or other lightweight systems. ISOLINUX is generally used by Linux LiveCD and bootable install CDs.”)
- http://en.wikipedia.org/wiki/Netcat (“In computing, netcat is a network utility for reading from and writing to network connections on either TCP or UDP. It is designed in a thin and simple way, which makes it easy to incorporate in larger applications. Because of its versatility, netcat is also called the “TCP/IP Swiss Army knife“.Netcat was twice voted the fourth most useful network security tool in polls conducted by insecure.org on the nmap users mailing list[1]“)
- http://fr.wikipedia.org/wiki/Administrateur_syst%C3%A8me (“Un administrateur système est un informaticien responsable des serveurs d’une organisation (entreprise, association, administration). Ses tâches comprennent l’installation, le paramétrage, le maintien, la mise à jour, l’évolution, la sauvegarde, la restauration, la planification, la supervision, le conseil, le support et la veille technologique dans le périmètre technique des matériels et logiciels de type serveur, principalement les systèmes d’exploitation. Il a parfois la tâche de l’administration du réseau et/ou de l’administration des bases de données dans des organisations de petite taille. Il travaille au sein d’une DSI (Direction des Systèmes d’Information) ou d’une SSII (Société de Services en Ingénierie Informatique).
Il intervient auprès du DSI (Directeur des systèmes d’information), des DBA (Database Administrator, administrateur de bases de données), des administrateurs réseau, des webmasters et apparentés, des développeurs, des responsables bureautique (postes de travail) et enfin des usagers.“)
- http://en.wikipedia.org/wiki/System_administrator
- http://guides-info.org/linux/admin/outils_sauvegarde.php (“Sauvegarder… LA tache que beaucoup repoussent… jusqu’à ce qu’une fausse manip ou une panne matérielle le leur fasse, souvent chèrement, regretter. Heureusement pour les prévoyants, il est excessivement simple de sauvegarder un (des) dossier(s), une partition ou même un disque dur complet, qu’ils soient linux, windows ou autre“)
- http://www.inference.phy.cam.ac.uk/saw27/notes/backup-hard-disk-partitions.html (“It’s often useful to make an image of either an entire hard disk or an entire partition. One reason is to duplicate an installed system onto another PC (probably over a network connection); another is to make a backup of your complete hard disk including every aspect of the installed operating systems, which you can restore if you have to replace your hard disk or if you screw things up. Typically it’s useful to be able to transfer these images over the network to another machine, although you may want to save images onto a different partition or hard disk…There is a open source Linux program called partimage which is similar to Ghost, but I prefer to make backups using basic tools which I know will always be to hand, and in a pure format which I understand. My preferred solution in some situations is to use raw linux commands. The backup technique uses linux, but you don’t have to have linux installed on your computer to do this, and you can use this technique to backup partitions containing any filesystem“)
dd if=/dev/hda1 bs=1k conv=sync,noerror | gzip -c | ssh -c blowfish user@hostname "dd of=filename.gz bs=1k"
This instructs dd to read the contents of /dev/hda1 (the first partition). conv=sync,noerror tells dd that if it can’t read a block due to a read error, then it should at least write something to its output of the correct length. Even if your hard disk exhibits no errors, remember that dd will read every single block, including any blocks which the OS avoids using because it has marked them as bad. So don’t be too surprised if dd seems to struggle to read some blocks. (But see the next section for a better way of handling this situation).
bs=1k sets the block size to be 1k. I’m not quite sure what the optimal value is, but it needs to be no larger the the block size for the disk, otherwise a bad block may mask the contents of a good one. 1k is a safe bet.
In the above example the output of dd is piped through gzip to compress it. We then pipe the compressed data stream over an ssh connection to another linux machine (which may also be running Knoppix – see Knoppix notes below). If you wanted to write straight to a local file, you could either just add of=filename to the first dd command (to write an uncompressed image), or if you want to compress it, just redirect the output of the gzip to a filename.
Continuing with our explanation, the -c blowfish option to ssh selects blowfish encryption which is much faster (useful since we’re sending tons of data) than the default. Finally another dd command is invoked on the remote machine to read the data stream and write it to a file there. Alternatively you could pipe it through gunzip -c and write it straight to a partition on the remote machine instead of to a file.
Note that, as long as its not compressed, you should be able to mount a file containing a single partition’s image using a loopback device in linux. (With a little more jiggery-pokery to find the correct offset, you can also mount partitions within a whole-disk image; see here).
Restoring partitions
The restore procedure is fairly similar. For example, on the machine with the image on it, you might do something like:
dd if=filename.gz | ssh -c blowfish root@deadhost "gunzip -c | dd of=/dev/hda1 bs=1k". This assumes you have linux (e.g. Knoppix) running on the target machine with an ssh server running. See ‘Knoppix tips’, below. Note that you should not include conv=sync,noerror in the restore dd – doing so can, in certain situations, corrupt the data being written, since it instructs dd not to wait for more data to arrive from the network or filesystem if a whole block isn’t available.
The partition needs to already exist before you do this, and needs to be large enough to take all the data. If it’s too big, that doesn’t matter, you’ll just be wasting space at the end. You should then be able to grow the filesystem to fill that extra space. For ext2 filesystems, try using the ext2resize tool. You may also be able to persuade the partition editing tool parted to do this, since it can handle resizing most filesystems.
Alternative to ssh
If you’re on a trusted network, you might think it silly to go to trouble of encrypting and unencrypting all that data with ssh. Yet these days your machines are probably not configured to allow rsh connections. nc (a.k.a. netcat) is one option here: it just sends raw streams of data across a network. On the destination machine you could run nc -l -p 10001 > imagefile to start a process which will listen on TCP port 10001 and dump everything it receives from the first thing to connect to it to imagefile. Then on the source machine, pipe the output of dd (or gz, or whatever) to nc remote 10001 where remote is the name or IP address of the destination machine.
File-level backup techniques
Backing up entire partitions or disks is most useful when replacating systems across hard disks or when backing up partitions containing operating systems which are otherwise hard to fully backup. For more every-day backups of your data, a file-level method is more appropriate. Here are some handy tools for doing backups over a network to a remote machine in various clever ways:
- http://doc.ubuntu-fr.org/tutoriel/sauvegarder_home_avec_grsync ( Sauvegarder “/home” avec Grsync, l’interface graphique de rsync)
- http://doc.ubuntu-fr.org/applications/sauvegarde_d_une_partition (“Sauvegarder une partition permet de restaurer, si besoin, l’intégralité du système de fichiers qu’elle contient au moment de la sauvegarde. Ce principe est souvent utilisé pour sauvegarder l’intégralité d’un système d’exploitation ( appelé aussi backup ) . Cela ne semble pas adapté à la sauvegarde de documents personnels où l’on préférera backup-manager“)
![[en]](http://linuxfr.org/images/en.png)
![[fr]](http://linuxfr.org/images/fr.png)
