Резервное копирование с windows на linux

Задача:
Резервное копирование файлов с windows сервера на Linux сервер выступающий в роли хранилища.

Инструменты:
Для windows:
CopSSH_3.1.4
cwRsyncServer_4.0.3
Для Linux:
rsyncd
ssh
Цель:
Использовать пакет rsync и ssh для организации безопасного тунелля передачи данных в ethernet сети между двумя серверами с разной ОС.

Инструкция:
На windows сервере:
Устанавливаем пакет CopSSH на windows сервер (жмем далее-далее).
Останавливаем службу OpenSSHD, если запущена, иначе не получится поставить пакет cwRsyncServer (связано с тем, что создается эмулируемая линукс среда и при работе службы OpenSSHD, используется библиотека необходимая для установки cwRsyncServer) — вылезет ошибка.
Ставим cwRsyncServer (жмем далее-далее).
В службах меняем запуск службы RsyncServer с ручного запуска на автоматический.
Проверяем службы OpenSSHD и RsyncServer запустив их. Если запустились — все хорошо, останавливаем их.
Если вылезет ошибка 1069 «Вход невозможен», то поменяйте пароли пользователям CWRSYNC и COPSSH в «Администрирование-Управление данных компьютером-Пользователи». Такое случается, если вы пере устанавливаете эти программы.
Затем, введите новые пароли в свойствах служб OpenSSHD и RsyncServer во вкладке «Вход в систему». Теперь службы должны запуститься.
Создайте нового пользователя (у меня это RSBService) и добавьте его в группу Администраторы.
Добавим пользователя RSBService в пользователи ssh. Выполните: Пуск-Программы-CopSSH-Activate User.
Там выбираем или вписываем имя нашего пользователя и кликаем «Далее»
В passphrase можете ввести что угодно.
Отредактируем конфиг rsyncd.conf
Для этого выполните: Пуск-Программы-cwRsyncServer-rsyncd.conf.
Там все довольно тривиально и идентично настройкам для unix rsync.
Распишу только стандартные опции.

use chroot = false   # Использовать не только диск С
strict modes = false # Проверка файла с паролем. Отключена так как его не использую.
hosts allow = 192.18.0.34 # Откуда подключаться разрешено по rsync
log file = rsyncd.log  # Лог файл

# Module definitions
# Remember cygwin naming conventions: c:\work becomes /cygwin/c/work
#
[backup]               # Название модуля
path = /cygdrive/c/_BackUp     # Путь до папки, путь всегда начинается с cygdrive
read only = yes  # разрешено только чтение
transfer logging = yes  # логировать перемещение

Все. Теперь запустим службы OpenSSHD и RsyncServer.
На windows сервера работы закончили.

На Linux сервере:
Создадим ключ для беспорольной авторизации на windows сервере под пользователем RSBService:

ssh-keygen -t rsa -b 2048 -N ''  # Создаем ключ rsa длиной 2048 байт без парольной фразы.

Скопируем публичный ключ на windows сервер:

ssh-copy-id RSBService@ip-адрес сервера

У вас запросят пароль на пользователя RSBService. Введите его.
Теперь проверим беспарольный вход:

ssh RSBService@ip-адрес сервера

Установите демон rsyncd на Linux сервер.
С помощью следующей команды мы скопируем данные с windows сервера.

rsync -a --delete RSBService@ip-адрес::backup /назначение куда класть данные

PROFIT

Использованные материалы:
Сайт производителя утилиты.
Здесь описано как запустить cwRsync в качестве демона, если вы не найдете версию 4.0.3, так как в 4.0.5 нет отдельного серверного пакета.
Запуск в качестве службы
3 видео мануала, но с использованием windows в качестве сервера и клиента.
1я часть.
2я часть.
3я часть.

From the world of “why would anyone ever bother to do this?” comes a new approach to backing up your favourite Windows 10 computer! OK so why am I even doing this? I guess the answer to that question is really “just to see if I could.” It’s not the best answer but not everything needs a practical reason right?

So how does one go about using the awesome rsync utility to perform a Windows backup? Well first off you’ll need to enable the Windows Subsystem for Linux on your Windows 10 computer. There’s some documentation here from Microsoft but essentially you just need to enable the Feature and then go into the Windows Store, search for your favourite supported distribution, and click the Get button. Once the installation has finished you should launch bash and create a user account. Depending on the distribution you chose rsync may not be installed already so you may need to add it as well.

The actual backup process is really easy from that point. When you launch bash it will automatically mount your Windows drives under /mnt/, so for example the C:\ drive gets mounted as /mnt/c. Therefore if you wanted to backup the whole C:\ drive to an external drive you plugged in that showed up as E:\ in Windows you could simply run something like:

rsync -rv /mnt/c /mnt/e

Now while this should more or less work it will result in a load of errors as rsync doesn’t know how to or isn’t allowed to access a bunch of the special directories and files in Windows.

Also note that I’m only using the -r option above instead of the more “traditional” -a option. This is because of an incompatibility between the linux and Windows file permissions, so it isn’t able to correctly preserve them anyway. There are other options you may want to pass rsync as well, for instance the –delete option which will remove previously copied files on the destination drive if they are subsequently deleted from the source drive to keep them in sync.

How about an example?

For the purposes of an example backup let’s say I just wanted to copy the files under the USER (substitute your username here!) directory, excluding some of the nonsense directories I didn’t need. And let’s assume I want the files to end up on that E:\ drive as before. Oh and let’s also assume I want to run this backup on a schedule. Here’s how I would set it up:

1) Create a script to store our rsync commands

Technically you don’t need to do this but I find it easier to maintain this way. I created a file in C:\Users\USER\rsyncbackup\ called rsyncbackup.sh. The reason I placed it under my Windows user directory, and not within the linux user directory, was simply one of preference but this way the script itself will also be part of the backup. Finally I placed the following in it:

#!/bin/bash
rsync -rv --delete --exclude-from '/mnt/c/Users/USER/rsyncbackup/rsyncbackup.exclude' --progress /mnt/c/Users/USER /mnt/e

Be careful when saving files in Windows that you use the UNIX line endings and not the Windows line endings option, otherwise rsync may have issues with it. As you can see from above this rsync command is referencing a file call rsyncbackup.exclude. This is the file where I will place directories, files and wildcards for things I don’t want rsync to bother with. Here is what that file could look like:

*.lnk
*.url
AppData
Application Data
Cookies
desktop.ini
Documents/My Music
Documents/My Pictures
Documents/My Videos
IntelGraphicsProfiles
Local Settings
MicrosoftEdgeBackups
My Documents
NetHood
NTUSER.DAT*
ntuser*
PrintHood
Recent
Searches
SendTo
Start Menu
Templates

2) Try it out!

Now we can try to see if this will work. Open up a Windows command prompt (e.g. cmd.exe) and run the following:

bash -c '/mnt/c/Users/USER/rsyncbackup/rsyncbackup.sh'

If it all looks good then we can move onto the next step of scheduling it.

3) Schedule the backup

This is where you might be thinking

Well if we’re already using the Windows Subsystem for Linux why don’t we just schedule it with cron?

The answer is that from my testing it doesn’t appear as though Windows starts the Subsystem automatically after a restart. So if that never gets started, cron never runs and your backup never happens. Instead what we’ll do is simply use good old Windows Task Scheduler to get the job done.

Create a new task, schedule it accordingly and place the command you used in cmd.exe above as the Action. Feel free to run the task to test and confirm the results!

4) Other cool tricks

What’s neat about doing a backup this way is that you get additional access to all of the cool Linux commands and functionality as well. For example what if you wanted to do the same backup above, but weren’t sure what drive letter your external drive would show up under? Well an easy solution would be to place a uniquely named text file on your backup drive, something like “THIS_IS_THE_BACKUP_DRIVE.txt“, and use bash to find it for you!

#!/bin/bash
BACKUPDIR="${find /mnt -type d -name "c" -prune -o -type f -name "THIS_IS_THE_BACKUP_DRIVE.txt" -print 2>/dev/null}"
BACKUPDIR="${BACKUPDIR/THIS_IS_THE_BACKUP_DRIVE.txt/BACKUP}"
rsync -rv --delete --exclude-from '/mnt/c/Users/USER/rsyncbackup/rsyncbackup.exclude' --progress /mnt/c/Users/USER $BACKUPDIR

How does this work?

BACKUPDIR="${find /mnt -type d -name "c" -prune -o -type f -name "THIS_IS_THE_BACKUP_DRIVE.txt" -print 2>/dev/null}"

This first looks in all of the mounted Windows drives (everything under /mnt), excluding directories that have a “c” in the name (e.g. the C:\ drive) for a file called “THIS_IS_THE_BACKUP_DRIVE.txt” and stores the path to that file in the variable BACKUPDIR. I added 2>/dev/null in order to omit any errors, such as “Permission denied” that rsync may echo out. So for example if the backup drive was mounted under E:\ then my variable BACKUPDIR would contain the text “/mnt/e/THIS_IS_THE_BACKUP_DRIVE.txt”

BACKUPDIR="${BACKUPDIR/THIS_IS_THE_BACKUP_DRIVE.txt/BACKUP}"

Next it replaces the text “THIS_IS_THE_BACKUP_DRIVE.txt” in the variable with the text “BACKUP” because I want my backup files to end up in a folder on the drive called BACKUP (e.g. E:\BACKUP).

Finally it runs the normal rsync command put sets the destination directory to the contents of our variable above.

rsync -rv --delete --exclude-from '/mnt/c/Users/USER/rsyncbackup/rsyncbackup.exclude' --progress /mnt/c/Users/USER $BACKUPDIR

Neat right? OK but what if I want to back up my computer over the internet using ssh? Well we can certainly do that too! To make this easier I’m going to pre-setup some ssh keys so we don’t need to worry about providing a password. This will make scheduling the backup easier.

First off in bash on Windows you’ll want to generate some ssh keys:

ssh-keygen

Make sure to just hit enter when prompted for the password so that the keys aren’t password protected. Next you’ll want to copy those keys to your remote machine. I find the ssh-copy-id command especially easy for this:

ssh-copy-id -i ~/.ssh/id_rsa.pub x.x.x.x

Where x.x.x.x is the IP address of your remote machine. If that was successful you should be able to ssh to the remote machine without providing a password:

ssh x.x.x.x

Note that it’s perfectly fine for your linux username to not match your Windows username, you just need to be sure to explicitly provide the correct username as part of the rsync command below. If you were able to ssh then we just need to setup our backup script to use the remote machine as the destination:

#!/bin/bash
rsync -rv --delete --exclude-from '/mnt/c/Users/USER/rsyncbackup/rsyncbackup.exclude' -e "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" --progress /mnt/c/Users/USER USER@x.x.x.x:/home/USER/BACKUP

Running the script now will automatically connect to the remote machine at x.x.x.x using the linux user USER and run the backup, placing the files under the USER home directory in a folder called BACKUP. If your remote machine listens for ssh connections on a different port you can specify that too using the -p option right after the ssh portion, so something like:

rsync -rv --delete --exclude-from '/mnt/c/Users/USER/rsyncbackup/rsyncbackup.exclude' -e "ssh -p 1234 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" --progress /mnt/c/Users/USER USER@x.x.x.x:/home/USER/BACKUP

Anyway I hope you found this information interesting if not helpful. Maybe you too could setup your very own convoluted backup system!

This post, re-published here with permission, was originally published on my personal site here.