Rsync server windows server 2012


devops


2 Minutes

  1. Configuring sshd from Windows server

There’re few options available such as Cygwin, WSL, MSYS2(Git for Windows) and Win32-OpenSSH. Personally I ‘thought’:

  • Cygwin: installing the Cygwin was relatively easy, but configuring the sshd wasn’t. Not providing uninstaller was also minus.
  • WSL: Looked promising, but it didn’t support background daemon.

Quote: https://blogs.windows.com/buildingapps/2017/08/08/windows-subsystem-linux-windows-server/

What this isn’t — WSL is not a Linux server

Just as with WSL on Windows Client, you can run daemons and jobs like MySQL, PostgreSQL, sshd, etc., via an interactive shell, but you cannot currently use WSL to run persistent Linux services, daemons, jobs, etc. as background tasks.

  • Win32-OpenSSH: It’s official way to use the sshd on Windows(It even recently started including OpenSSH from the OS additional features). Detailed instruction, (un)install Powershell scripts are also provided here https://github.com/PowerShell/Win32-OpenSSH/.

So I installed Win32-OpenSSH following by https://github.com/PowerShell/Win32-OpenSSH/wiki/Install-Win32-OpenSSH page.

The server already installed the ‘Git for Windows(MSYS2)’ so that I made the bash.exe as a default shell of the sshd.

Name: DefaultShell

Value: C:\Program Files\Git\bin\bash.exe

Few more tweaks required,

  • I appended the client’s public key into the Windows Server(where rsync.exe server will be running)’s ~/.ssh/ authorized_keys
  • The server’s sshd_config file is located at ‘C:\ProgramData\ssh’ (unless you changed). ‘PubkeyAuthentication yes’ option is commented out by default. Let’s enable it.

So far, you should be able to ssh login to the Windows Server and see the MSYS2’s bash.exe shell as the result.

2. Installing rsync on the Server

I was struggling with this part at first because when I googled ‘rsync over ssh windows’ the results were outdated or insufficient. However, I found this and if you’re using MSYS2(Git for Windows), it’s extremely simple. Just grab the compiled binary from msys2 repo and paste it to the bin folder(C:\Program Files\Git\usr\bin).

Refs: https://serverfault.com/a/872557 & https://blog.tiger-workshop.com/add-rsync-to-git-bash-for-windows/

Then I was able to successfully ‘rsync’ing by:

rsync …… -e ssh -vvv /c/somedir/folderA user@pcname.xxx.xxx:/c/detination_folder

Last bonus:

3. Working it through Jenkins

Additional chores remained if you want it to work through Jenkins because Jenkins service process is running under the SYSTEM account (if you didn’t modify). For example, if someone tried to launch a job does Jenkins rsync over SSH, then they’ll see the following error:

[rsync test] $ C:\…\Git\bin\bash.exe -xe C:\Windows\TEMP\jenkins500000000000000003.sh

+ whoami

SYSTEM

+ rsync … -e ‘ssh -vvv -i /c/Users/xxxxx/.ssh/id_rsa’ /c/….

OpenSSH_7.5p1, OpenSSL 1.0.2k 16 Jan 2017

debug1: Reading configuration data /etc/ssh/ssh_config

: : :

: : :

debug3: hostkeys_foreach: reading file “/etc/ssh/ssh_known_hosts”

debug3: hostkeys_foreach: reading file “/etc/ssh/ssh_known_hosts”

debug1: read_passphrase: can’t open /dev/tty: No such device or address

Host key verification failed.

rsync: connection unexpectedly closed (0 bytes received so far) [sender]

rsync error: error in rsync protocol data stream (code 12) at io.c(226) [sender=3.1.3]

Build step ‘Execute shell’ marked build as failure

Finished: FAILURE

I believe this problem can be fixed by making the Jenkins to run under normal user account, but I approached bit differently by

  • Launching cmd.exe as the SYSTEM privilege using Sysinternals’ PsExec.exe

  • From that console, run bash.exe again and just login to the server once to create known_hosts for %SYSTEMPROFILE%.

Published by Heejune


View all posts by Heejune

Published


devops


2 Minutes

  1. Configuring sshd from Windows server

There’re few options available such as Cygwin, WSL, MSYS2(Git for Windows) and Win32-OpenSSH. Personally I ‘thought’:

  • Cygwin: installing the Cygwin was relatively easy, but configuring the sshd wasn’t. Not providing uninstaller was also minus.
  • WSL: Looked promising, but it didn’t support background daemon.

Quote: https://blogs.windows.com/buildingapps/2017/08/08/windows-subsystem-linux-windows-server/

What this isn’t — WSL is not a Linux server

Just as with WSL on Windows Client, you can run daemons and jobs like MySQL, PostgreSQL, sshd, etc., via an interactive shell, but you cannot currently use WSL to run persistent Linux services, daemons, jobs, etc. as background tasks.

  • Win32-OpenSSH: It’s official way to use the sshd on Windows(It even recently started including OpenSSH from the OS additional features). Detailed instruction, (un)install Powershell scripts are also provided here https://github.com/PowerShell/Win32-OpenSSH/.

So I installed Win32-OpenSSH following by https://github.com/PowerShell/Win32-OpenSSH/wiki/Install-Win32-OpenSSH page.

The server already installed the ‘Git for Windows(MSYS2)’ so that I made the bash.exe as a default shell of the sshd.

Name: DefaultShell

Value: C:\Program Files\Git\bin\bash.exe

Few more tweaks required,

  • I appended the client’s public key into the Windows Server(where rsync.exe server will be running)’s ~/.ssh/ authorized_keys
  • The server’s sshd_config file is located at ‘C:\ProgramData\ssh’ (unless you changed). ‘PubkeyAuthentication yes’ option is commented out by default. Let’s enable it.

So far, you should be able to ssh login to the Windows Server and see the MSYS2’s bash.exe shell as the result.

2. Installing rsync on the Server

I was struggling with this part at first because when I googled ‘rsync over ssh windows’ the results were outdated or insufficient. However, I found this and if you’re using MSYS2(Git for Windows), it’s extremely simple. Just grab the compiled binary from msys2 repo and paste it to the bin folder(C:\Program Files\Git\usr\bin).

Refs: https://serverfault.com/a/872557 & https://blog.tiger-workshop.com/add-rsync-to-git-bash-for-windows/

Then I was able to successfully ‘rsync’ing by:

rsync …… -e ssh -vvv /c/somedir/folderA user@pcname.xxx.xxx:/c/detination_folder

Last bonus:

3. Working it through Jenkins

Additional chores remained if you want it to work through Jenkins because Jenkins service process is running under the SYSTEM account (if you didn’t modify). For example, if someone tried to launch a job does Jenkins rsync over SSH, then they’ll see the following error:

[rsync test] $ C:\…\Git\bin\bash.exe -xe C:\Windows\TEMP\jenkins500000000000000003.sh

+ whoami

SYSTEM

+ rsync … -e ‘ssh -vvv -i /c/Users/xxxxx/.ssh/id_rsa’ /c/….

OpenSSH_7.5p1, OpenSSL 1.0.2k 16 Jan 2017

debug1: Reading configuration data /etc/ssh/ssh_config

: : :

: : :

debug3: hostkeys_foreach: reading file “/etc/ssh/ssh_known_hosts”

debug3: hostkeys_foreach: reading file “/etc/ssh/ssh_known_hosts”

debug1: read_passphrase: can’t open /dev/tty: No such device or address

Host key verification failed.

rsync: connection unexpectedly closed (0 bytes received so far) [sender]

rsync error: error in rsync protocol data stream (code 12) at io.c(226) [sender=3.1.3]

Build step ‘Execute shell’ marked build as failure

Finished: FAILURE

I believe this problem can be fixed by making the Jenkins to run under normal user account, but I approached bit differently by

  • Launching cmd.exe as the SYSTEM privilege using Sysinternals’ PsExec.exe

  • From that console, run bash.exe again and just login to the server once to create known_hosts for %SYSTEMPROFILE%.

На чтение2 мин

Опубликовано

Обновлено

Rsync — это удобный инструмент для синхронизации файлов между удаленными системами. Он позволяет эффективно передавать только измененные части файлов, что существенно уменьшает объем передаваемых данных и ускоряет процесс синхронизации.

Запустить Rsync сервер на операционной системе Windows Server 2012 можно с помощью Cygwin – набора утилит, позволяющих запускать ПО, разработанное для Unix-подобных систем, в среде Windows. Установка Cygwin и настройка Rsync требуют некоторых навыков работы с командной строкой, но этот подробный гайд поможет вам разобраться в процессе.

Важно: перед установкой Cygwin на сервер рекомендуется выполнить резервное копирование данных. Ошибки в процессе установки могут привести к непредвиденным последствиям и потере информации.

Шаг 1. Скачайте и запустите установщик Cygwin с официального сайта проекта. Установите Cygwin в каталог по умолчанию.

Для установки Rsync на Windows Server 2012 следуйте инструкции ниже:

1. Скачайте последнюю версию Rsync для Windows Server 2012 с официального сайта или из надежного источника.

2. Разархивируйте скачанный файл в удобную для вас директорию, например, «C:

sync».

3. Перейдите в директорию, в которую вы разархивировали Rsync, например, «C:

sync».

4. Скопируйте файлы «rsync.exe», «cygiconv-2.dll» и «cygwin1.dll» в директорию «C:\Windows\System32».

5. Создайте директорию для хранения Rsync-сервера, например, «C:\RsyncServer».

6. Создайте новый текстовый файл в редакторе кода и сохраните его с именем «rsyncd.conf».

7. Откройте файл «rsyncd.conf» и добавьте следующие строки:

uid = 0gid = 0use chroot = falsemax connections = 10timeout = 300[Открытый доступ]path = /path/to/your/foldercomment = Открытый доступ к файламread only = falselist = trueuid = 0gid = 0auth users = usernamesecrets file = /path/to/your/secrets.file

8. Замените «/path/to/your/folder» на реальный путь к вашей директории, в которой вы хотите хранить данные.

9. Замените «username» на имя пользователя, которому нужно предоставить доступ к Rsync-серверу.

10. Сохраните файл «rsyncd.conf» и закройте его.

11. Создайте новый текстовый файл и сохраните его с именем «secrets.file».

12. Откройте файл «secrets.file» и добавьте следующую строку:

username:password

13. Замените «username» на имя пользователя, которое вы указали в файле «rsyncd.conf», и «password» на пароль этого пользователя.

14. Сохраните файл «secrets.file» и закройте его.

Теперь у вас установлен Rsync на Windows Server 2012 и сконфигурирован Rsync-сервер для открытого доступа к файлам.

Подготовка сервера для Rsync

Перед установкой и настройкой Rsync сервера на Windows Server 2012, необходимо выполнить несколько шагов подготовки:

1. Установка Windows Server 2012

Убедитесь, что на сервере установлена операционная система Windows Server 2012. Если еще не установлена, установите ее в соответствии с инструкциями от Microsoft.

2. Установка Git Bash

Для работы с Rsync на Windows необходимо установить Git Bash — эмулятор терминала, который предоставляет командную оболочку Git вместе с утилитой Rsync. Получить инсталлятор Git Bash можно с официального сайта Git.

3. Загрузка Rsync

После установки Git Bash, пройдите на официальный сайт Rsync и загрузите подходящую версию Rsync для Windows. Рекомендуется выбрать Precompiled Binaries, соответствующие архитектуре вашего сервера.

4. Распаковка архива Rsync

После загрузки архива Rsync, распакуйте его в удобное для вас место на сервере.

5. Добавление пути Rsync в переменную среды PATH

Для того чтобы Rsync был доступен из любого места в системе, необходимо добавить путь к исполняемому файлу Rsync в переменную среды PATH. Откройте Панель управления -> Система -> Дополнительные параметры системы -> Переменные среды и измените переменную PATH, добавив путь к исполняемому файлу Rsync.

После завершения этих шагов, сервер будет готов к установке и настройке Rsync.

Windows, WHY ARE YOU ALWAYS SO DIFFICULT! Gosh. Damn you are. This time. i wanted to do schedule a backup from my windows server 2012r2 to my linux backup drive. Its as simple as that (while i though it was at least). Google doesn’t help with so many rubbish online. Hence, here is a guide that will help us out (me included)

Environment

Enterprise server (Windows 2012 R2)

This is a windows environment server 2012 R2 where our data is

Backup server (Debian Linux)

This is my backup server where i would like to rsync over.

Installation

On Windows server 2012 r2

On Linux

  • Install openssh-server and rsync.
  • Provide data to a partition (eg.: /backup/).
  • Place the public key in /home/USER/.ssh/ and rename the file to authorized_keys. (assuming its root)

On Windows

  • Test the connection without a password with the following command:
  • Test Rsync:
rsync -v -rlt -z --delete "/myfiles/" "USER@BackupServerIP:/backups/"
  • where cygdrive is the directory on C:\cygdrive so the above  C:\cygdrive\myfiles
  • To Test Other port
rsync  -e "ssh -p 14000" -arv "--exclude=.svn/" /myfiles USER@BackupServerIP:/backups/
  • Create a bat file with the rsync command and place it in C:\cwRsync\bin.
  • Schedule execution every day at 0:30 (half past midnight).

Helpful Resources

  • http://stackoverflow.com/questions/34147565/rsync-uid-gid-impossible-to-set-cases-cause-future-hard-link-failure-how-to
  • http://www.smellems.com/tiki-read_article.php?articleId=14

Download CWRSYNCSERVER software installation (this is file synchronization software under Windows)

First, (1)File client serverInstall the software, find rsyncd.conf under the installation location, modify the configuration file (Note that the format is: / cygdrive / c / ztb indicate: / cygdrive / c / ztb represents the ztb directory under the C disc

use chroot = false

strict modes = false

hosts allow =*

log file = rsyncd.log

pid file = rsyncd.pid

port = 873

uid = 0

gid = 0

max connections = 10

lock file = rsyncd.lock

 

[backup]

path = /cygdrive/c/ztb/

transfer logging = yes

ignore errors

list = no

secrets file =/cygdrive/c/Rsync/rsyncd.secrets

hosts allow = 192.168.0.2/255.255.255.0

hosts deny = *

(2) Establish rsyncd.secrets

Enter directly:

Second, inBackup file server(Database Server) Install the software, create new password file rsyncd.secrets, input format is:

Third, inBackup file serverManual synchronous test (recommended to test first, if an error can be seen directly)

Enter the software installation directory C: \ Program Files (x86) \ ICW \ bin; enter the command

rsync.exe -avzPu--progress --delete rsync://192.168.0.2:873/backup /cygdrive/c/Appbackup--password-file=/cygdrive/c/Rsync/rsyncd.secrets

Fourth, automatic synchronization test

inBackup file server(Database Server) New file synchronization script rsync.cmd, the content is as follows:

@echo off

set rsynclog="C:\Rsync\rsync.log"

set nowtime=te:~0,10% %time%

echo %nowtime% >> %rsynclog%

cd /d

C:\Program Files (x86)\ICW\bin

rsync.exe  -avzPu  --progress  --delete  rsync://192.168.0.2:873/backup/cygdrive/c/Appbackup  --password-file=/cygdrive/c/Rsync/rsyncd.secrets

>> %rsynclog%

@echo -------------------------------------------------- >>%rsynclog%

@echo. >> %rsynclog%

V. Join the system mission plan

Click the task plan program, enter the right click [Create Task]

Creating timing tasks and databases can be found, you can see another blog post

https://www.cnblogs.com/marshu/p/9842697.html

Sixth, encounter problems and solutionsThis step is referred to.https://www.cnblogs.com/janas/p/3321087.html

Error 1: Rsync: Read error: Connection Reset by Peer (104) 
rsync error: error in rsync protocol data stream (code 12) at io.c(794) [receive r=3.0.2] 
 Solve: Very large might be to open the RSYNC service server. Open service.
Error 2: @ERROR: ChDIR FAILED 
rsync error: error starting client-server protocol (code 5) at main.c(1495) [receiver=3.0.2] 
 SOLUTION: The server-side synchronization directory has no permissions, and the CWRSync default user is SVCWRSYNC. Add user SVCWRSync permissions for synchronous directories.
Error 3: @error: failed to open lock file 
rsync error: error starting client-server protocol (code 5) at main.c(1495) [receiver=3.0.2] 
 Solve: The configuration file rsync.conf adds lock file = rsyncd.lock to resolve.
Error 4: rsync: could not open password file "/cygwin/e/setting/rsync/rsync_db.pwd": No Such File or Directory (2)
 Solve: The directory of the password file must exist, and use the POSIX style to write: /cygdrive/e/setting/rsync/rsync_db.pwd
Error 5: @ERROR: Auth failed on module data_backup rsync error: Error Starting Client-Server Protocol (Code 5) at main.c (1506) [receivediver = 3.0.7]
 Solve: The password is incorrect, enter the correct password. If the username and password are correct, it is possible to be 600 permissions for the account password file of the remote RSYNC server.
Error 6: Password File Must Not Be Other-Accessible
 Solve: This is because rsyncd.pwd rsyncd.sec is not right, should be set to 600. Such as: chmod 600 rsyncd.pwd, Windows should change the owner of the password file to the user running.
Error 7: @Error: Invalid Uid Nobody. Rsync Error: Error Starting Client-Server Protocol (CODE 5) at main.c (1506) [received = 3.0.7]
 Solve: Add the following two lines in the rsyncd.conf file.
uid = 0
gid = 0 
Question 8: @error: chroot failed
rsync error: error starting client-server protocol (code 5) at main.c(1522) [receiver=3.0.3]
 Cause: The directory of the server does not exist or no permissions. Creating a directory and correcting permissions to solve the problem.
Question 9: @error: unknown module 'Tee_nonexists'
rsync error: error starting client-server protocol (code 5) at main.c(1522) [receiver=3.0.3]
 Cause: The server does not exist the specified module. Provide the correct module name or modify the module you want to solve the problem in the server.
Question 10: rsync: failed to connect to 218.107.243.2: no route to host (113)
rsync error: error in socket IO (code 10) at clientserver.c(104) [receiver=2.6.9]
 Cause: The other party does not boot, the firewall is blocked, and there is a firewall block on the network. Turn off the firewall, in fact, open the 873 or specified RSYNC port of the TCP UDP.
Question 11: rsync error: Error Starting Client-Server Protocol (Code 5) at main.c (1524) [receivediver = 3.0.7]
 Cause: /etc/rsyncd.conf configuration file content has an error. Please correctly check the configuration file.
Question 12: rsync: chown "" failed: invalid argument (22)
 Cause: Permissions cannot be copied. Remove the parameters of the synchronous permissions. (This situation is more than Linux time to Windows)
Question 13: @error: daemon security Issue - Contact Admin
rsync error: error starting client-server protocol (code 5) at main.c(1530) [sender=3.0.6]
 Cause: There is a soft connection file in the synchronized directory, which requires the server-side /etc/rsyncd.conf to open Use Chroot = YES. Brush soft connection file.
Question 14: Rsync: Read error: Connection Reset by Peer (104)
rsync error: error in rsync protocol data stream (code 12) at io.c(794) [receiver=3.0.2]
 Solve: Very large might be to open the RSYNC service on the server side, turn on the service.

I have a dream to run back, and it is natural to be lucky!

This articles is published 697 days ago and last updated 665 days ago, some information may be out of date.

rsync-windows.jpg

rsync-windows.jpg

img from unsplash

These are the basic steps to set up and use an rsync tool for synchronization on Windows.


Setup RSYNC Server on Windows

i. Manual Setup

  1. Install RSYNC on the server machines.
    Install Rsync software like a linux through Chocolatey
choco install rsync -y
  1. On the server, create a folder to share
mkdir -p C:\rsync_share

2023-06-08_164244.png

2023-06-08_164244.png

e.g. C:\rsync_share. as rsync share folder.

  1. On the server,Create the C:\rsync\rsyncd.conf file with the following contents:
[rsync_share] 
path = /cygdrive/c/rsync_share
use chroot = no
ignore errors  
read only = no
list = yes

This configures the share name as «rsync_share» pointing to the C:\rsync_share folder.

Use cygdrive path prefix as /cygdrive/c/rsync_share

  1. On the server, open PowerShell and run the rsync daemon:
rsync --daemon --config=C:\rsync\rsyncd.conf

This will run the rsync daemon using the config file C:\rsync\rsyncd.conf.

  1. You may need to configure firewall rules on the server to allow incoming TCP connections to port 873

2023-06-08_151454.png

2023-06-08_151454.png

(The rsync daemon default port).

  1. Rsync service is set to run automatically on startup on the server via «Task Scheduler«.

2023-06-08_151641.png

2023-06-08_151641.png

2023-06-08_151656.png

2023-06-08_151656.png

ii. Automatic via Script

Here are Windows scripts to automatically install rsync, configure it, and open the firewall port:

  1. Update Policy to allow execute self PowerShell script.
Set-ExecutionPolicy RemoteSigned

Execution Policy Change
The execution policy helps protect you from scripts that you do not trust. Changing the execution policy might expose
you to the security risks described in the about_Execution_Policies help topic at
https:/go.microsoft.com/fwlink/?LinkID=135170. Do you want to change the execution policy?
[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is «N»): y

  1. Create Windows PowerShell Script

Save all below content to a PowerShell script as names «rsync-config.ps1»

# Create rsync config and share folders
New-Item -Path C:\rsync -ItemType Directory
New-Item -Path C:\rsync_share -ItemType Directory

# Create rsync.conf file
@"
[rsync_share]
path = /cygdrive/c/rsync_share
ignore errors
read only = no
list = yes
auth users = rsync
secrets file = /cygdrive/c/rsync/rsync.passwd
"@ | Add-Content -Path C:\rsync\rsyncd.conf

# Open port 873 for rsync
New-NetFirewallRule `
    -Name "Rsync Daemon (873)" `
    -DisplayName "Rsync Daemon (873)" `
    -Protocol TCP `
    -LocalPort 873

# Create scheduled task to run rsync daemon
$Trigger = New-ScheduledTaskTrigger -AtStartup
$Action = New-ScheduledTaskAction `
    -Execute "Powershell.exe" `
    -Argument "rsync --daemon --config=C:\rsync\rsyncd.conf"
Register-ScheduledTask `
    -TaskName "Rsync Daemon" `
    -Trigger $Trigger `
    -Action $Action `
    -RunLevel Highest `
    -User System
Parameter Description Path
rsync Configuration folder for rsync C:rsync\
rsync_share Shared folder for rsync C:rsync_share\
/cygdrive/c/rsync_share Cygwin path prefix up to the shared folder location C:rsync_share\
auth users = rsync Specifies that only the «rsync» user can connect
secrets file = /cygdrive/c/rsync/rsync.passwd Specifies the location of the password file that will be used for authenticating rsync connections. C:rsyncrsync.passwd

The format of the rsync password file(C:rsyncrsync.passwd) is as follows:

username:password
  • username: This is the username for the rsync account.
  • password: This is the password for the rsync account.

Each line in the file represents a single rsync account. Create multiple accounts by adding additional lines to the file.

  1. Execute the PowerShell Script

2023-06-08_151758.png

2023-06-08_151758.png
run on Windows server to automatically setup rsync daemon, create share folder names «rsync_share» and configure the firewall. moreover, the rsync service startup on reboot.


Setup RSYNC Client on Windows

Install RSYNC on the client machines.
Install Rsync software like a linux through Chocolatey

choco install rsync -y

Operating

i. Manual

On the client PC, create a folder to store rsync data

mkdir -p C:\rsync_client
Directory: C:\

Mode LastWriteTime Length Name

—- ————- —— —-

d—— 7/11/2023 12:46 PM rsync_client

For example:

To sync files from client to server (PUSH):

rsync -avz /cygdrive/c/rsync_client  rsync://server_ip_address/rsync_share

To sync files from server to client (PULL):

rsync -avz rsync://server_ip_address/rsync_share /cygdrive/c/rsync_client 

To sync files from server to client with --progress flag on Linux

rsync -avz --progress --update --chmod=ugo=rwX --delete rsync://server_ip_address/rsync_share ./rsync_client

2023-06-08_163246.png

2023-06-08_163246.png
Option Description
-a Enables archive mode
-v Enables verbose output
-z Enables compression
--progress Displays progress during the transfer
--update Skips files that are newer on the destination
--chmod=ugo=rwX Sets permissions of synced files
--delete Removes files on the destination that do not exist on the source
rsync://server_ip_address/rsync_share Specifies the remote directory as rsync_share
./ClientFolder Specifies the current directory on the local machine

i. Automatic via Script

Update Policy to allow execute self PowerShell script.

Set-ExecutionPolicy RemoteSigned
  1. Create Windows PowerShell Script

Save all below content to a PowerShell script as names «rsync-client.ps1»

# Set variables
$RsyncPullFolder = "C:\rsync_pull"
$RsyncConfigFolder = "C:\rsync"
$Source = "/cygdrive/c/rsync_pull"
$Destination = "rsync_server_ip::share"
$Username = "rsync"
$Password = "P@ssw0rd123!"
$TaskName = "Rsync Pull"
$TriggerTime = "08:00"


# Check if rsync_pull and rsync folders exist, and create them if they don't
if (!(Test-Path $RsyncPullFolder)) {
    New-Item -ItemType Directory -Path $RsyncPullFolder
}
if (!(Test-Path $RsyncConfigFolder)) {
    New-Item -ItemType Directory -Path $RsyncConfigFolder
}
New-Item -Path "$RsyncConfigFolder" -Name "rsync.passwd" -ItemType "File"
Add-Content -Path "$RsyncConfigFolder\rsync.passwd" -Value "$Password"


# Build rsync command
$RsyncCommand = "rsync -avz --progress --update --chmod=ugo=rwX --delete --password-file=/cygdrive/c/rsync/rsync.passwd $Username@$Destination $Source"

# Create scheduled task to run rsync command
$Trigger = New-ScheduledTaskTrigger -Daily -At $TriggerTime
$Action = New-ScheduledTaskAction -Execute "Powershell.exe" -Argument "$RsyncCommand"
Register-ScheduledTask -TaskName $TaskName -Trigger $Trigger -Action $Action -RunLevel Highest -User "SYSTEM"
Variable Description
$RsyncPullFolder Specifies the download directory on the local machine
$RsyncConfigFolder Specifies the config file storage directory on the local machine
$Source Specifies the source directory on the local machine
$Destination Specifies the destination directory on the remote server
$Username Specifies the username of the rsync user on the remote server
$Password Specifies the password for the rsync user on the remote server
$TaskName Specifies the name of the scheduled task
$TriggerTime Specifies the time when the scheduled task will run

This script runs on a Windows client and automatically sets up an rsync schedule, creates an rsync folder named ‘rsync_pull’, and configures the username and password for authorization.


Conclusion

rsync can be a reliable and efficient solution for file synchronization on Windows, with its ability to preserve file attributes, compress data during transfer, and show verbose output.

To use rsync on Windows, you can install a version of rsync compatible with Windows, configure your system’s firewall and network settings, and create a configuration file to specify the files and directories to be synced.


Reference

  • How-to Run a PowerShell Script
  • The cygdrive path prefix

Related

  • RSYNC USAGE
Windowsrsync

На чтение2 мин

Опубликовано

Обновлено

Rsync — это удобный инструмент для синхронизации файлов между удаленными системами. Он позволяет эффективно передавать только измененные части файлов, что существенно уменьшает объем передаваемых данных и ускоряет процесс синхронизации.

Запустить Rsync сервер на операционной системе Windows Server 2012 можно с помощью Cygwin – набора утилит, позволяющих запускать ПО, разработанное для Unix-подобных систем, в среде Windows. Установка Cygwin и настройка Rsync требуют некоторых навыков работы с командной строкой, но этот подробный гайд поможет вам разобраться в процессе.

Важно: перед установкой Cygwin на сервер рекомендуется выполнить резервное копирование данных. Ошибки в процессе установки могут привести к непредвиденным последствиям и потере информации.

Шаг 1. Скачайте и запустите установщик Cygwin с официального сайта проекта. Установите Cygwin в каталог по умолчанию.

Установка Rsync на Windows Server 2012

Для установки Rsync на Windows Server 2012 следуйте инструкции ниже:

1. Скачайте последнюю версию Rsync для Windows Server 2012 с официального сайта или из надежного источника.

2. Разархивируйте скачанный файл в удобную для вас директорию, например, «C:

sync».

3. Перейдите в директорию, в которую вы разархивировали Rsync, например, «C:

sync».

4. Скопируйте файлы «rsync.exe», «cygiconv-2.dll» и «cygwin1.dll» в директорию «C:\Windows\System32».

5. Создайте директорию для хранения Rsync-сервера, например, «C:\RsyncServer».

6. Создайте новый текстовый файл в редакторе кода и сохраните его с именем «rsyncd.conf».

7. Откройте файл «rsyncd.conf» и добавьте следующие строки:

uid = 0gid = 0use chroot = falsemax connections = 10timeout = 300[Открытый доступ]path = /path/to/your/foldercomment = Открытый доступ к файламread only = falselist = trueuid = 0gid = 0auth users = usernamesecrets file = /path/to/your/secrets.file

8. Замените «/path/to/your/folder» на реальный путь к вашей директории, в которой вы хотите хранить данные.

9. Замените «username» на имя пользователя, которому нужно предоставить доступ к Rsync-серверу.

10. Сохраните файл «rsyncd.conf» и закройте его.

11. Создайте новый текстовый файл и сохраните его с именем «secrets.file».

12. Откройте файл «secrets.file» и добавьте следующую строку:

username:password

13. Замените «username» на имя пользователя, которое вы указали в файле «rsyncd.conf», и «password» на пароль этого пользователя.

14. Сохраните файл «secrets.file» и закройте его.

Теперь у вас установлен Rsync на Windows Server 2012 и сконфигурирован Rsync-сервер для открытого доступа к файлам.

Подготовка сервера для Rsync

Перед установкой и настройкой Rsync сервера на Windows Server 2012, необходимо выполнить несколько шагов подготовки:

1. Установка Windows Server 2012

Убедитесь, что на сервере установлена операционная система Windows Server 2012. Если еще не установлена, установите ее в соответствии с инструкциями от Microsoft.

2. Установка Git Bash

Для работы с Rsync на Windows необходимо установить Git Bash — эмулятор терминала, который предоставляет командную оболочку Git вместе с утилитой Rsync. Получить инсталлятор Git Bash можно с официального сайта Git.

3. Загрузка Rsync

После установки Git Bash, пройдите на официальный сайт Rsync и загрузите подходящую версию Rsync для Windows. Рекомендуется выбрать Precompiled Binaries, соответствующие архитектуре вашего сервера.

4. Распаковка архива Rsync

После загрузки архива Rsync, распакуйте его в удобное для вас место на сервере.

5. Добавление пути Rsync в переменную среды PATH

Для того чтобы Rsync был доступен из любого места в системе, необходимо добавить путь к исполняемому файлу Rsync в переменную среды PATH. Откройте Панель управления -> Система -> Дополнительные параметры системы -> Переменные среды и измените переменную PATH, добавив путь к исполняемому файлу Rsync.

После завершения этих шагов, сервер будет готов к установке и настройке Rsync.

Windows, WHY ARE YOU ALWAYS SO DIFFICULT! Gosh. Damn you are. This time. i wanted to do schedule a backup from my windows server 2012r2 to my linux backup drive. Its as simple as that (while i though it was at least). Google doesn’t help with so many rubbish online. Hence, here is a guide that will help us out (me included)

Environment

Enterprise server (Windows 2012 R2)

This is a windows environment server 2012 R2 where our data is

Backup server (Debian Linux)

This is my backup server where i would like to rsync over.

Installation

On Windows server 2012 r2

On Linux

  • Install openssh-server and rsync.
  • Provide data to a partition (eg.: /backup/).
  • Place the public key in /home/USER/.ssh/ and rename the file to authorized_keys. (assuming its root)

On Windows

  • Test the connection without a password with the following command:
  • Test Rsync:
rsync -v -rlt -z --delete "/myfiles/" "USER@BackupServerIP:/backups/"
  • where cygdrive is the directory on C:\cygdrive so the above  C:\cygdrive\myfiles
  • To Test Other port
rsync  -e "ssh -p 14000" -arv "--exclude=.svn/" /myfiles USER@BackupServerIP:/backups/
  • Create a bat file with the rsync command and place it in C:\cwRsync\bin.
  • Schedule execution every day at 0:30 (half past midnight).

Helpful Resources

  • http://stackoverflow.com/questions/34147565/rsync-uid-gid-impossible-to-set-cases-cause-future-hard-link-failure-how-to
  • http://www.smellems.com/tiki-read_article.php?articleId=14

This articles is published 705 days ago and last updated 673 days ago, some information may be out of date.

rsync-windows.jpg

rsync-windows.jpg

img from unsplash

These are the basic steps to set up and use an rsync tool for synchronization on Windows.


Setup RSYNC Server on Windows

i. Manual Setup

  1. Install RSYNC on the server machines.
    Install Rsync software like a linux through Chocolatey
choco install rsync -y
  1. On the server, create a folder to share
mkdir -p C:\rsync_share
2023-06-08_164244.png

2023-06-08_164244.png

e.g. C:\rsync_share. as rsync share folder.

  1. On the server,Create the C:\rsync\rsyncd.conf file with the following contents:
[rsync_share] 
path = /cygdrive/c/rsync_share
use chroot = no
ignore errors  
read only = no
list = yes

This configures the share name as «rsync_share» pointing to the C:\rsync_share folder.

Use cygdrive path prefix as /cygdrive/c/rsync_share

  1. On the server, open PowerShell and run the rsync daemon:
rsync --daemon --config=C:\rsync\rsyncd.conf

This will run the rsync daemon using the config file C:\rsync\rsyncd.conf.

  1. You may need to configure firewall rules on the server to allow incoming TCP connections to port 873
2023-06-08_151454.png

2023-06-08_151454.png

(The rsync daemon default port).

  1. Rsync service is set to run automatically on startup on the server via «Task Scheduler«.
2023-06-08_151641.png

2023-06-08_151641.png
2023-06-08_151656.png

2023-06-08_151656.png

ii. Automatic via Script

Here are Windows scripts to automatically install rsync, configure it, and open the firewall port:

  1. Update Policy to allow execute self PowerShell script.
Set-ExecutionPolicy RemoteSigned

Execution Policy Change
The execution policy helps protect you from scripts that you do not trust. Changing the execution policy might expose
you to the security risks described in the about_Execution_Policies help topic at
https:/go.microsoft.com/fwlink/?LinkID=135170. Do you want to change the execution policy?
[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is «N»): y

  1. Create Windows PowerShell Script

Save all below content to a PowerShell script as names «rsync-config.ps1»

# Create rsync config and share folders
New-Item -Path C:\rsync -ItemType Directory
New-Item -Path C:\rsync_share -ItemType Directory

# Create rsync.conf file
@"
[rsync_share]
path = /cygdrive/c/rsync_share
ignore errors
read only = no
list = yes
auth users = rsync
secrets file = /cygdrive/c/rsync/rsync.passwd
"@ | Add-Content -Path C:\rsync\rsyncd.conf

# Open port 873 for rsync
New-NetFirewallRule `
    -Name "Rsync Daemon (873)" `
    -DisplayName "Rsync Daemon (873)" `
    -Protocol TCP `
    -LocalPort 873

# Create scheduled task to run rsync daemon
$Trigger = New-ScheduledTaskTrigger -AtStartup
$Action = New-ScheduledTaskAction `
    -Execute "Powershell.exe" `
    -Argument "rsync --daemon --config=C:\rsync\rsyncd.conf"
Register-ScheduledTask `
    -TaskName "Rsync Daemon" `
    -Trigger $Trigger `
    -Action $Action `
    -RunLevel Highest `
    -User System
Parameter Description Path
rsync Configuration folder for rsync C:rsync\
rsync_share Shared folder for rsync C:rsync_share\
/cygdrive/c/rsync_share Cygwin path prefix up to the shared folder location C:rsync_share\
auth users = rsync Specifies that only the «rsync» user can connect
secrets file = /cygdrive/c/rsync/rsync.passwd Specifies the location of the password file that will be used for authenticating rsync connections. C:rsyncrsync.passwd

The format of the rsync password file(C:rsyncrsync.passwd) is as follows:

username:password
  • username: This is the username for the rsync account.
  • password: This is the password for the rsync account.

Each line in the file represents a single rsync account. Create multiple accounts by adding additional lines to the file.

  1. Execute the PowerShell Script
2023-06-08_151758.png

2023-06-08_151758.png
run on Windows server to automatically setup rsync daemon, create share folder names «rsync_share» and configure the firewall. moreover, the rsync service startup on reboot.


Setup RSYNC Client on Windows

Install RSYNC on the client machines.
Install Rsync software like a linux through Chocolatey

choco install rsync -y

Operating

i. Manual

On the client PC, create a folder to store rsync data

mkdir -p C:\rsync_client
Directory: C:\

Mode LastWriteTime Length Name

—- ————- —— —-

d—— 7/11/2023 12:46 PM rsync_client

For example:

To sync files from client to server (PUSH):

rsync -avz /cygdrive/c/rsync_client  rsync://server_ip_address/rsync_share

To sync files from server to client (PULL):

rsync -avz rsync://server_ip_address/rsync_share /cygdrive/c/rsync_client 

To sync files from server to client with --progress flag on Linux

rsync -avz --progress --update --chmod=ugo=rwX --delete rsync://server_ip_address/rsync_share ./rsync_client
2023-06-08_163246.png

2023-06-08_163246.png
Option Description
-a Enables archive mode
-v Enables verbose output
-z Enables compression
--progress Displays progress during the transfer
--update Skips files that are newer on the destination
--chmod=ugo=rwX Sets permissions of synced files
--delete Removes files on the destination that do not exist on the source
rsync://server_ip_address/rsync_share Specifies the remote directory as rsync_share
./ClientFolder Specifies the current directory on the local machine

i. Automatic via Script

Update Policy to allow execute self PowerShell script.

Set-ExecutionPolicy RemoteSigned
  1. Create Windows PowerShell Script

Save all below content to a PowerShell script as names «rsync-client.ps1»

# Set variables
$RsyncPullFolder = "C:\rsync_pull"
$RsyncConfigFolder = "C:\rsync"
$Source = "/cygdrive/c/rsync_pull"
$Destination = "rsync_server_ip::share"
$Username = "rsync"
$Password = "P@ssw0rd123!"
$TaskName = "Rsync Pull"
$TriggerTime = "08:00"


# Check if rsync_pull and rsync folders exist, and create them if they don't
if (!(Test-Path $RsyncPullFolder)) {
    New-Item -ItemType Directory -Path $RsyncPullFolder
}
if (!(Test-Path $RsyncConfigFolder)) {
    New-Item -ItemType Directory -Path $RsyncConfigFolder
}
New-Item -Path "$RsyncConfigFolder" -Name "rsync.passwd" -ItemType "File"
Add-Content -Path "$RsyncConfigFolder\rsync.passwd" -Value "$Password"


# Build rsync command
$RsyncCommand = "rsync -avz --progress --update --chmod=ugo=rwX --delete --password-file=/cygdrive/c/rsync/rsync.passwd $Username@$Destination $Source"

# Create scheduled task to run rsync command
$Trigger = New-ScheduledTaskTrigger -Daily -At $TriggerTime
$Action = New-ScheduledTaskAction -Execute "Powershell.exe" -Argument "$RsyncCommand"
Register-ScheduledTask -TaskName $TaskName -Trigger $Trigger -Action $Action -RunLevel Highest -User "SYSTEM"
Variable Description
$RsyncPullFolder Specifies the download directory on the local machine
$RsyncConfigFolder Specifies the config file storage directory on the local machine
$Source Specifies the source directory on the local machine
$Destination Specifies the destination directory on the remote server
$Username Specifies the username of the rsync user on the remote server
$Password Specifies the password for the rsync user on the remote server
$TaskName Specifies the name of the scheduled task
$TriggerTime Specifies the time when the scheduled task will run

This script runs on a Windows client and automatically sets up an rsync schedule, creates an rsync folder named ‘rsync_pull’, and configures the username and password for authorization.


Conclusion

rsync can be a reliable and efficient solution for file synchronization on Windows, with its ability to preserve file attributes, compress data during transfer, and show verbose output.

To use rsync on Windows, you can install a version of rsync compatible with Windows, configure your system’s firewall and network settings, and create a configuration file to specify the files and directories to be synced.


Reference

  • How-to Run a PowerShell Script
  • The cygdrive path prefix

Related

  • RSYNC USAGE
Windowsrsync

Понравилась статья? Поделить с друзьями:
0 0 голоса
Рейтинг статьи
Подписаться
Уведомить о
guest

0 комментариев
Старые
Новые Популярные
Межтекстовые Отзывы
Посмотреть все комментарии
  • Ms windows для пк является примером системы
  • Logitech g hub download windows 10
  • Расширение файла heic чем открыть в windows 10
  • Английский язык на экране приветствия windows 10 реестр
  • L2tp сервер mikrotik для windows