Merge pull request #368 from sspencerwire/edit_365

Edit 365

Author: sspencerwire

docs/books/learning_rsync/01_rsync_overview.md

@@ -23,9 +23,9 @@ What are the backup methods?
 
 ##  rsync in brief
 
-On a server, I backed up the first partition to the second partition, which is commonly known as "Local backup." The specific backup tools are `tar` , `dd` , `dump` , `cp `, etc. can be achieved. But in fact, it is still "Don't put the eggs in the same basket." Once the hardware fails and cannot boot and start normally, the data still cannot be retrieved. In order to solve the local backup For this problem, we introduced another kind of backup --- "remote backup".
+On a server, I backed up the first partition to the second partition, which is commonly known as "Local backup." The specific backup tools are `tar` , `dd` , `dump` , `cp `, etc. can be achieved. But you shouldn't "put all of your eggs in the same basket." Once the hardware fails and cannot start normally, the data still cannot be retrieved." In order to solve the local backup For this problem, we introduced another kind of backup --- "remote backup".
 
-Some people will say, I use the `tar` or `cp` command on the first server, and then transfer it to the second server via `scp` or `sftp`.
+Some people will say, can't I just use the `tar` or `cp` command on the first server and send it to the second server via `scp` or `sftp`?
 
 In a production environment, the amount of data is relatively large. First of all, `tar` or `cp` consumes a lot of time and occupies system performance. Transmission via `scp` or `sftp` also occupies a lot of network bandwidth, which is not allowed in the actual production environment. Secondly, these commands or tools need to be manually entered by the administrator and need to be combined with the crontab of the scheduled task. However, the time set by crontab is not easy to grasp, and the set time is too short. For example, if it is executed once every 1 minute, it may happen that the first script is not executed, and the second script is executed again; the set time has passed For example, if it is executed once every 5 hours, there may be data loss because the data is not backed up in time.
 
@@ -48,10 +48,13 @@ The original `rsync` was maintained by the Australian programmer <font color=red
     **rsync itself is only an incremental backup tool and does not have the function of real-time data synchronization. It needs to be supplemented with another program. In addition to this, synchronization is one-way, and if you want two-way backup, you need to use another tool to achieve it. **
 
 ###  Basic Principles and Features
+
 How does `rsync` achieve efficient one-way data synchronization backup?
-The core of `rsync` is its **Checksum algorithm** . If you are interested, you can go to [ Rsync Working Principle ](https://rsync.samba.org/how-rsync-works.html) and [ rsync Algorithm ](https ://rsync.samba.org/tech_report/) I understand that this part is beyond the scope of the author's ability, so I won't give too much explanation.
+
+The core of `rsync` is its **Checksum algorithm**. If you are interested, you can go to [How Rsync works](https://rsync.samba.org/how-rsync-works.html) and [The rsync algorithm](https://rsync.samba.org/tech_report/) for more information, This section is beyond the author's competence and will not be covered too much.
 
 The characteristics of `rsync` are:
+
 * The entire directory can be updated recursively;
 * Can selectively retain file synchronization attributes, such as hard link, soft link, owner, group, corresponding permissions, modification time, etc., and can retain some of the attributes;
 * Support two protocols for transmission, one is ssh protocol, the other is rsync protocol

docs/books/learning_rsync/02_rsync_demo01.md

@@ -15,7 +15,7 @@ update: 2021-11-04
 Before the specific demonstration of rsync synchronization, you need to use the `rsync` command. In Rocky Linux 8, the rsync rpm package is installed by default, and the version is 3.1.3-12, as follows:
 
 ```bash
-[root@Rocky ~ ] # rpm -qa|grep rsync
+[root@Rocky ~]# rpm -qa|grep rsync
 rsync-3.1.3-12.el8.x86_64
 ```
 
@@ -72,14 +72,14 @@ Fedora34-->|pull/download|RockyLinux8;
 Since it is based on the SSH protocol, we first create a user in the server:
 
 ```bash
-[root@Rocky ~ ] # useradd testrsync
-[root@Rocky ~ ] # passwd testrsync
+[root@Rocky ~]# useradd testrsync
+[root@Rocky ~]# passwd testrsync
 ```
 
 On the client side, we pull/download it, and the file on the server is /rsync/aabbcc
 
 ```bash
-[root@fedora ~ ] # rsync -avz [email protected]:/rsync/aabbcc /root
+[root@fedora ~]# rsync -avz [email protected]:/rsync/aabbcc /root
 [email protected] ' s password:
 receiving incremental file list
 aabbcc
@@ -92,7 +92,7 @@ aabbcc
 The transfer was successful.
 
 !!! tip "Attention"
-    If the server's SSH port is not the default 22, you can specify the port in a similar way-`rsync -avz -e ' ssh -p [port] ' `.
+    If the server's SSH port is not the default 22, you can specify the port in a similar way---`rsync -avz -e 'ssh -p [port]' `.
 
 ### push/upload
 
@@ -110,7 +110,8 @@ total size is 883 speedup is 0.91
 rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1330) [sender = 3.2.3]
 ```
 
-**Prompt permission denied, how to deal with it? **
+**Prompt permission denied, how to deal with it?**
+
 First check the permissions of the /rsync/ directory. Obviously, there is no permission. We can use `setfacl` to give permission:
 
 ```bash

docs/books/learning_rsync/03_rsync_demo02.md

@@ -5,15 +5,16 @@ contributors: Steven Spencer
 update: 2021-11-04
 ---
 
-#  Based on the presentation rsync protocol
+# Demonstration based on rsync protocol
 In vsftpd, there are virtual users (impersonated users customized by the administrator) because it is not safe to use anonymous users and local users. We know that a server based on the SSH protocol must ensure that there is a system of users. When there are many synchronization requirements, it may be necessary to create many users. This obviously does not meet the GNU/Linux operation and maintenance standards (the more users, the more insecure), in rsync, for security reasons, there is an rsync protocol authentication login method.
 
-**How ​​to do it? **
+**How ​​to do it?**
+
 Just write the corresponding parameters and values ​​in the configuration file. In Rocky Linux 8, you need to manually create the file <font color=red>/etc/rsyncd.conf</font>.
 
 ```bash
-[root@Rocky ~ ] # touch /etc/rsyncd.conf
-[root@Rocky ~ ] # vim /etc/rsyncd.conf
+[root@Rocky ~]# touch /etc/rsyncd.conf
+[root@Rocky ~]# vim /etc/rsyncd.conf
 ```
 
 Some parameters and values ​​of this file are as follows, [ here ](04_rsync_configure.md) has more parameter descriptions:
@@ -28,7 +29,7 @@ Some parameters and values ​​of this file are as follows, [ here ](04_rsync_
 | comment = rsync | Remarks or description information |
 | path = /rsync/ | The system path location where it is located |
 | read only = yes| yes means read only, no means read and write |
-| dont compress = *.gz *.gz2 *.zip | Which file types do not compress it |
+| dont compress = \*.gz \*.gz2 \*.zip | Which file types do not compress it |
 | auth users = li| Enable virtual users and define what a virtual user is called. Need to create it yourself|
 | secrets file = /etc/rsyncd_users.db | Used to specify the location of the virtual user's password file, which must end in .db. The content format of the file is "Username: Password", one per line |
 
@@ -38,7 +39,7 @@ Some parameters and values ​​of this file are as follows, [ here ](04_rsync_
 Write some file content to <font color=red>/etc/rsyncd.conf</font>, and write the user name and password to /etc/rsyncd_users.db, the permission is 600
 
 ```bash
-[root@Rocky ~ ] # cat /etc/rsyncd.conf
+[root@Rocky ~]# cat /etc/rsyncd.conf
 address = 192.168.100.4
 port = 873
 pid file = /var/run/rsyncd.pid
@@ -47,44 +48,43 @@ log file = /var/log/rsyncd.log
 comment = rsync
 path = /rsync/
 read only = yes
-dont compress = * .gz * .bz2 * .zip
+dont compress = *.gz *.bz2 *.zip
 auth users = li
 secrets file = /etc/rsyncd_users.db
-[root@Rocky ~ ] # ll /etc/rsyncd_users.db
+[root@Rocky ~]# ll /etc/rsyncd_users.db
 -rw------- 1 root root 9 November 2 16:16 /etc/rsyncd_users.db
-[root@Rocky ~ ] # cat /etc/rsyncd_users.db
+[root@Rocky ~]# cat /etc/rsyncd_users.db
 li:13579
 ```
 
 You may need to `dnf -y install rsync-daemon` before you can start the service: `systemctl start rsyncd.service`
 
 ```bash
-[root@Rocky ~ ] # systemctl start rsyncd.service
-[root@Rocky ~ ] # netstat -tulnp
-Active Internet connections (only servers)
-Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
-tcp 0 0 0.0.0.0:22 0.0.0.0: * LISTEN 691/sshd
-tcp 0 0 192.168.100.4:873 0.0.0.0: * LISTEN 4607/rsync
-tcp6 0 0 :::22 ::: * LISTEN 691/sshd
-udp 0 0 127.0.0.1:323 0.0.0.0: * 671/chronyd
-udp6 0 0 ::1:323 ::: * 671/chronyd
+[root@Rocky ~]# systemctl start rsyncd.service
+[root@Rocky ~]# netstat -tulnp
+Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
+tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      691/sshd            
+tcp        0      0 192.168.100.4:873       0.0.0.0:*               LISTEN      4607/rsync          
+tcp6       0      0 :::22                   :::*                    LISTEN      691/sshd            
+udp        0      0 127.0.0.1:323           0.0.0.0:*                           671/chronyd         
+udp6       0      0 ::1:323                 :::*                                671/chronyd  
 ```
 
-##  pull/download
+## pull/download
 
 Create a file in the server for verification: `[root@Rocky]# touch /rsync/rsynctest.txt`
 
 The client does the following:
 
 ```bash
-[root@fedora ~ ] # rsync -avz [email protected]::share /root
+[root@fedora ~]# rsync -avz [email protected]::share /root
 Password:
 receiving incremental file list
 ./
 rsynctest.txt
 sent 52 bytes received 195 bytes 7.16 bytes/sec
 total size is 883 speedup is 3.57
-[root@fedora ~ ] # ls
+[root@fedora ~]# ls
 aabbcc anaconda-ks.cfg fedora rsynctest.txt
 ```
 
@@ -93,8 +93,8 @@ success! In addition to the above writing based on the rsync protocol, you can a
 ##  push/upload
 
 ```bash
-[root@fedora ~ ] # touch /root/fedora.txt
-[root@fedora ~ ] # rsync -avz /root/* [email protected]::share
+[root@fedora ~]# touch /root/fedora.txt
+[root@fedora ~]# rsync -avz /root/* [email protected]::share
 Password:
 sending incremental file list
 rsync: [sender] read error: Connection reset by peer (104)
@@ -106,7 +106,7 @@ You are prompted that the reading error is related to the "read only = yes" of t
 Try again, prompting you permission denied:
 
 ```bash
-[root@fedora ~ ] # rsync -avz /root/* [email protected]::share
+[root@fedora ~]# rsync -avz /root/* [email protected]::share
 Password:
 sending incremental file list
 fedora.txt
@@ -119,7 +119,7 @@ rsync error: some files/attrs were not transferred (see previous errors) (code 2
 Our virtual user here is <font color=red>li</font>, which is mapped to the system user <font color=red>nobody</font> by default. Of course, you can change it to other system users. In other words, nobody does not have write permission to the /rsync/ directory. Of course, we can use `[root@Rocky ~]# setfacl -mu:nobody:rwx /rsync/` , try again, and succeed.
 
 ```bash
-[root@fedora ~ ] # rsync -avz /root/* [email protected]::share
+[root@fedora ~]# rsync -avz /root/* [email protected]::share
 Password:
 sending incremental file list
 fedora.txt

docs/books/learning_rsync/05_rsync_authentication-free_login.md

@@ -11,7 +11,7 @@ From [rsync Brief Description](01_rsync_overview.md) we know that rsync is an in
 
 With inotify-tools, this program tool can realize one-way real-time synchronization. Since it is real-time data synchronization, the prerequisite is to log in without password authentication.
 
-**Regardless of whether it is rsync protocol or SSH protocol, both can achieve password-free authentication login. **
+**Regardless of whether it is rsync protocol or SSH protocol, both can achieve password-free authentication login.**
 
 ## SSH protocol password-free authentication login
 
@@ -40,42 +40,59 @@ The key's randomart image is:
 | .o.o. |
 +----[SHA256]-----+
 ```
+
 Then, use the `scp` command to upload the public key file to the server. For example, I upload this public key to the user **testrsync**
+
 ```bash
 [root@fedora ~]# scp -P 22 /root/.ssh/id_rsa.pub [email protected]:/home/testrsync/
 ```
+
 ```bash
 [root@Rocky ~]# cat /home/testrsync/id_rsa.pub >> /home/testrsync/.ssh/authorized_keys
 ```
+
 Try to log in without secret authentication, success!
+
 ```bash
 [root@fedora ~]# ssh -p 22 [email protected]
 Last login: Tue Nov 2 21:42:44 2021 from 192.168.100.5
 [testrsync@Rocky ~]$
 ```
+
 !!! tip "Attention!"
     The server configuration file **/etc/ssh/sshd_config** should be opened <font color=red>PubkeyAuthentication yes</font>
+
 ## rsync protocol password-free authentication login
+
 On the client side, the rsync service prepares an environment variable for the system-**RSYNC_PASSWORD**, which is empty by default, as shown below:
+
 ```bash
 [root@fedora ~]# echo "$RSYNC_PASSWORD"
+
 [root@fedora ~]#
 ```
+
 If you want to achieve password-free authentication login, you only need to assign a value to this variable. The value assigned is the password previously set for the virtual user <font color=red>li</font>. At the same time, declare this variable as a global variable.
+
 ```bash
 [root@Rocky ~]# cat /etc/rsyncd_users.db
 li:13579
 ```
+
 ```bash
 [root@fedora ~]# export RSYNC_PASSWORD=13579
 ```
+
 Try it, success! No new files appear here, so the list of transferred files is not displayed.
+
 ```bash
 [root@fedora ~]# rsync -avz [email protected]::share /root/
 receiving incremental file list
 ./
+
 sent 30 bytes received 193 bytes 148.67 bytes/sec
 total size is 883 speedup is 3.96
 ```
+
 !!! tip "tip!"
     You can write this variable into **/etc/profile** to make it take effect permanently. The content is: `export RSYNC_PASSWORD=13579`

docs/books/learning_rsync/06_rsync_inotify.md

@@ -28,12 +28,12 @@ Append the environment variable PATH, write it to the configuration file and let
 [root@Rocky ~]# vim /etc/profile
 ...
 PATH=$PATH:/usr/local/inotify-tools/bin/
-[root@Rocky ~]#. /etc/profile
+[root@Rocky ~]# . /etc/profile
 ```
 
-**Why not use the inotify-tools RPM package of the EPEL repository? And the way to use source code to compile and install? **
+**Why not use the inotify-tools RPM package of the EPEL repository? And the way to use source code to compile and install?**
 
-The author personally believes that remote data transmission is related to efficiency issues, especially in the production environment, when the number of files to be synchronized is large and the single file is particularly large, this is particularly important. In addition, the new version will have some bug fixes and function expansions, and perhaps the transmission efficiency of the new version will be higher, so I recommend installing inotify-tools by source code. Of course, this is the author's personal suggestion, not every user must follow.
+The author personally believes that remote data transmission is a matter of efficiency, especially in a production environment, where there are a large number of files to be synchronized and a single file is particularly large. In addition, the new version will have some bug fixes and function expansions, and perhaps the transmission efficiency of the new version will be higher, so I recommend installing inotify-tools by source code. Of course, this is the author's personal suggestion, not every user must follow.
 
 ## Kernel parameter adjustment
 
@@ -46,6 +46,7 @@ You can adjust the kernel parameters according to the needs of the production en
 128
 28014
 ```
+
 * max_queued_events-maximum monitor queue size, default 16384
 * max_user_instances-the maximum number of monitoring instances, the default is 128
 * max_user_watches-the maximum number of files monitored per instance, the default is 8192
@@ -101,6 +102,7 @@ Type the command in the first terminal pts/0, and the window is locked after pre
 
 ```bash
 [root@Rocky ~]# inotifywait -mrq -e create,delete /rsync/
+
 ```
 
 In the second terminal pts/1, go to the /rsync/ directory and create a file.
@@ -141,7 +143,7 @@ $a | while read directory event file
 
 !!! tip "emphasize again!"
     When using the SSH protocol for data synchronization transmission, if the SSH service port of the target machine is not 22, you can use a method similar to this——
-    `b="/usr/bin/rsync -avz -e'ssh -p [port-number]' /rsync/* [email protected]:/home/testfedora/"`
+    `b="/usr/bin/rsync -avz -e 'ssh -p [port-number]' /rsync/* [email protected]:/home/testfedora/"`
 
 !!! tip "Attention!"
     If you want to start this script at boot

docs/books/learning_rsync/07_rsync_unison_use.md

@@ -11,7 +11,7 @@ As we mentioned earlier, one-way synchronization uses rsync + inotify-tools. In
 
 ## Environmental preparation
 
-* Both Rocky Linix 8 and Fedora 34 require source code compilation and installation **inotify-tools**, which is not specifically expanded here.
+* Both Rocky Linux 8 and Fedora 34 require source code compilation and installation **inotify-tools**, which is not specifically expanded here.
 * Both machines must be password-free login authentication, here we use the SSH protocol for
 * [ocaml](https://github.com/ocaml/ocaml/) uses v4.12.0, [unison](https://github.com/bcpierce00/unison/) uses v2.51.4.
 
@@ -49,7 +49,7 @@ Ocaml is a programming language, and the bottom layer of unison depends on it.
 [root@Rocky ~]# ls /usr/local/ocaml/
 bin lib man
 [root@Rocky ~]# echo PATH=$PATH:/usr/local/ocaml/bin >> /etc/profile
-[root@Rocky ~]#. /etc/profile
+[root@Rocky ~]# . /etc/profile
 ```
 
 ```bash
@@ -87,7 +87,7 @@ bin lib man
 ...
 [root@fedora /usr/local/src/unison-2.51.4]# ls src/unison
 src/unison
-[root@fedora /usr/local/src/unison-2.51.4] cp -p src/unison /usr/local/bin
+[root@fedora /usr/local/src/unison-2.51.4]# cp -p src/unison /usr/local/bin
 ```
 
 
@@ -98,9 +98,9 @@ src/unison
 ### Configure Rcoky Linux 8
 
 ```bash
-[root@Rocky ~] mkdir /dir1
-[root@Rocky ~] setfacl -m u:testrsync:rwx /dir1/
-[root@Rocky ~] vim /root/unison1.sh
+[root@Rocky ~]# mkdir /dir1
+[root@Rocky ~]# setfacl -m u:testrsync:rwx /dir1/
+[root@Rocky ~]# vim /root/unison1.sh
 #!/bin/bash
 a="/usr/local/inotify-tools/bin/inotifywait -mrq -e create,delete,modify,move /dir1/"
 b="/usr/local/bin/unison -batch /dir1/ ssh://[email protected]//dir2"
@@ -116,9 +116,9 @@ done
 ### Configure Fedora 34
 
 ```bash
-[root@fedora ~] mkdir /dir2
-[root@fedora ~] setfacl -m u:testrsync:rwx /dir2/
-[root@fedora ~] vim /root/unison2.sh
+[root@fedora ~]# mkdir /dir2
+[root@fedora ~]# setfacl -m u:testrsync:rwx /dir2/
+[root@fedora ~]# vim /root/unison2.sh
 #!/bin/bash
 a="/usr/local/inotify-tools/bin/inotifywait -mrq -e create,delete,modify,move /dir2/"
 b="/usr/local/bin/unison -batch /dir2/ ssh://[email protected]//dir1"