blog-image

SysAdmin – Automation

  • dHENRY
  • 05/07/2019
  • (Reading time : 10 mn)

(**) Translated with www.DeepL.com/Translator

To automatically deploy services, system administrators use “scripts” to sequence desired operations.
This requires a culture, research, implementation of good execution tests, etc….
Today there are several systems for creating “recipes”.

These tools make it possible to industrialize installation and maintenance operations on several operating systems at the same time.

Not having in my job any use of these tools, I preferred to make one, “old school” while in bash :) This tool is not intended to be distributed, the tools on the market are obviously very sophisticated and adapted to hybrid environments.

I started from a simple example provided by “Ansible”, and built a tool adapted to the size of my infrastructure (exclusively for Debian).

The example is available on this page. It explains how to make your first Ansible playbook and concerns the installation of a nginx service, including the installation of a specific index.html file. It is therefore a question of writing this :

---
- name: Install nginx
    hosts: host.name.ip
    become: true

    tasks:
    - name: Add epel-release repo
    yum:
        name: epel-release
        state: present

    - name: Install nginx
    yum:
        name: nginx
        state: present

    - name: Insert Index Page
    template:
        src: index.html
        dest: /usr/share/nginx/html/index.html

    - name: Start NGiNX
    service:
        name: nginx
        state: started

In this example, we can see the “yum” command and deduce that the “yum” command will be used by specifying the “epl-release” source:

    tasks:
      - name: Add epel-release repo
        yum:
          name: epel-release
          state: present
    
      - name: Install nginx
        yum:
          name: nginx
          state: present

What I want

In my case I will use native commands, check that each command has been executed, create a “Capitalization” tree structure in order to reuse the scripts. The big problem with shells is the repetition of code, because of the good execution tests that must be performed after each order.

I wish:

  • Execute system commands, on the local server or on remote server(s) through SSH communication (authentication by key exchange)
  • Add comments (important)
  • Copy files (cp for local, scp for remote servers via SSH, to simplify I will always use the scp command, whose behavior is similar to the cp command for local use)
  • Perform string replacements in a file (sed or awk)
  • Allow the capitalization of these order sequences (storage space, organization, metadata)
  • ….

Translation of the script Ansible

I start from the Ansible script presented above and transform it in a very basic way into shell instructions ( for DEBIAN). This will give us a shell :

File “./script” :

apt update
apt install nginx
scp index.html /usr/share/nginx/html/index.html
systemctl start nginx

Each of the indicated commands will follow one another without ever knowing if the previous one ended well, so its complete correct execution is not guaranteed.

Comments

Add some comments :

# Update of the list of DEBIAN applications available
apt update
# Installation of the nginx service
apt install nginx 
# copy of the internal file installed by default 
# on all new installations of nginx to check by PLC that it works 
scp index.html /usr/share/nginx/html/index.html
# Starting the nginx service 
systemctl start nginx

Building an interpreter

First of all, I will make a program that will read the contents of the “script” file and execute the indicated commands, one by one, checking that each of the commands has been correctly executed.

Retrieve the list of commands in order and execution

The POC will simply be done with the Bash language. The name of the script : “execute.sh

#!/bin/bash
while IFS= read -r line; do
        # Does not process comments (starts with space++++# or #
        # Comparison with the use of regular expressions
        if [[ $line =~ ^[[:space:]]*# ]];then
            continue
        fi
        # command execution
        $line
done < ./script

Save the file, then: chmod 755 execute.sh which makes the shell executable, and execute the shell : ./execute.sh

The commands are executed one by one…. Since I use a “no admin” account, I inevitably get an error !!!

Introducing a test of good execution

#!/bin/bash
while IFS= read -r line; do
    # Does not process comments (starts with space++++# or #
    if [[ $line =~ ^[[:space:]]*# ]];then
        continue
    fi
    # command execution
    $line
    # Retrieving error code
    ERR=$?
    if [ ! "$ERR" = "0" ];then
        echo ""
        echo "---------------"
        echo "[**ERR] - Une erreur s'est produite  - [Commande :$line - code erreur: $ERR]"
        echo "---------------"
        echo ""
        # Process stop, exit code = last code Error intercepted
        exit $ERR
    fi
done < ./script

Save, then run the shell : ./execute.sh
The script now stops at the first error encountered. To perform this type of operation, you must have “root” privileges. You can therefore execute the operation by prefixing the command with: “sudo” and set up your account for this type of use, or connect to “root”.

Running in “root” mode

Root connection to the console, or use of the “sudo” command (requires system configuration for your account)

**Need to get 1,588 kB of archives.
After this operation, 2,865 kB of additional disk space will be used.
Do you want to continue? [Y/n] Abort.
[ERR] - Une erreur s’est produite - [Commande :apt install nginx - code erreur: 1]

An error occurs because the apt command is waiting for a user response : Y | N for respectively “Yes”, “No”. I want to install whatever happens, so I will indicate the parameter “-y” to the command “apt install nginx” in the script file

# Updating the content of DEBIAN repositories for the system
apt update
# Installation of the nginx service
apt -y install nginx 
# copy of the internal file installed by default on all new installations of nginx in order to check by PLC that it works 
scp index.html /usr/share/nginx/html/index.html
# Starting the nginx service 
systemctl start nginx

Save, then run the shell : ./execute.sh

**…
Setting up nginx-full (1.10.3-1+deb9u2) …
Setting up nginx (1.10.3-1+deb9u2) …
cp: cannot stat ‘index.html’: No such file or directory
[ERR] - Une erreur s’est produite - [Commande :cp index.html /usr/share/nginx - code erreur: 1]

The script asks you to copy an index.html file to the destination /usr/share/nginx/html/index.html, but this file does not exist. I will therefore plan and in the spirit of “Package” a directory for resources and thus allow to keep the context of execution of this PLC. One could imagine a structure such as :

–[Name of the automation project] file : script Directory : ressources file : ressource1.txt file : ressource2.txt file : ….

but I’m not going to change the content of the “script” file that will remain: scp index.html /usr/share/nginx/html/html/index.html and consider that all sources to be copied without prefix (../, ./test/, /root/ etc) are available in the “resources” directory of the project. You are a system admin, you know what you are doing, so you explicitly know the destination of the file on the host.
mkdir ressources
touch ressources/index.html
So we now have an empty file to copy

Modification du répertoire de travail de “execute.sh”

#!/bin/bash
# Répertoire des ressources
RESSOURCESDIR="./ressources"
# Nom du script d'automatisation à exécuter
SCRIPT="script"
#Déplacement dans le répertoire de travail "ressources" si existe
if [ -d "$RESSOURCESDIR" ];then
    SCRIPT="../$SCRIPT"
    cd "$RESSOURCESDIR"
else
    SCRIPT="./$SCRIPT"
fi
while IFS= read -r line; do
    #Ne traite pas les commentaires (démarre par space+++# ou bien #
    if [[ $line =~ ^[[:space:]]*# ]];then
        continue
    fi
    # Execution de la commande
    $line
    # Récupération code Erreur
    ERR=$?
    if [ ! "$ERR" = "0" ];then
        echo ""
        echo "---------------"         echo "[*ERR] - Une erreur s'est produite  - [Commande :$line - code erreur: $ERR]"
        echo "---------------"
        echo ""
        # Arrêt du processus, code de sortie = dernier code Erreur intercepté
        exit $ERR
    fi
done < $SCRIPT

End of execution warning

I add a little blabla at the end of “execute.sh”, to indicate the end and proper execution of the automation script.

echo ""
echo "---------------"
echo "[*FINOK] - Le script d'automatisation s'est correctement exécuté"
echo "---------------"
echo ""
exit 0

Save, then run the shell : ./execute.sh
./execute.sh
Result obtained :

    [*FINOK] - Le script d'automatisation s'est correctement exécuté

With a basic script written in bash, and a list of tasks to be executed on a local server, the method works perfectly. You will also notice that the scp command behaves like the “cp” command when executing locally.

Running on a remote host

here we can imagine two methods:

  • Passing from the list of hosts in parameters to the execute.sh command
  • or pass as a parameter to the execute.sh command of a file containing the list of servers to be processed.

It should be noted that in both cases one or more parameters must be indicated.

Passing one or more hosts to parameters

Passing a host as a parameter is the same as executing this command: Execute.sh [host name or IP address] [host name or IP address]
Because of the use of ssh, you will need to copy your ~/.ssh/id_rsa.pub key to the remote host before attempting an automatic ssh connection. If this key does not exist, use the ssh-keygen command to generate the private/public key pair associated with your user account. Let us consider that the key is installed and that the SSH connection does not require a password. In this example, the remote host for me is the IP address: 192.168.1.1.56
Running the operation suite automatically will require you to connect, so I will adapt the shell execute.sh

Detection of host parameters in “execute.sh”

I will consider that there is always at least one host in the parameters. If the command does not have a parameter, then the host will be the local server or “local”. If I want to process several hosts including the local server, I will add “local” to the list of servers. Without parameter the local server will be the only server processed.

#Hôtes à traiter
HOSTS=""
#Detection des hosts
for PARAMHOST in "$@"
do
    HOSTS="$HOSTS $PARAMHOST"
done
#Aucun paramètre pas de host donc traitement local
if [ "$HOST" == '' ];then
    HOST="local"
fi

After this processing I will have access to the list of hosts to be processed. Simply iterate this list and execute the destination commands on the host. I’m adding a few more checks along the way :

#!/bin/bash
PROCESSERRFILE=dirname $0
PROCESSERRFILE=$PROCESSERRFILE/log-install.log
touch $PROCESSERRFILE
#Répertoire des ressources
RESSOURCESDIR="./ressources"
#Paramétres de commande SSH
PARAMSSH="-o PasswordAuthentication=no -o ConnectTimeout=5 -o StrictHostKeyChecking=no"
SSHCMD="ssh $PARAMSSH"
SCPCMD="scp $PARAMSSH"
#Nom du script d'automatisation à exécuter
SCRIPT="script"
#Hôtes à traiter
HOSTS=""
# Détection des hosts
for PARAMHOST in "$@"
do
    HOSTS="$HOSTS $PARAMHOST"
done
# Aucun paramètre pas de host donc traitement local
if [ "$HOSTS" == '' ];then
    HOSTS="local"
fi
# Erreurs détectées
ERREURS=""
# Déplacement dans le répertoire de travail "ressources" si existe
if [ -d "$RESSOURCESDIR" ];then
    SCRIPT="../$SCRIPT"
    cd "$RESSOURCESDIR"
else
    SCRIPT="./$SCRIPT"
fi
for SERVER in $HOSTS;
do
ERREURSERVER=""
AUTOMATE=()
while IFS= read -r LINE; do
        AUTOMATE+=("$LINE")
done < $SCRIPT
for LINE in "${AUTOMATE[@]}"
do
    #Ne traite pas les commentaires (démarrés par space+++# ou bien #
    if [[ $LINE =~ ^[[:space:]]# ]];then
            continue
    fi
    # Execution de la commande pour chacun des hosts - trim de la commande
    CMD=echo $LINE|xargs
    if [ $SERVER" != "local" ];then
        CMD="$SSHCMD root@$SERVER $CMD"
    fi
    # Altération de la commande scp pour ajouter l'hôte final
    if [[ $LINE =~ ^scp ]];then
        if [ "$SERVER" != "local" ];then
                CMD=""
                I=0
                for word in $LINE;
                do
                    # substitution de la commande scp par la commande interne
                    if [ $I -eq 0 ];then
                        CMD="$SCPCMD"
                    else
                        #Ajout des infos hosts dans la destination
                        if [ $I -gt 1 ];then
                            # exception si déjà mentionné
                            if [[ ! $word =~ @ ]];then
                                CMD="$CMD root@$SERVER:$word"
                            fi
                        else
                            CMD="$CMD $word"
                        fi
                fi
                I=$(($I+1))
                done
            fi
        fi
        echo "  [Exec : $LINE - CMD : $CMD]"
        $CMD
        # Récupération code Erreur
        ERR=$?
        if [ "$ERR" != "0" ];then
            echo ""
            echo "---------------"
            echo "[ERR] - Une erreur s'est produite  - [Commande :$LINE - code erreur: $ERR]"
            echo "     Commande réellement exécutée : $CMD"
            echo "---------------" 
            echo ""
            # Arrêt du processus, code de sortie = dernier code Erreur intercepté
            ERREUR="$ERREUR###$SERVER###$LINE###$CMD###$ERR"
            ERREURSERVER=$ERR
            break
    fi
done
if [ "$ERREURSERVER" = "" ];then
echo "" 
echo "---------------" echo "[FINOK] - Le script d'automatisation s'est correctement exécuté pour : $SERVER"
echo "---------------"
echo ""
fi
done
if [ "$ERREUR" != "" ];then
    echo "Erreurs detéctées : $ERREUR"
fi
exit 0

And still from the script file, I run execute.sh by providing a list of several servers. Operations are proceeding normally.

Conclusion

As said at the beginning it’s “old school” but it works and is enough for me in my use…

Going further

These first lines of code in Bash are at the origin of the “Mytinydc-automation” project [Click here to access the Gitlab (french language only)].

Document licence : Creative Commons (CC BY-NC-ND 4.0)

THIS DOCUMENTATION IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND AND DISTRIBUTED FOR EDUCATIONAL PURPOSES ONLY. THE AUTHOR, CONTRIBUTORS TO THIS DOCUMENTATION OR ©MYTINYDC.COM SHALL IN NO EVENT BE LIABLE FOR ANY DIRECT OR INDIRECT DAMAGE THAT MAY RESULT FROM THE APPLICATION OF THE PROCEDURES IMPLEMENTED IN THIS DOCUMENTATION, OR FROM THE INCORRECT INTERPRETATION OF THIS DOCUMENT.

(**) Translated with www.DeepL.com/Translator

Post a comment

Sorry, the answer to the Captcha question is incorrect
Thank you. The Mailman is on His Way :)
Sorry, don't know what happened. Please to try later :(
One field is not valid

Comments (not translated)

No comments