blog-image

Installation of go-neb bot for Synapse-Matrix (arm64)

  • dHENRY
  • 25/04/2019
  • (Reading time : 9 mn)

(**) Translated with www.DeepL.com/Translator

Update of 19/05/2019 - This procedure also works for a Raspberry PI3 (armhf)

This bot allows the integration of many features, including Giphy (animated Gifs) and the extension of the alert system of the Prometheus monitoring application. **This instruction can also be applied to amd64 (PC) platforms.

The “go-neb” project is hosted herehttps://github.com/matrix-org/go-neb, the documentation clearly explains how to install the service. However, the compilation did not proceed as indicated.

Compilation

I encountered two problems: the package included in the “mattn/go-sqlite3” distribution is impossible to compile and indicates this type of error:

github.com/matrix-org/go-neb/api/handlers  
sqlite3-binding.c: In function'wherePathSolver':  
sqlite3-binding.c:130692:1: internal compiler error: in schedule_block, at haifa-sched.c:6593  
 }  

so I installed the DEBIAN golang-github-mattn-go-sqlite3-dev package as a replacement.
The second problem is located at the level of an “include” that the builder “go” does not find (detected by running the command :

go build /opt/go-neb/vendor/src/github.com/mattn/go-sqlite3/sqlite3.go 

and presents the error:

tmp/go-build020128959/command-line-arguments/_obj/sqlite3.cgo2.o: In function `_sqlite3_bind_blob': vendor/src/github.com/mattn/go-sqlite3/sqlite3.go:44: undefined reference to`sqlite3_bind_blob  
/tmp/go-build020128959/command-line-arguments/_obj/sqlite3.cgo2.o: In function `_sqlite3_bind_text': vendor/src/github.com/mattn/go-sqlite3/sqlite3.go:39: undefined reference to`sqlite3_bind_text

Looking at the code, it mentions an include

#include <sqlite3-binding.h> 

which also uses the include:

#include <sqlite3.h> 

not present in the package, but available through the DEBIAN libsqlite3-dev package. I think it’s just a problem with setting the includes search path. For ease of use, I chose to copy the missing file into the distribution package.

Preparation

apt install go-lang libsqlite3-dev golang-github-mattn-go-sqlite3-dev git  
cd /opt  
git clone https://github.com/matrix-org/go-neb  
export GOPATH=/usr/lib/go-1.7/  
export PATH=$PATH:$GOPATH/bin  
go get github.com/constabulary/gb/....  
cd go-neb  
rm -r vendor/src/github.com/mattn/go-sqlite3  
cp -R /usr/share/gocode/src/github.com/mattn/go-sqlite3 vendor/src/github.com/mattn/  
cp /usr/include/sqlite3.h ./vendor/src/github.com/mattn/go-sqlite3/

Project compilation

gb build github.com/matrix-org/go-neb  

The go-neb binary has been created in the “bin” directory of the project.

Directory of the sqlite3 database

Create the db directory

mkdir db

Create an installer as an archive (tar.gz)

cd /opt/go-neb  
tar cfz /root/go-neb.tgz /opt/go-neb/bin /opt/go-neb/db  
You can now install this archive on any arm64 server:  
Copy the archive into the directory** /root/** of the server  
cd /  
tar xfz /root/go-neb.tgz  
rm /root/go-neb.tgz

File for the systemd service

This service will communicate with the Synapse-Matrix service. To make it simple, I installed the service on the server that hosts the Synapse-Matrix service.
It listens on the port: TCP/4050 at the IP address: localhost.
The database used by this service is of type sqlite3 and stored in the directory /opt/go-neb/db. These settings can be customized in the following file.
After installation, if you need to modify this file, always remember to execute the command :

systemctl daemon-relaod  
#Create the group and use execution  
addgroup go-neb 

adduser --system --home /opt/go-neb/ --no-create-home --disabled-password --shell /bin/nologin --ingroup go-neb go-neb

#Changing the owner of the installation directory  
chown -R go-neb:go-neb /opt/go-neb

#Create the file for systemd  
vi /etc/systemd/system/system/go-neb.service  

and insert these instructions:

[Unit]  
Description=Go-neb Bot For Synapse-Matrix server  
After=network.target   
[Service]  
WorkingDirectory=/opt/go-neb  
ExecStart=/opt/go-neb/bin/go-neb  
StandardOutput=syslog  
StandardError=syslog  
SyslogIdentifier=go-neb  
User=go-neb  
Group=go-neb  
Environment=BIND_ADDRESS=localhost:4050 DATABASE_TYPE=sqlite3 DATABASE_URL=db/go-neb.db?_busy_timeout=5000 BASE_URL=http://localhost:4050  
Restart=always  
RestartSec=10  
StartLimitInterval=900  
StartLimitBurst=3  
[Install]  
WantedBy=multi-user.target

Starting the service

systemctl start go-neb 

# Check the launch  
tail -f /var/log/syslog

Sample answer :

After 25 10:03:40 localhost systemd[1]: Started Go-neb Bot For Synapse-Matrix server.  
After 25 10:03:40 localhost go-neb[5662]: time="2019-04-25T10:03:40+05:30" level=info msg="Go-NEB ({BindAddress:localhost:4050 DatabaseType:sqlite3 DatabaseURL:db/go-neb.db?_busy_timeout=5000 BaseURL:http://localhost:4050 LogDir: ConfigFile:})"

The service is started and listened to on port 4050 of the localhost interface:

#Audit
netstat -ant|grep 4050

Activation of the service at the server boot

systemctl enable go-neb  

Bot setting

The bot must be able to connect to the Synapse-Matrix server. Create a Synapse-Matrix service access account, adapting the settings to your configuration:

/opt/synapse/env/bin/register_new_matrix_user -c /opt/synapse/homeserver.yaml http://matrix.mondomaine.com  

I used the account “botgoneb” with a strong password, not admin.

To obtain the token of this new user, use the following command, adapting it to your configuration:

curl -X POST "http://matrix.mondomaine.com/_matrix/client/r0/login" -d "type": "m.login.password", "user": "botgoneb", "password":"[Password previously entered]" }''  

Response of the command if it is successful:

{  
"access_token":"[USER TOKEN]",  
"device_id": "XXXXXXXX",  
"home_server": "matrix.mydomain.com",  
"user_id": "@botgoneb:matrix.mydomain.com"  
}  

The TOKEN of this new user is the value corresponding to the key “access_token” of the answer.

Connect to the Synapse-Matrix server and execute the following command, it tells the “go-neb” service how to connect to the Synapse-Matrix service

curl -X POST localhost:4050/admin/configureClient --data-binary'{  
    "UserID": "@botgoneb:matrix.mydomain.com",  
    "HomeserverURL": "http://matrix.mondomaine.com",  
    "AccessToken": "<TOKEN obtained previously>",  
    "Sync": true,  
    "AutoJoinRooms": true,  
    "DisplayName": "Bot goneb"  
}'

The command is executed normally and returns a value in JSON format.

I activate the “echo” function of the “go-neb” service for the test

curl -X POST localhost:4050/admin/configureService --data-binary'{"Type": "echo", "Id": "00001", "UserID": "@botgoneb:matrix.mydomain.com", "Config": {}' }'

The command returns a value in JSON format, the function is activated.

Log in to a Riot client, create a lounge and invite the bot: @botgoneb:matrix.mydomain.com**, his reaction is immediate, he is connected. Type in this room the message:

!echo hello world

The bot responds by indicating the text entered after the"!echo" command.

Giphy Integration - Synapse-Matrix Server

Enabling the function: Connect to the console of the server with the “go-neb” service, and type the command :

curl -X POST localhost:4050/admin/configureService --data-binary'{  
    "Type": "giphy",  
    "Id": "00002",  
    "UserID": "@botgoneb:matrix.mydomain.com",  
    "Config": {"api_key": "dc6zaTOxFJmzC"}  
}'  

I advise you to register with Giphy (https://developers.giphy.com/) by creating an application (“Create app”). This allows you to obtain an API key. The key indicated above is the Giphy public key, with a daily usage quota. When the quota is reached, the Giphy server does not send anything back.

Invite the bot to the lounge of your choice, it connects immediately, and use the message: !giphy[search word] Example : !giphy test
the bot returns the first Gif image returned by Giphy.
Riot.im allows you to disable the automatic start of Gifs, check your account settings.

AlertManager/Prometheus monitoring integration - Synapse-Matrix Server

This function allows the AlertManager service (part of the Prometheus project - Monitoring system developed by Soundcloud - installation[click here] (https://www.mytinydc.com/index.php/2018/12/26/installer-serveur-monitoring-prometheus-grafana/)), to send alerts to the Synapse-Matrix system. This part is not documented on the web, after several hours of source code analysis concerning the bot, and testing, this part is now functional.

Prometheus communications flow / Alertmanager / go-neb / Synapse-Matrix / Riot.im user

To separate the functions of each bot, I create a new user on Synapse-Matrix: @botPrometheus:matrix.mydomain.com. I get his access token and add it to the go-neb system (see above).

I create the “Monitoring” lounge, intended to receive the messages of the bot: “#Monitoring:matrix.mydomain.com”.
Retrieve the internal ID of this room, using the Riot client: Select the room, enter “Setting”, “Advanced”, the internal ID appears as: !qmElAGdFYCHoCJuaNt:matrix.mydomain.com

Changing the listening address of the go-neb service

The original configuration of the “go-neb” service, indicates a listening at the address “localhost”. The “Alertmanager” service, located on another server, cannot therefore join the “go-neb” service. So I will change the configuration of the “go-neb” service so that it listens on the network interface connected to the local network.

vi /etc/systemd/system/system/go-neb.service  
#Change the value of the environment variables: Environment=BIND_ADDRESS=localhost:4050 DATABASE_TYPE=sqlite3 DATABASE_URL=db/go-neb.db?_busy_timeout=5000 BASE_URL=http://localhost:4050  

Replace “localhost” with the IP address of this server :

Environment=BIND_ADDRESS=192.168.168.1.12:4050 DATABASE_TYPE=sqlite3 DATABASE_URL=db/go-neb.db?_busy_timeout=5000 BASE_URL=http://192.168.1.12:4050  

Save and exit vi
Update systemd and restart go-neb :

systemctl daemon-reload  
systemctl restart go-neb  

Check with :

netstat -ant|grep 4050  

**Add the Firewall rules on the interface concerned: INPUT TCP/4050 and filter only the server with the Alertmanager service.
The “Alertmanager” service can now join the “go-neb” service.

Enabling the go-neb alertmanager function

This function is activated using the following curl command:

curl -X POST http://192.168.1.12:4050/admin/configureService --data-binary {           
"Type": "alert manager",  
"Id": "00003",                             
"UserID": "@botprometheus:matrix.mydomain.com",  
"Config": {"rooms":  
            {"!qmElAGdFYCHoCJuaNt:matrix.mydomain.com":  
                {  
"text_template": "Prometheus Notification[{{.Status}}]",  
"html_template":"<h1>Notification Prometheus[{{.Status}}]</h1><hr/>{- range.Alerts}}}<div>Alerte : {{{.Labels.alertname}}}<br/>Instance : {{{.Labels.instance}}}<br/>Summary : {{{.Annotations.severity}}<br/>Description: {{.Annotations.description}}<br/>Date of detection : {{.StartsAt}}<br/>{{{if ne .EndsAt \"0001-01-01-01T00:00:00:00Z\" -}}Date and time resolution: {{.EndsAt}}<br/>{- end }}Url : {{{.GeneratorUrl}}</div>{{- end}}", "msg_type": "m.text"}}}}''.

The parameters “text_template” and “html_template” are of type “text” whose form must respect the syntax concerning the “Golang” templates.
I created these templates based on the Javascript source code used for the “Alertmanager” integration of the “Rocketchat” instant messaging service (See installation here).

The “Golang” templating system allows you to create loops, include conditions, etc., just like other systems such as “Ejs for Javascript”, or “Twig” for PHP.
Warning, I couldn’t insert the “style” parameters in the HTML tags, if someone wants to start… Data formatting is not supported ( e. g. StartsAt | AsDate format).
This tutorial helped me a lot: http://goinbigdata.com/example-of-using-templates-in-golang/

Two “go-neb” templates for Alermanager/Prometheus are required:

  • The template “text_template” is used for the system notification, the small popup that appears when the Riot application receives a message.
  • The template “html_template” is used to build the body of the message as you see it appear in the living room.

**Attention the server will return the webhook address, which will be used by Alertmanager.
Example of data obtained (JSON format):

{"ID": "00003", "Type": "alertmanager", "OldConfig":{}, "NewConfig":{"webhook_url": "http://localhost:4050/services/hooks/EDFEERdDM", "rooms":......  

The address to be provided to Alertmanager’s webhook systems will be in this case: http://localhost:4050/services/hooks/EDFEERdD

Changing the Alertmanager configuration

I indicate to Alertmanager the address of the webhook to reach (url given by the activation of the “alertmanager” service of go-neb. Edit the Alertmanager configuration file (/opt/alertmanager-[version]/alertmanager.yml), adding in the “receiver” block, the “webhook_config” block:

Attention YAML file, do not use tabs but spaces, to indent lines.

[..]
receivers:  
- name: "email  
email_configs:   
- send_resolved: true  
to: 
[..] 
webhook_configs:  
- send_resolved: true  
url: "**http://localhost:4050/services/hooks/EDFEERdDM**'' 

Reload the Alertmanager configuration:

systemctl reload prometheus-alertmanager

Test the notification

Execute the following commands, adapting them to your settings, and from the server that hosts the AlertManager service (due to Firewall rules).
You will immediately receive a message in the “#Monitoring:matrix.mydomain.com” room.

Test : Detection of a defect

curl -X POST http://192.168.1.12:4050/services/hooks/EDFEERdD --data-binary "{  
"status": "firing",  
"alerts":[{"labels" : {"alertname": "InstanceDown", "instance" : "localhost:9100"},  
"annotations": {"summary": "The service no longer responds", "severity": "High", "description": "The node_exporter service is stopped or the server is switched off"},  
"startsAt": "2019-04-26T06:28:39.456891295+05:30",  
"endsAt": "0001-01-01-01T00:00:00:00Z",         
"GeneratorUrl": "http://test"}   
]}'

The resolution time does not appear in the alert

Test : Default resolved (has a real end date - endsAt)

curl -X POST http://192.168.1.12:4050/services/hooks/EDFEERdD --data-binary {  
"status": "firing",  
"alerts":[{"labels" : {"alertname": "InstanceDown", "instance" : "localhost:9100"},  
 "annotations": {"summary": "The service no longer responds", "severity": "High", "description": "The node_exporter service is stopped or the server is switched off"},  
"startsAt": "2019-04-26T06:28:39.456891295+05:30",  
"endsAt": "2019-04-26T07:28:39.456891295+05:30",         
"GeneratorUrl": "http://test"}   
]}'

The resolution time appears in the alert

Enjoy!!!!!

Complements for production

If you start several go-neb services for the same Matrix instance, you will find that each call to the Go-neb service will be executed as many times as you have active go-neb services. If you want to set up high availability for this service, set up a heartbeat system, with ipfailover.

Document licence : Creative Commons (CC BY-NC-ND 4.0)

THIS DOCUMENTATION IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND AND DISTRIBUTED FOR EDUCATIONAL PURPOSES ONLY. THE AUTHOR, CONTRIBUTORS TO THIS DOCUMENTATION OR ©MYTINYDC.COM SHALL IN NO EVENT BE LIABLE FOR ANY DIRECT OR INDIRECT DAMAGE THAT MAY RESULT FROM THE APPLICATION OF THE PROCEDURES IMPLEMENTED IN THIS DOCUMENTATION, OR FROM THE INCORRECT INTERPRETATION OF THIS DOCUMENT.

(**) Translated with www.DeepL.com/Translator

Post a comment

Sorry, the answer to the Captcha question is incorrect
Thank you. The Mailman is on His Way :)
Sorry, don't know what happened. Please to try later :(
One field is not valid

Comments (not translated)

No comments