(**) Translated with www.DeepL.com/Translator
Update of 19/05/2019 - This procedure also works for a Raspberry PI3 (armhf)
This bot allows the integration of many features, including Giphy (animated Gifs) and the extension of the alert system of the Prometheus monitoring application. **This instruction can also be applied to amd64 (PC) platforms.
The “go-neb” project is hosted herehttps://github.com/matrix-org/go-neb, the documentation clearly explains how to install the service. However, the compilation did not proceed as indicated.
Compilation
I encountered two problems: the package included in the “mattn/go-sqlite3” distribution is impossible to compile and indicates this type of error:
github.com/matrix-org/go-neb/api/handlers
sqlite3-binding.c: In function'wherePathSolver':
sqlite3-binding.c:130692:1: internal compiler error: in schedule_block, at haifa-sched.c:6593
}
so I installed the DEBIAN golang-github-mattn-go-sqlite3-dev package as a replacement.
The second problem is located at the level of an “include” that the builder “go” does not find (detected by running the command :
go build /opt/go-neb/vendor/src/github.com/mattn/go-sqlite3/sqlite3.go
and presents the error:
tmp/go-build020128959/command-line-arguments/_obj/sqlite3.cgo2.o: In function `_sqlite3_bind_blob': vendor/src/github.com/mattn/go-sqlite3/sqlite3.go:44: undefined reference to`sqlite3_bind_blob
/tmp/go-build020128959/command-line-arguments/_obj/sqlite3.cgo2.o: In function `_sqlite3_bind_text': vendor/src/github.com/mattn/go-sqlite3/sqlite3.go:39: undefined reference to`sqlite3_bind_text
Looking at the code, it mentions an include
#include <sqlite3-binding.h>
which also uses the include:
#include <sqlite3.h>
not present in the package, but available through the DEBIAN libsqlite3-dev package. I think it’s just a problem with setting the includes search path. For ease of use, I chose to copy the missing file into the distribution package.
Preparation
apt install go-lang libsqlite3-dev golang-github-mattn-go-sqlite3-dev git
cd /opt
git clone https://github.com/matrix-org/go-neb
export GOPATH=/usr/lib/go-1.7/
export PATH=$PATH:$GOPATH/bin
go get github.com/constabulary/gb/....
cd go-neb
rm -r vendor/src/github.com/mattn/go-sqlite3
cp -R /usr/share/gocode/src/github.com/mattn/go-sqlite3 vendor/src/github.com/mattn/
cp /usr/include/sqlite3.h ./vendor/src/github.com/mattn/go-sqlite3/
Project compilation
gb build github.com/matrix-org/go-neb
The go-neb binary has been created in the “bin” directory of the project.
Directory of the sqlite3 database
Create the db directory
mkdir db
Create an installer as an archive (tar.gz)
cd /opt/go-neb
tar cfz /root/go-neb.tgz /opt/go-neb/bin /opt/go-neb/db
You can now install this archive on any arm64 server:
Copy the archive into the directory** /root/** of the server
cd /
tar xfz /root/go-neb.tgz
rm /root/go-neb.tgz
File for the systemd service
This service will communicate with the Synapse-Matrix service. To make it simple, I installed the service on the server that hosts the Synapse-Matrix service.
It listens on the port: TCP/4050 at the IP address: localhost.
The database used by this service is of type sqlite3 and stored in the directory /opt/go-neb/db. These settings can be customized in the following file.
After installation, if you need to modify this file, always remember to execute the command :
systemctl daemon-relaod
#Create the group and use execution
addgroup go-neb
adduser --system --home /opt/go-neb/ --no-create-home --disabled-password --shell /bin/nologin --ingroup go-neb go-neb
#Changing the owner of the installation directory
chown -R go-neb:go-neb /opt/go-neb
#Create the file for systemd
vi /etc/systemd/system/system/go-neb.service
and insert these instructions:
[Unit]
Description=Go-neb Bot For Synapse-Matrix server
After=network.target
[Service]
WorkingDirectory=/opt/go-neb
ExecStart=/opt/go-neb/bin/go-neb
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=go-neb
User=go-neb
Group=go-neb
Environment=BIND_ADDRESS=localhost:4050 DATABASE_TYPE=sqlite3 DATABASE_URL=db/go-neb.db?_busy_timeout=5000 BASE_URL=http://localhost:4050
Restart=always
RestartSec=10
StartLimitInterval=900
StartLimitBurst=3
[Install]
WantedBy=multi-user.target
Starting the service
systemctl start go-neb
# Check the launch
tail -f /var/log/syslog
Sample answer :
After 25 10:03:40 localhost systemd[1]: Started Go-neb Bot For Synapse-Matrix server.
After 25 10:03:40 localhost go-neb[5662]: time="2019-04-25T10:03:40+05:30" level=info msg="Go-NEB ({BindAddress:localhost:4050 DatabaseType:sqlite3 DatabaseURL:db/go-neb.db?_busy_timeout=5000 BaseURL:http://localhost:4050 LogDir: ConfigFile:})"
The service is started and listened to on port 4050 of the localhost interface:
#Audit
netstat -ant|grep 4050
Activation of the service at the server boot
systemctl enable go-neb
Bot setting
The bot must be able to connect to the Synapse-Matrix server. Create a Synapse-Matrix service access account, adapting the settings to your configuration:
/opt/synapse/env/bin/register_new_matrix_user -c /opt/synapse/homeserver.yaml http://matrix.mondomaine.com
I used the account “botgoneb” with a strong password, not admin.
To obtain the token of this new user, use the following command, adapting it to your configuration:
curl -X POST "http://matrix.mondomaine.com/_matrix/client/r0/login" -d "type": "m.login.password", "user": "botgoneb", "password":"[Password previously entered]" }''
Response of the command if it is successful:
{
"access_token":"[USER TOKEN]",
"device_id": "XXXXXXXX",
"home_server": "matrix.mydomain.com",
"user_id": "@botgoneb:matrix.mydomain.com"
}
The TOKEN of this new user is the value corresponding to the key “access_token” of the answer.
Connect to the Synapse-Matrix server and execute the following command, it tells the “go-neb” service how to connect to the Synapse-Matrix service
curl -X POST localhost:4050/admin/configureClient --data-binary'{
"UserID": "@botgoneb:matrix.mydomain.com",
"HomeserverURL": "http://matrix.mondomaine.com",
"AccessToken": "<TOKEN obtained previously>",
"Sync": true,
"AutoJoinRooms": true,
"DisplayName": "Bot goneb"
}'
The command is executed normally and returns a value in JSON format.
I activate the “echo” function of the “go-neb” service for the test
curl -X POST localhost:4050/admin/configureService --data-binary'{"Type": "echo", "Id": "00001", "UserID": "@botgoneb:matrix.mydomain.com", "Config": {}' }'
The command returns a value in JSON format, the function is activated.
Log in to the Element.io client, create a lounge and invite the bot: @botgoneb:matrix.mydomain.com**, his reaction is immediate, he is connected. Type in this room the message:
!echo hello world
The bot responds by indicating the text entered after the"!echo" command.
Giphy Integration - Synapse-Matrix Server
Enabling the function: Connect to the console of the server with the “go-neb” service, and type the command :
curl -X POST localhost:4050/admin/configureService --data-binary'{
"Type": "giphy",
"Id": "00002",
"UserID": "@botgoneb:matrix.mydomain.com",
"Config": {"api_key": "dc6zaTOxFJmzC"}
}'
I advise you to register with Giphy (https://developers.giphy.com/) by creating an application (“Create app”). This allows you to obtain an API key. The key indicated above is the Giphy public key, with a daily usage quota. When the quota is reached, the Giphy server does not send anything back.
Invite the bot to the lounge of your choice, it connects immediately, and use the message: !giphy[search word] Example : !giphy test
the bot returns the first Gif image returned by Giphy.
Element.io allows you to disable the automatic start of Gifs, check your account settings.
AlertManager/Prometheus monitoring integration - Synapse-Matrix Server
This function allows the AlertManager service (part of the Prometheus project - Monitoring system developed by Soundcloud - installation[click here] (https://www.mytinydc.com/index.php/2018/12/26/installer-serveur-monitoring-prometheus-grafana/)), to send alerts to the Synapse-Matrix system. This part is not documented on the web, after several hours of source code analysis concerning the bot, and testing, this part is now functional.
Prometheus communications flow / Alertmanager / go-neb / Synapse-Matrix / Element.io user
To separate the functions of each bot, I create a new user on Synapse-Matrix: @botPrometheus:matrix.mydomain.com. I get his access token and add it to the go-neb system (see above).
I create the “Monitoring” lounge, intended to receive the messages of the bot: “#Monitoring:matrix.mydomain.com”.
Retrieve the internal ID of this room, using the Element.io client: Select the room, enter “Setting”, “Advanced”, the internal ID appears as: !qmElAGdFYCHoCJuaNt:matrix.mydomain.com
Changing the listening address of the go-neb service
The original configuration of the “go-neb” service, indicates a listening at the address “localhost”. The “Alertmanager” service, located on another server, cannot therefore join the “go-neb” service. So I will change the configuration of the “go-neb” service so that it listens on the network interface connected to the local network.
vi /etc/systemd/system/system/go-neb.service
#Change the value of the environment variables: Environment=BIND_ADDRESS=localhost:4050 DATABASE_TYPE=sqlite3 DATABASE_URL=db/go-neb.db?_busy_timeout=5000 BASE_URL=http://localhost:4050
Replace “localhost” with the IP address of this server :
Environment=BIND_ADDRESS=192.168.168.1.12:4050 DATABASE_TYPE=sqlite3 DATABASE_URL=db/go-neb.db?_busy_timeout=5000 BASE_URL=http://192.168.1.12:4050
Save and exit vi
Update systemd and restart go-neb :
systemctl daemon-reload
systemctl restart go-neb
Check with :
netstat -ant|grep 4050
**Add the Firewall rules on the interface concerned: INPUT TCP/4050 and filter only the server with the Alertmanager service.
The “Alertmanager” service can now join the “go-neb” service.
Enabling the go-neb alertmanager function
This function is activated using the following curl command:
curl -X POST http://192.168.1.12:4050/admin/configureService --data-binary {
"Type": "alert manager",
"Id": "00003",
"UserID": "@botprometheus:matrix.mydomain.com",
"Config": {"rooms":
{"!qmElAGdFYCHoCJuaNt:matrix.mydomain.com":
{
"text_template": "Prometheus Notification[{{.Status}}]",
"html_template":"<h1>Notification Prometheus[{{.Status}}]</h1><hr/>{- range.Alerts}}}<div>Alerte : {{{.Labels.alertname}}}<br/>Instance : {{{.Labels.instance}}}<br/>Summary : {{{.Annotations.severity}}<br/>Description: {{.Annotations.description}}<br/>Date of detection : {{.StartsAt}}<br/>{{{if ne .EndsAt \"0001-01-01-01T00:00:00:00Z\" -}}Date and time resolution: {{.EndsAt}}<br/>{- end }}Url : {{{.GeneratorUrl}}</div>{{- end}}", "msg_type": "m.text"}}}}''.
The parameters “text_template” and “html_template” are of type “text” whose form must respect the syntax concerning the “Golang” templates.
I created these templates based on the Javascript source code used for the “Alertmanager” integration of the “Rocketchat” instant messaging service (See installation here).
The “Golang” templating system allows you to create loops, include conditions, etc., just like other systems such as “Ejs for Javascript”, or “Twig” for PHP.
Warning, I couldn’t insert the “style” parameters in the HTML tags, if someone wants to start… Data formatting is not supported ( e. g. StartsAt | AsDate format).
This tutorial helped me a lot: http://goinbigdata.com/example-of-using-templates-in-golang/
Two “go-neb” templates for Alermanager/Prometheus are required:
- The template “text_template” is used for the system notification, the small popup that appears when the Element.io application receives a message.
- The template “html_template” is used to build the body of the message as you see it appear in the living room.
**Attention the server will return the webhook address, which will be used by Alertmanager.
Example of data obtained (JSON format):
{"ID": "00003", "Type": "alertmanager", "OldConfig":{}, "NewConfig":{"webhook_url": "http://localhost:4050/services/hooks/EDFEERdDM", "rooms":......
The address to be provided to Alertmanager’s webhook systems will be in this case: http://localhost:4050/services/hooks/EDFEERdD
Changing the Alertmanager configuration
I indicate to Alertmanager the address of the webhook to reach (url given by the activation of the “alertmanager” service of go-neb. Edit the Alertmanager configuration file (/opt/alertmanager-[version]/alertmanager.yml), adding in the “receiver” block, the “webhook_config” block:
Attention YAML file, do not use tabs but spaces, to indent lines.
[..]
receivers:
- name: "email
email_configs:
- send_resolved: true
to:
[..]
webhook_configs:
- send_resolved: true
url: "**http://localhost:4050/services/hooks/EDFEERdDM**''
Reload the Alertmanager configuration:
systemctl reload prometheus-alertmanager
Test the notification
Execute the following commands, adapting them to your settings, and from the server that hosts the AlertManager service (due to Firewall rules).
You will immediately receive a message in the “#Monitoring:matrix.mydomain.com” room.
Test : Detection of a defect
curl -X POST http://192.168.1.12:4050/services/hooks/EDFEERdD --data-binary "{
"status": "firing",
"alerts":[{"labels" : {"alertname": "InstanceDown", "instance" : "localhost:9100"},
"annotations": {"summary": "The service no longer responds", "severity": "High", "description": "The node_exporter service is stopped or the server is switched off"},
"startsAt": "2019-04-26T06:28:39.456891295+05:30",
"endsAt": "0001-01-01-01T00:00:00:00Z",
"GeneratorUrl": "http://test"}
]}'
The resolution time does not appear in the alert
Test : Default resolved (has a real end date - endsAt)
curl -X POST http://192.168.1.12:4050/services/hooks/EDFEERdD --data-binary {
"status": "firing",
"alerts":[{"labels" : {"alertname": "InstanceDown", "instance" : "localhost:9100"},
"annotations": {"summary": "The service no longer responds", "severity": "High", "description": "The node_exporter service is stopped or the server is switched off"},
"startsAt": "2019-04-26T06:28:39.456891295+05:30",
"endsAt": "2019-04-26T07:28:39.456891295+05:30",
"GeneratorUrl": "http://test"}
]}'
The resolution time appears in the alert
Enjoy!!!!!
Complements for production
If you start several go-neb services for the same Matrix instance, you will find that each call to the Go-neb service will be executed as many times as you have active go-neb services. If you want to set up high availability for this service, set up a heartbeat system, with ipfailover.
Document licence : Creative Commons (CC BY-NC-ND 4.0)
THIS DOCUMENTATION IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND AND DISTRIBUTED FOR EDUCATIONAL PURPOSES ONLY. THE AUTHOR, CONTRIBUTORS TO THIS DOCUMENTATION OR ©MYTINYDC.COM SHALL IN NO EVENT BE LIABLE FOR ANY DIRECT OR INDIRECT DAMAGE THAT MAY RESULT FROM THE APPLICATION OF THE PROCEDURES IMPLEMENTED IN THIS DOCUMENTATION, OR FROM THE INCORRECT INTERPRETATION OF THIS DOCUMENT.