By default, every second we send to the GUI the collected statistics by
the daemon: metrics + events (connections).
This call has a hard-coded timeout of one second, and if it reaches the
deadline, a warning message is printed to the log.
After several timeouts, the daemon closes the connection and tries to
reconnect again.
The daemon invokes this call even if there're no statistics to send.
With one daemon and one GUI, it's not much of a problem. But if there're
n nodes, the GUI will receive a minimum of n connections every second.
In many scenarios, these empty calls are unnecessary.
So, in order to optimize this behaviour, if the daemon has not collected
new statistics, we won't invoke this call. This will slightly reduce the
load on the GUI (server) when managing multiple nodes, and will save
some CPU cycles on the daemon side.
The connection status between the daemon and the GUI is managed with the
following gRPC options:
- The server and the daemon configures a 5-seconds keepalive to keep
the channel open:
'grpc.keepalive_time_ms', 5000
- If after 5 seconds the keepalive msg is not received, then after 20
seconds the connection with the daemon will be closed if the server
doesn't receive a response:
'grpc.keepalive_timeout_ms', 20000
So after about 30 seconds, if the daemon can't communicate with the GUI
(server), it'll close the connection and retry it every second.
Sometimes, when closing the notifications channel, we could end up
trying to reply to a notification after the channel was already
closed. For example when changing the server address of a node.
In that scenario, the goroutine never finished and attempted to
reply to the notification.
When sending the events to the GUI (server), there's a hardcoded timeout
of one second. If the call takes longer, we were writing a warning to
the logs for each call, which in some cases could be excessive.
Hitting the timeout can be normal under some circumstances, such as when
connecting to a GUI with multiple nodes and a lot of activity.
More info: #1453
We have a hardcoded timeout of one second to send the daemon stats to
the GUI. In some cases it's not unusual to hit the deadline.
So in order to just measure the network call, we now initiate the
timeout after collecting the daemon statistics.
See: #1453
Some kernels or systems may not have tracefs mounted or available
by default.
We need the path /sys/kernel/tracing to be mounted, in order to add
syscalls ebpf probes.
So from now on we'll try to mount /sys/kernel/tracing if it's not
already mounted.
-check-requirements parameter updated to check if this option is
available.
Closes#1450.
added tools name as constants, and explain how to use them.
The name of a tool identifies an instance of that tool. Each tool has
its own configuration and logic.
In order to create multiple instances of a tool (for example to run
multiple commands or scripts), just append details to the tool name:
"name": "script",
"cmd": ["/usr/bin/clamscan"]
"name": "script-unhide",
"cmd": ["/usr/sbin/unhide"]
Preliminary version of a task for scanning IOCs in background
(Indicator Of Compromise).
It can execute scripts and commands in background periodically.
- Added a new task scheduler to configure when the task is launched.
Previously we only had an "Interval" field to run the task
periodically. Now it's possible to select the day of the week, specify
multiple times, hours or minutes.
"schedule": [
{
"weekday": [5,6],
"time": ["09:55:00", "20:15:20"],
},
{
"weekday": [0,1,2,3,4],
"time": ["15:00:00"],
}
],
Used only in this task for now.
- This task supports 4 "tools": yara, scripts and debsums/dpkg.
* The "yara" tool acts as a frontend to the yara cli tool, used to scan
for IOCs with YARA rules.
* The "debsums"/"dpkg" tools can be used to:
- report changes in the MD5 checksums of installed Debian
packages (debsums -c).
- report changes in configuration files of installed Debian
packages (debsums -ce).
- verify the integrity of Debian packages (dpkg --verify, dpkg
--verify <package1> <packag2>).
The output of the format (rpm) is not parsed yet.
Configuration example of the task to report the output of the command
debsums:
{
"name": "IOC-scanner",
"data": {
"interval": "15s",
"schedule": [
{
"weekday": [0,1,2,3,4,5,6],
"time": ["22:00:00"]
}
],
"tools": [
{
"name": "debsums",
"msgStart": "IOC scanner debsums started",
"msgEnd": "IOC scanner debsums finished",
"enabled": false,
"cmd": ["debsums", "-c"],
"options": {
"dirs": [],
"files": [],
"reports": {
"path": "/etc/opensnitchd/tasks/iocscanner/reports",
"format": ""
}
}
}
]
}
The field "name" defines which tool to use. It's also used to parse and
format the output of the specified command.
The output of the command is sent to the GUI as an alert, and the
results can be reviewed under the Rules tab -> Alerts.
Restrict permissions to 127.0.0.1 and ::1 instead of the whole loopback
range.
This way, we allow localhost connections from common processes (dirmngr,
kdeinit5, xbrlapi, etc.), while giving users control to allow or deny
DNS queries to the local DNS stub resolver (which for example usually
listens on 127.0.0.53).
More info: #1416.
Added a new task to schedule files downloads.
Whenever the task is loaded, the files will be downloaded. From that on,
files will be downloaded according to the duration specified in the
"interval" field.
The format of the interval field is time.Duration:
https://pkg.go.dev/time#Duration
configuration example:
/etc/opensnitchd/tasks/tasks.json:
{
"tasks":[
{
"enabled": false,
"name": "downloader",
"configfile": "/etc/opensnitchd/tasks/downloader/downloader.json"
}
]
}
/etc/opensnitchd/downloader/downloader.json:
{
"name": "downloader",
"parent": "downloader",
"description": "",
"data": {
"interval": "6h",
"timeout": "5s",
"urls": [
{
"name": "adaway",
"enabled": true,
"remote": "https://adaway.org/hosts.txt",
"localfile": "/tmp/bl3/ads-adaway-hosts.txt"
}
],
"notify": {
"enabled": true
}
}
Now it's possible to load tasks from disk.
There's a central file which contains the defined tasks:
/etc/opensnitchd/tasks/tasks.json
with this format:
{
"tasks": [
{
"enabled": false,
"name": "downloader",
"configfile":
"/etc/opensnitchd/tasks/downloader/downloader.json",
}
]
}
The field "name" defines the type of task to load, defined in each task.
It's static.
Whenever the tasks.json is modified, if the tasks are enabled, the task
configuration file defined in the "configfile" field will be loaded.
Configuration format of each task:
{
"name": "downloader",
"parent": "downloader",
"data": {}
}
The field "name" is dynamic, unique, and identifies the instance of the
task. If there're two tasks configured with the same name, only one will
be loaded.
The "parent" field is used to know what's the base task. If it's not
specified, we'll try to use the "name" as the base task.
This allows you to run multiple instances of the same task with
different configurations: a downloader to update malware blocklists,
another to update IP blocklists, etc.
The "data" field depends of each task. It's generic, and each task is
responsible for defining it. For example for the Downloader task:
"data": {
"interval": "1h",
"timeout": "5s",
"urls": [
{
"name": "adaway",
"enabled": true,
"remote": "https://adaway.o/hosts.txt",
"localfile": "/tmp/blocklist/ads-adaway-hosts.txt"
}
]
}
The tasks configuration file are also monitored for changes, and
whenever a change is detected the task is reloaded.
----
Notifications (WIP)
It's not yet fully defined and it'll change, but this is the state as
of today.
For real-time tasks, like pid-monitor, sockets monitor and node monitor,
the GUI initiates the task with a unique ID (a timestamp), to know who
is sending what, and display the data properly on the GUI.
For permanent tasks we use a unique ID defined for each task.
Right now, if we don't have the notifications channel opened and the
notification ID corresponds with one of the tasks, we'll post an alert
to the GUI, to display a desktop notification.
Each task can define if send notifications or not. For example:
{
"name": "downloader",
"data": {
(...)
"notify": {
"enabled": true
}
}
}
then in each task, act accordingly to what is defined in the
configuration.
- distribute daemon/data/rules/ with the daemon packages.
- added a new system fw rule to allow localhost connections (disabled
by default).
Closes: #965.
When replacing a process with the same PID (fork/clone),
and ebpf.QueueEventsSize is > 0 we may receive out or order events.
For example if the checksum calculation takes too much time.
Now we avoid updating the process in this scenario.