Preliminary version of a task for scanning IOCs in background
(Indicator Of Compromise).
It can execute scripts and commands in background periodically.
- Added a new task scheduler to configure when the task is launched.
Previously we only had an "Interval" field to run the task
periodically. Now it's possible to select the day of the week, specify
multiple times, hours or minutes.
"schedule": [
{
"weekday": [5,6],
"time": ["09:55:00", "20:15:20"],
},
{
"weekday": [0,1,2,3,4],
"time": ["15:00:00"],
}
],
Used only in this task for now.
- This task supports 4 "tools": yara, scripts and debsums/dpkg.
* The "yara" tool acts as a frontend to the yara cli tool, used to scan
for IOCs with YARA rules.
* The "debsums"/"dpkg" tools can be used to:
- report changes in the MD5 checksums of installed Debian
packages (debsums -c).
- report changes in configuration files of installed Debian
packages (debsums -ce).
- verify the integrity of Debian packages (dpkg --verify, dpkg
--verify <package1> <packag2>).
The output of the format (rpm) is not parsed yet.
Configuration example of the task to report the output of the command
debsums:
{
"name": "IOC-scanner",
"data": {
"interval": "15s",
"schedule": [
{
"weekday": [0,1,2,3,4,5,6],
"time": ["22:00:00"]
}
],
"tools": [
{
"name": "debsums",
"msgStart": "IOC scanner debsums started",
"msgEnd": "IOC scanner debsums finished",
"enabled": false,
"cmd": ["debsums", "-c"],
"options": {
"dirs": [],
"files": [],
"reports": {
"path": "/etc/opensnitchd/tasks/iocscanner/reports",
"format": ""
}
}
}
]
}
The field "name" defines which tool to use. It's also used to parse and
format the output of the specified command.
The output of the command is sent to the GUI as an alert, and the
results can be reviewed under the Rules tab -> Alerts.
Restrict permissions to 127.0.0.1 and ::1 instead of the whole loopback
range.
This way, we allow localhost connections from common processes (dirmngr,
kdeinit5, xbrlapi, etc.), while giving users control to allow or deny
DNS queries to the local DNS stub resolver (which for example usually
listens on 127.0.0.53).
More info: #1416.
Added a new task to schedule files downloads.
Whenever the task is loaded, the files will be downloaded. From that on,
files will be downloaded according to the duration specified in the
"interval" field.
The format of the interval field is time.Duration:
https://pkg.go.dev/time#Duration
configuration example:
/etc/opensnitchd/tasks/tasks.json:
{
"tasks":[
{
"enabled": false,
"name": "downloader",
"configfile": "/etc/opensnitchd/tasks/downloader/downloader.json"
}
]
}
/etc/opensnitchd/downloader/downloader.json:
{
"name": "downloader",
"parent": "downloader",
"description": "",
"data": {
"interval": "6h",
"timeout": "5s",
"urls": [
{
"name": "adaway",
"enabled": true,
"remote": "https://adaway.org/hosts.txt",
"localfile": "/tmp/bl3/ads-adaway-hosts.txt"
}
],
"notify": {
"enabled": true
}
}
Now it's possible to load tasks from disk.
There's a central file which contains the defined tasks:
/etc/opensnitchd/tasks/tasks.json
with this format:
{
"tasks": [
{
"enabled": false,
"name": "downloader",
"configfile":
"/etc/opensnitchd/tasks/downloader/downloader.json",
}
]
}
The field "name" defines the type of task to load, defined in each task.
It's static.
Whenever the tasks.json is modified, if the tasks are enabled, the task
configuration file defined in the "configfile" field will be loaded.
Configuration format of each task:
{
"name": "downloader",
"parent": "downloader",
"data": {}
}
The field "name" is dynamic, unique, and identifies the instance of the
task. If there're two tasks configured with the same name, only one will
be loaded.
The "parent" field is used to know what's the base task. If it's not
specified, we'll try to use the "name" as the base task.
This allows you to run multiple instances of the same task with
different configurations: a downloader to update malware blocklists,
another to update IP blocklists, etc.
The "data" field depends of each task. It's generic, and each task is
responsible for defining it. For example for the Downloader task:
"data": {
"interval": "1h",
"timeout": "5s",
"urls": [
{
"name": "adaway",
"enabled": true,
"remote": "https://adaway.o/hosts.txt",
"localfile": "/tmp/blocklist/ads-adaway-hosts.txt"
}
]
}
The tasks configuration file are also monitored for changes, and
whenever a change is detected the task is reloaded.
----
Notifications (WIP)
It's not yet fully defined and it'll change, but this is the state as
of today.
For real-time tasks, like pid-monitor, sockets monitor and node monitor,
the GUI initiates the task with a unique ID (a timestamp), to know who
is sending what, and display the data properly on the GUI.
For permanent tasks we use a unique ID defined for each task.
Right now, if we don't have the notifications channel opened and the
notification ID corresponds with one of the tasks, we'll post an alert
to the GUI, to display a desktop notification.
Each task can define if send notifications or not. For example:
{
"name": "downloader",
"data": {
(...)
"notify": {
"enabled": true
}
}
}
then in each task, act accordingly to what is defined in the
configuration.
With PyQt5 we used plupdate5 + *.pro file to update the translations
file with new strings, extracted from the source code and UI forms.
In PyQt6, pylupdate6 doesn't support parsing .pro files, so we need to do it
manually.
Debian's QtDesigner 6.8.2 defines horizontal/vertical spacers as
Qt::Orientation::Horizontal, but the Qt definitions since 6.5 is
Qt::Horizontal: https://doc.qt.io/qt-6.5/qt.html#Orientation-enum
Due to this problem, the GUI fails to load on some Qt versions, like
6.6.x (Mint 22.1). It doesn't fail on Arch with PyQt6 6.9 though.
Discussion: #1222
unicode-slugify is basically abandoned upstream. Its repository [1] has seen
the last actual code change in 2018. It uses the deprecated `nose` testing framework.
In nixpkgs, we patched it to use pytest, but made some mistakes in those patches,
resulting in no tests actually executing. I actually fixed the tests to run,
but most of the tests fail: unicode does not reliably get replaced with a slugified version.
There does exist an actual need for slugification. However, the used slugify
package should actually work and be reasonably maintained. The replacement proposed here is
`python-slugify` [2], which sees active development in its repository while the
used API (both import name and the basic function signature `slugify(str)`) is identical.
The version requirement of >=7.0.0 is from their official support matrix for py >= 3.7,
though i did my testing with python-slugify 8.0.4.
[1] https://github.com/mozilla/unicode-slugify
[2] https://pypi.org/project/python-slugify/
This should simplify the search for the lrelease binary:
lrelease - Debian/Ubuntu (?)
lrelease6 - openSUSE
lrelease-qt6 - Fedora
In addition, you can specify the path to an lrelease binary with LRELEASE, e.g.:
export LRELEASE="/usr/local/bin/my-lrelease-v6"
The GUI has been ported to PyQt6.
New dependencies:
- python3-pyqt6
- python3-pyqt6.qtsvg (otherwise, the svg icons are not loaded)
On Arch and other systems qt6-svg is neeeded.
- pyqt6-dev-tools (for uic calls, .ui loading).
- libqt6sql6
Known issues:
- pylupdate6 not longer update .pro files.
It doesn't affect existing translations.
- There might be a delay closing the GUI. Reason still unknown.
Closes: #1009
Discussion: #1222
- distribute daemon/data/rules/ with the daemon packages.
- added a new system fw rule to allow localhost connections (disabled
by default).
Closes: #965.
When the daemon connects to the GUI, the firewall dialog is configured
based on the system firewall configuration (inbound/outbpund policies).
Due to an error, a fw-change notification was being triggered, causing
unnecessary firewall rules reloads.
Allow to use commas to specify multiple data:
- src/dst ip: 1.2.3.4, 5.6.7.8
- dst host: www.a-b-c.org, www.abcd.org
- proto: tcp, udp
- net ifaces: eth0, eth1
- src/dst port: 123, 456
The data will be translated to ^(..|..|..)$ when saving the rule, and
back to "..,..,.." when loading it.
Closes: #1392.
When replacing a process with the same PID (fork/clone),
and ebpf.QueueEventsSize is > 0 we may receive out or order events.
For example if the checksum calculation takes too much time.
Now we avoid updating the process in this scenario.