Sometimes, when closing the notifications channel, we could end up
trying to reply to a notification after the channel was already
closed. For example when changing the server address of a node.
In that scenario, the goroutine never finished and attempted to
reply to the notification.
Now it's possible to load tasks from disk.
There's a central file which contains the defined tasks:
/etc/opensnitchd/tasks/tasks.json
with this format:
{
"tasks": [
{
"enabled": false,
"name": "downloader",
"configfile":
"/etc/opensnitchd/tasks/downloader/downloader.json",
}
]
}
The field "name" defines the type of task to load, defined in each task.
It's static.
Whenever the tasks.json is modified, if the tasks are enabled, the task
configuration file defined in the "configfile" field will be loaded.
Configuration format of each task:
{
"name": "downloader",
"parent": "downloader",
"data": {}
}
The field "name" is dynamic, unique, and identifies the instance of the
task. If there're two tasks configured with the same name, only one will
be loaded.
The "parent" field is used to know what's the base task. If it's not
specified, we'll try to use the "name" as the base task.
This allows you to run multiple instances of the same task with
different configurations: a downloader to update malware blocklists,
another to update IP blocklists, etc.
The "data" field depends of each task. It's generic, and each task is
responsible for defining it. For example for the Downloader task:
"data": {
"interval": "1h",
"timeout": "5s",
"urls": [
{
"name": "adaway",
"enabled": true,
"remote": "https://adaway.o/hosts.txt",
"localfile": "/tmp/blocklist/ads-adaway-hosts.txt"
}
]
}
The tasks configuration file are also monitored for changes, and
whenever a change is detected the task is reloaded.
----
Notifications (WIP)
It's not yet fully defined and it'll change, but this is the state as
of today.
For real-time tasks, like pid-monitor, sockets monitor and node monitor,
the GUI initiates the task with a unique ID (a timestamp), to know who
is sending what, and display the data properly on the GUI.
For permanent tasks we use a unique ID defined for each task.
Right now, if we don't have the notifications channel opened and the
notification ID corresponds with one of the tasks, we'll post an alert
to the GUI, to display a desktop notification.
Each task can define if send notifications or not. For example:
{
"name": "downloader",
"data": {
(...)
"notify": {
"enabled": true
}
}
}
then in each task, act accordingly to what is defined in the
configuration.
- Fixed configuration loading, when the Process Monitor Method
selected failed to work (e.g.: when ebpf modules are missing,
auditd is not configured, etc).
In this scenario, on the first run, the daemon was not loading the
configuration from disk, and was instead using the default values, which
led to some glitches (unexpected reconnections to the GUI, checksums
option ignored, etc).
- added more tests.
Added new task to monitor the resources of remote nodes, like
ram, swap, number of processes or load average of the system.
The task is initiated when the user selects a node, and the data
received from the node is added to the right panel of the Nodes tab.
The task is stopped when changing to another tab, or when deselecting a
node.
Particularly useful for monitoring remote nodes.
daemon tasks are actions that are executed in background by the daemon.
They're started from the GUI (server) via a Notification (protobuf),
with the type TASK_START (protobuf).
Once received in the daemon, the TaskManager starts the task in
background.
Tasks may run at interval times (every 5s, 2days, etc), until they
finish an operation, until a timeout, etc.
Each task has each own configuration options, which will customize the
behaviour of its operations.
In this version, if the GUI is closed, the daemon will stop all the
running tasks.
Each Task has a flag to ignore this behaviour, for example if they need
to run until they finish and only send a notification to the GUI,
instead of streaming data continuously to the GUI (server).
- Up until now we only had one task that could be initiated from the GUI:
the process monitor dialog. It has been migrated to a Task{}.
- go.mod bumped to v1.20, to use unsafe string functions.
- go.sum updated accordingly.
Allow to customize:
- EventsWorkers: number of goroutines to handle kernel events.
Default 8.
- QueueEventsSize: max number of events in the queue.
By default 0, meaning that it'll relay on the available goroutines to
process the events. If it's > 0, and the daemon can't process the
events fast enough, they'll be queued. Once the queue is full, it'll
behave as it was of size 0.
If there're lost events, a message will be logged: "Lost ebpf events..."
When disabling the interception from the server (GUI), the network
interception was stopped, but the procs monitor kept running.
Now the procs monitor in use is also stopped, not to interfere with
the rest of the system (except 'proc').
Now it's possible to configure eBPF modules path from the
default-config.json file:
"Ebpf": {
"ModulesPath": "..."
}
If the option is not provided, or if it's empty, we'll keep loading from
the default directories:
- /usr/local/lib/opensnitchd/ebpf
- /usr/lib/opensnitchd/ebpf
- /etc/opensnitchd/ebpf (deprecated, will be removed in the future).
Closes#928
- Allow to configure system firewall configuration file path:
* via cli (-fw-config-file).
* via global configuration file.
- Allow to configure fw rules check interval.
The system fw config file contains regular iptables/nftables rules.
Previously it was hardcoded to /etc/opensnitchd/system-fw.json
The interval to check if the interception rules were added was also
hardcoded to 10 seconds. Now it's possible to configure it.
A value of "0s" disables the interval, while "" defaults to 10 seconds.
- Fixed several leaks.
- Cache of events reorganized and improved.
* items are added faster.
* proc details are rebuilt if needed (checksums, proc tree, etc)
* proc's tree is reused if we've got the parent in cache.
rel: #413
- Obtain the process's parent hierarchy.
- Display the hierarchy on the pop-ups and the process dialog.
- [pop-ups] Added a Detailed view with all the metadata of the
process.
- [cache-events] Improved the cache of processes.
- [ruleseditor] Fixed enabling md5 checksum widget.
Related: #413, #406
Now you can create rules to filter processes by checksum. Only md5 is
available at the moment.
There's a global configuration option that you can use to enable or
disable this feature, from the config file or from the Preferences
dialog.
As part of this feature there have been more changes:
- New proc monitor method (PROCESS CONNECTOR) that listens for
exec/exit events from the kernel.
This feature depends on CONFIG_PROC_EVENTS kernel option.
- Only one cache of active processes for ebpf and proc monitor
methods.
More info and details: #413.
- Send errors to the server (GUI) if there's any error when reloading
the system fw rules (far from being perfect/optimal, needs a
rewrite).
- Don't load the configuration after saving it, let the watcher reload
it on write change to avoid double reload/duplicated errors.
- Configuration of system firewall rules from the GUI is not supported for
iptables. Up until now only a warning was displayed, encouring to change
fw type manually.
Now if configured fw type is iptables (default-config.json, Firewall:),
and the user opens the fw dialog, we'll ask the user to change it from
the GUI.
- Add fw rules before connecting to the GUI. Otherwise we send to the
GUI an invalid fw state.
Improved process detections by monitoring new processes execution.
It allow us to know the path of a process before a socket is opened.
Closes#617
Other improvements:
- If we fail to retrieve the path of a process, then we'll use the comm
name of the connection/process.
- Better kernel connections detection.
- If debugfs is not loaded, we'll try to mount it, to allow to use
eBPF monitor method.
Future work (help wanted):
- Extract command line arguments from the kernel (sys_execve, or mm
struct).
- Monitor other functions (execveat, clone*, fork, etc).
- Send these events to the server (GUI), and display all the commands
an application has executed.
* Allow to configure firewall rules from the GUI (WIP)
New features:
- Configure and list system firewall rules from the GUI (nftables).
- Configure chains' policies.
- Add simple rules to allow incoming ports.
- Add simple rules to exclude apps (ports) from being intercepted.
This feature is only available for nftables. iptables is still supported,
you can add rules to the configuration file and they'll be loaded, but
you can't configure them from the GUI.
More information: #592
There are two issues when connecting to the GUI via TCP on localhost:
- Sometimes when the daemon is launched with the GUI already running, the
notifications channel is not established.
* Give 10 seconds to connect, if it timeouts, disconnect everything
and try to reconnect again.
- After some time the notifications channel is closed from the server
side (i.e., the GUI), and it is not restablished.
* Forcefully disconnect everything, and let it reconnect again.
Both issues should be investigated further to find the root problem.
- Fixed reloading process monitor method if the configuration changes on
disk. This can occur in two situations: 1) if it's changed from the
UI, 2) if the user changes it manually.
- Ensure that we don't crash if there's an error changing the
method and ebpf is active.
- When changing monitor method to ebpf and it fails to start, stop it
anyway. It helps cleaning up kprobes and avoiding the error
"cannot write...: file exists".
When the daemon connects to the GUI, use the default action
configured on the GUI to apply a verdict on new connections.
We were using daemon's default action, so if it was Allow but the user
had configured Deny on the GUI it was allowing connections, causing some
confusion (#489)
TODO: apply this action not only on connecting to the GUI, but also when
saving the configuration from the GUI.
Before this change, we tried to determine what firewall to use based on
the version of iptables (if -V legacy -> nftables, otherwise iptables).
This caused problems (#455), and as there's no support yet for nftables
system firewall rules, it can't be configured to workaround these
errors.
Now the default firewall to use will be iptables.
If it's not available (installed), can't be used or the configuration
option is empty/missing, we'll use nftables.
Added basic nftables support, which adds the needed rules to intercept
outgoing network traffic and DNS responses. System rules will be added
soon.
What netfilter subsystem to use is determined based on the following:
- nftables: if the _iptables_ binary is not present in the system, or
if the iptables version (iptables -V) is
"iptables vX.Y.Z (nf_tables)".
- iptables: in the rest of the cases.
problem:
- after losing network connectivity node<->server, the node didn't restore
the connection. In reality, the connection with the server was not
closed, but the notifications channel was closed due to inactivity
after 20s.
set inactivity timeouts to 20s on both node and server. Previous
timeouts were 2h for the main connection and 20s for the streaming
channels (notifications).
- get rid of the logic to determine if the server is alive or not based
on sending pings.
Instead, use the connection events when a node connects/disconnects
(Subscribe).
The Ping call is still used to send the statistics.
other:
- fixed exception when updating the status of a node.
If we change the monitor method from the GUI, and it fails to start
(eBPF, audit or ftrace), use the old configured method and don't save
the configuration to disk, to avoid configuring a monitor method that
doesn't work.
* Use ebpf program to find PID of new connections.
before running the branch you have to compile ebpf_prog/opensnitch.c
opensnitch.c is an eBPF program. Compilation requires getting kernel source.
cd opensnitch
wget https://github.com/torvalds/linux/archive/v5.8.tar.gz
tar -xf v5.8.tar.gz
patch linux-5.8/tools/lib/bpf/bpf_helpers.h < ebpf_prog/file.patch
cp ebpf_prog/opensnitch.c ebpf_prog/Makefile linux-5.8/samples/bpf
cd linux-5.8 && yes "" | make oldconfig && make prepare && make headers_install # (1 min)
cd samples/bpf && make
objdump -h opensnitch.o #you should see many section, number 1 should be called kprobe/tcp_v4_connect
llvm-strip -g opensnitch.o #remove debug info
sudo cp opensnitch.o /etc/opensnitchd
cd ../../../daemon
--opensnitchd expects to find opensnitch.o in /etc/opensnitchd/
--start opensnitchd with:
opensnitchd -rules-path /etc/opensnitchd/rules -process-monitor-method ebpf
Co-authored-by: themighty1 <you@example.com>
Co-authored-by: Gustavo Iñiguez Goia <gooffy1@gmail.com>
- ui: fixed error getting the icon of an app.
- ui: fixed getting the list of pids of an app.
- ui: improved proc details start/stop icon behaviour.
- daemon: improved error message when we fail getting the details of a
process.
others:
- changed icon search by system-search.
New dialog added to display details of a process in realtime, gathered
from ProcFS.
Process tab -> double click on an app -> click on the button with the
search icon.
We have also improved the discovery of apps icons and names. It should
work better on systems where the DE is not properly configured.
Tested, but not bulletproof, still in beta.
we were not switching between process monitor methods properly, so we're
falling back to proc method in some cases.
Besides, there's seems to be a descriptors leaking problem in ftrace package
when closing resources.
When the daemon is stopped, we need to close opened netfilter recurses.
Otherwise we can fall into a situation where we leave NFQUEUE queues
opened, which causes opensnitch to not run anymore until system restart
or a manual intervention, because there's a NFQUEUE queue already created
with the same ID.
This is what was happening as a collateral effect of #41.