From 0671fa522ba932fe100a279f990ea0178e12d8b8 Mon Sep 17 00:00:00 2001 From: Michael Reber Date: Mon, 23 Feb 2026 11:03:15 +0100 Subject: [PATCH] Update Documentations to include the new alert providers --- README.md | 5 +- docs/alert-providers.md | 222 ++++++++++++++++++++++++++++++++++++++++ docs/api.md | 11 +- docs/architecture.md | 17 +-- docs/configuration.md | 22 +++- docs/security.md | 23 +++++ docs/troubleshooting.md | 73 ++++++++++++- 7 files changed, 361 insertions(+), 12 deletions(-) create mode 100644 docs/alert-providers.md diff --git a/README.md b/README.md index f13f37c..7d0803d 100644 --- a/README.md +++ b/README.md @@ -31,7 +31,7 @@ Fail2Ban UI does not replace Fail2Ban. It connects to existing Fail2Ban instance - Ban Insights with an interactive 3D threat globe showing blocks per country - Advanced ban actions for recurring offenders e.g. automatically ban on pfSense, Mikrotik, or OPNsense when threshold is reached - Data management possibility for permanent block logs and stored ban events -- Optional email alerts with GeoIP/Whois enrichment for selected "alert countries" only +- Configurable alert notifications (Email/SMTP, Webhook, or Elasticsearch) with GeoIP/Whois enrichment and country-based filtering - Optional OIDC login (Keycloak, Authentik, Pocket-ID) - Least-privilege, SELinux-aware container deployment (policies provided) - .. and much more to come. @@ -76,6 +76,7 @@ Next steps: * Security guidance (recommended deployment posture): [`docs/security.md`](https://github.com/swissmakers/fail2ban-ui/blob/main/docs/security.md) * Architecture overview: [`docs/architecture.md`](https://github.com/swissmakers/fail2ban-ui/blob/main/docs/architecture.md) * API reference: [`docs/api.md`](https://github.com/swissmakers/fail2ban-ui/blob/main/docs/api.md) +* Alert providers (Email, Webhook, Elasticsearch): [`docs/alert-providers.md`](https://github.com/swissmakers/fail2ban-ui/blob/main/docs/alert-providers.md) * Troubleshooting: [`docs/troubleshooting.md`](https://github.com/swissmakers/fail2ban-ui/blob/main/docs/troubleshooting.md) Existing deployment guides in this repository: @@ -162,7 +163,7 @@ Configuration for advanced ban actions including permanent blocking, firewall in #### Alert Settings ![Alert Settings](screenshots/4.3_Settings_AlertSettings.png) -Email alert configuration with SMTP settings, country-based filtering (blocks from what country to raport), GeoIP provider selection, and alert preferences for bans and unbans. +Alert configuration supporting three providers: Email (SMTP), Webhook, and Elasticsearch. Includes country-based filtering, GeoIP provider selection, and per-event toggles for bans and unbans. See [`docs/alert-providers.md`](https://github.com/swissmakers/fail2ban-ui/blob/main/docs/alert-providers.md) for details. #### Global Settings ![Global Settings](screenshots/4.4_Settings_GlobalSettings.png) diff --git a/docs/alert-providers.md b/docs/alert-providers.md new file mode 100644 index 0000000..1d61f46 --- /dev/null +++ b/docs/alert-providers.md @@ -0,0 +1,222 @@ +# Alert providers + +Fail2Ban UI can send alert notifications when a ban or unban event occurs. Three providers are available: **Email (SMTP)**, **Webhook**, and **Elasticsearch**. Only one provider can be active at a time. + +All providers share the same global settings: + +- **Enable alerts for bans / unbans** -> master toggles that control whether any alert fires. +- **Alert Countries** -> only events for IPs geolocated to the selected countries trigger alerts. Set to `ALL` to alert on every country. +- **GeoIP Provider** -> determines how country lookups are performed (built-in API or local MaxMind database). +- **Maximum Log Lines** -> limits the number of log lines attached to alert payloads. + +## Email (SMTP) + +The default provider. Sends HTML-formatted emails via a configured SMTP server. + +### Settings + +| Field | Description | +|---|---| +| Destination Email | The recipient address for all alert emails | +| SMTP Host | Mail server hostname (e.g. `smtp.office365.com`) | +| SMTP Port | Common values: 587 (STARTTLS), 465 (implicit TLS), 25 (plain) | +| SMTP Username | Login username for the mail server | +| SMTP Password | Login password or app password | +| Sender Email | The `From:` address on outgoing alerts | +| Authentication Method | `Auto` (LOGIN preferred), `LOGIN`, `PLAIN`, or `CRAM-MD5` | +| Use TLS | Enable TLS encryption (recommended) | +| Skip TLS Verification | Disable certificate validation (not recommended for production) | + +### Email content + +Ban alerts include: IP address, jail name, hostname, failure count, country, whois data, and relevant log lines. The email uses an HTML template with two style variants (`modern` and `classic`, controlled by the `emailStyle` environment variable). + +### Testing + +Click **Send Test Email** in the UI after saving settings. The test email uses the same SMTP path as real alerts, so a successful test confirms the full delivery chain. + +### Notes + +- Office365 and Gmail typically require the `LOGIN` auth method (selected automatically by the `Auto` option). +- Emails include RFC-compliant `Message-ID` and `Date` headers to minimize spam classification. (but can still happen because of the log-content that is sended within a ban-mail.) + +## Webhook + +Sends a JSON payload to any HTTP endpoint. Compatible with ntfy, Matrix bridges, Slack/Mattermost incoming webhooks, Gotify, custom REST APIs, and any system that accepts JSON over HTTP. + +### Settings + +| Field | Description | +|---|---| +| Webhook URL | The target endpoint (e.g. `https://my-ntfy.example.com/fail2ban-alerts`) | +| HTTP Method | `POST` (default) or `PUT` | +| Custom Headers | One per line in `Key: Value` format. Useful for auth tokens, content-type overrides, or ntfy-specific headers like `Title` and `Priority` | +| Skip TLS Verification | Disable certificate validation for self-signed endpoints | + +### Payload format + +Every alert sends the following JSON body: + +```json +{ + "event": "ban", + "ip": "1.2.3.4", + "jail": "sshd", + "hostname": "webserver-01", + "country": "CN", + "failures": "5", + "whois": "...", + "logs": "...", + "timestamp": "2026-02-23T12:00:00Z" +} +``` + +The `event` field is `"ban"`, `"unban"`, or `"test"` (for the test button). + +### ntfy integration example + +ntfy expects either plain text to a topic URL, or its own JSON format to the root URL. The simplest approach: + +1. Set the Webhook URL to the **topic URL**: `https://my-ntfy.example.com/fail2ban-alerts` +2. Add custom headers for better notifications (optional): + ``` + Title: Fail2ban Alert + Priority: high + Tags: rotating_light + ``` + +The JSON payload will appear as the notification body. For ntfy with authentication, add an `Authorization: Bearer ` header. + +### Slack / Mattermost example + +For Slack or Mattermost incoming webhooks, the endpoint expects a `text` field. Since Fail2Ban UI sends a generic JSON payload, use a middleware or Slack workflow to parse it, or use a webhook-to-Slack bridge. + +### Testing + +Click **Send Test Webhook** after saving settings. This sends a test payload (`"event": "test"`) with a dummy IP (`203.0.113.1`) to verify connectivity. + +### Technical details + +- Timeout: 15 seconds per request. +- The `Content-Type` header is always set to `application/json`. +- Custom headers override any default header except `Content-Type`. +- TLS verification can be disabled for self-signed certificates. +- HTTP responses with status >= 400 are treated as errors and logged. + +## Elasticsearch + +Indexes ban/unban events as structured documents into Elasticsearch, using ECS (Elastic Common Schema) field names for native Kibana compatibility. + +### Settings + +| Field | Description | +|---|---| +| Elasticsearch URL | Cluster endpoint (e.g. `https://elasticsearch.example.com:9200`) | +| Index Name | Base name for the index (default: `fail2ban-events`). A daily suffix `-YYYY.MM.DD` is appended automatically. | +| API Key | Base64-encoded API key (preferred authentication). Leave empty for username/password auth. | +| Username | Basic auth username (used when API Key is empty) | +| Password | Basic auth password | +| Skip TLS Verification | Disable certificate validation for self-signed clusters | + +### Document structure + +Each event is indexed to `-YYYY.MM.DD` (e.g. `fail2ban-events-2026.02.23`): + +```json +{ + "@timestamp": "2026-02-23T12:00:00Z", + "event.kind": "alert", + "event.type": "ban", + "source.ip": "1.2.3.4", + "source.geo.country_iso_code": "CN", + "observer.hostname": "webserver-01", + "fail2ban.jail": "sshd", + "fail2ban.failures": "5", + "fail2ban.whois": "...", + "fail2ban.logs": "..." +} +``` + +### Elasticsearch setup + +**1. Create an index template** + +In Kibana Dev Tools or via the API: + +``` +PUT _index_template/fail2ban +{ + "index_patterns": ["fail2ban-events-*"], + "template": { + "settings": { + "number_of_shards": 1, + "number_of_replicas": 0 + }, + "mappings": { + "properties": { + "@timestamp": { "type": "date" }, + "event.kind": { "type": "keyword" }, + "event.type": { "type": "keyword" }, + "source.ip": { "type": "ip" }, + "source.geo.country_iso_code": { "type": "keyword" }, + "observer.hostname": { "type": "keyword" }, + "fail2ban.jail": { "type": "keyword" }, + "fail2ban.failures": { "type": "keyword" }, + "fail2ban.whois": { "type": "text" }, + "fail2ban.logs": { "type": "text" } + } + } + } +} +``` + +**2. Create an API key** + +In Kibana: Stack Management → API Keys → Create API key. The key needs write access to `fail2ban-events-*` indices. + +**3. Configure Fail2ban-UI** + +Enter the Elasticsearch URL, index name, and API key in the Alert Settings. Save and click **Test Connection** to verify. The test creates the first document. + +**4. Create a Kibana Data View** + +In Kibana: Stack Management → Data Views → Create data view. Use `fail2ban-events-*` as the name and index pattern. Select `@timestamp` as the time field. + +**5. Explore in Discover** + +Go to Kibana Discover, select the `fail2ban-events-*` data view, and you should see your indexed events. + +### Testing + +**Test Connection** indexes a test document (`"event.type": "test"`) with a dummy IP. A successful test confirms authentication, network connectivity, and index write permissions. + +### Technical details + +- Authentication: API key (sent as `Authorization: ApiKey `) or basic auth. +- Index naming: `-YYYY.MM.DD` using the UTC date of the event. +- Timeout: 15 seconds per request. +- Documents are sent via `POST //_doc`. +- TLS verification can be disabled for self-signed clusters. +- HTTP responses with status >= 400 are treated as errors and logged. + +## Alert dispatch flow + +When a ban or unban event arrives via the Fail2Ban callback (and the pharsing was valid): + +1. The event is stored in the database and broadcast via WebSocket (always, regardless of alerts). +2. The system checks whether alerts are enabled for the event type (ban/unban). +3. The IP is geolocated and checked against the configured alert countries. +4. If the country matches, the alert is dispatched to the configured provider. + +``` +Ban/Unban Event + → Store in DB + WebSocket broadcast + → Check alerts enabled? + → Check country filter? + → Dispatch to provider: + ├── email → sendBanAlert() → sendEmail() via SMTP + ├── webhook → sendWebhookAlert() → HTTP POST/PUT + └── elasticsearch → sendElasticsearchAlert() → POST //_doc +``` + +Switching providers does not affect event storage or WebSocket broadcasting. Only the notification delivery channel changes. diff --git a/docs/api.md b/docs/api.md index ac85b03..cf70c75 100644 --- a/docs/api.md +++ b/docs/api.md @@ -52,7 +52,11 @@ All endpoints that accept IP addresses validate them server-side using Go's `net ### Settings - `GET /api/settings` -> Get current application settings - `POST /api/settings` -> Update application settings -- `POST /api/settings/test-email` -> Send a test email +- `POST /api/settings/test-email` -> Send a test email (Email provider) +- `POST /api/settings/test-webhook` -> Send a test webhook payload (Webhook provider) +- `POST /api/settings/test-elasticsearch` -> Index a test document (Elasticsearch provider) + +The settings payload includes alert provider configuration (`alertProvider`, `webhook`, `elasticsearch` fields). See [`alert-providers.md`](https://github.com/swissmakers/fail2ban-ui/blob/main/docs/alert-providers.md) for the full provider documentation. ### Filter management - `GET /api/filters` -> List available filters @@ -88,6 +92,11 @@ Callbacks require: All IPs in callback payloads are validated before processing. +After validation, the callback triggers: +1. Event storage in the database +2. WebSocket broadcast to connected clients +3. Alert dispatch to the configured provider (Email, Webhook, or Elasticsearch) if alerts are enabled and the country filter matches + ### Authentication routes (OIDC) - `GET /auth/login` -> Initiate OIDC login flow - `GET /auth/callback` -> OIDC provider callback diff --git a/docs/architecture.md b/docs/architecture.md index 222c8ea..2d4acf6 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -4,7 +4,7 @@ Fail2Ban UI consists of : - a Go HTTP API (Gin) - a single-template web frontend (with static assets) - an embedded SQLite database for state and event history -- optional integrations (Email, GeoIP/Whois, firewalls) +- optional integrations (alert providers, GeoIP/Whois, firewall-integrations) ## Data flows @@ -26,12 +26,14 @@ Fail2Ban UI consists of : - REST API: server management, jail/filter config read/write, ban/unban actions, settings, data management (clear events/blocks) - WebSocket hub: streams real-time ban/unban events and (optional) debug console logs, protected by origin validation and session auth - Storage: server definitions, settings, ban history, permanent block records +- Alert providers: pluggable notification dispatch (Email/SMTP, Webhook, Elasticsearch) with country-based filtering and GeoIP enrichment - Integrations: MikroTik (SSH), pfSense (REST API), OPNsense (REST API) with input validation on all parameters - Ban Insights: country-level analytics with interactive 3D threat globe visualization Additional resources: -- Container deployment guide: `deployment/container/README.md` -- systemd setup guide: `deployment/systemd/README.md` +- Alert provider documentation: [`docs/alert-providers.md`](https://github.com/swissmakers/fail2ban-ui/blob/main/docs/alert-providers.md) +- Container deployment guide: [`deployment/container/README.md`](https://github.com/swissmakers/fail2ban-ui/blob/main/deployment/container/README.md) +- systemd setup guide: [`deployment/systemd/README.md`](https://github.com/swissmakers/fail2ban-ui/blob/main/deployment/systemd/README.md) ## More detailed diagrams @@ -109,15 +111,16 @@ Additional resources: │ │ SQLite Storage │ │ Whois / GeoIP │ │ │ │ • ban_events │ │ • IP → country/hostname │ │ │ │ • app_settings, servers │ │ MaxMind or ip-api.com │ │ -│ │ • permanent_blocks │ │ • Used in UI and emails │ │ +│ │ • permanent_blocks │ │ • Used in UI and alerts │ │ │ └────────────────────────────┘ └────────────────────────────┘ │ │ ┌────────────────────────────┐ ┌────────────────────────────┐ │ -│ │ Connector Manager │ │ Integrations + Email │ │ +│ │ Connector Manager │ │ Integrations + Alerts │ │ │ │ • Local (fail2ban.sock) │ │ • Mikrotik / pfSense / │ │ │ │ • SSH (exec on remote) │ │ OPNsense (block/unblock)│ │ │ │ • Agent (HTTP to agent) │ │ • Input validated (IP + │ │ │ │ • New server init: ensure │ │ identifiers sanitized) │ │ -│ │ │ │ • SMTP alert emails │ │ +│ │ │ │ • Alert dispatch (Email / │ │ +│ │ │ │ Webhook / Elasticsearch)│ │ │ │ │ └────────────────────────────┘ │ │ │ action.d (ui-custom- │ │ │ │ action.conf) │ │ @@ -148,7 +151,7 @@ Additional resources: │ │ 3. Whois/GeoIP enrichment │ │ │ │ 4. Store event in SQLite DB (ban_events) if nothing was invalid │ │ │ │ 5. Broadcast current event to WebSocket clients (ban_event / unban_event) │ │ -│ │ 6. Optional: send SMTP alert │ │ +│ │ 6. Optional: dispatch alert (Email / Webhook / Elasticsearch) │ │ │ │ 7 Run additional actions (e.g. block on pfSense for recurring offenders) │ │ │ │ 8. Respond status 200 OK - if all above was without an error │ │ │ └────────────────────────────────────────────────────────────────────────────┘ │ diff --git a/docs/configuration.md b/docs/configuration.md index 249db30..f5cbd27 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -60,7 +60,27 @@ Callbacks must include: ## Email template selection * `emailStyle=classic` - Switches back alert emails from the modern template to the classic template. + Switches back alert emails from the modern template to the classic template (only applies when the Email alert provider is selected). + +## Alert providers + +Alert settings are configured through the UI (Settings → Alert Settings). Three providers are available: + +| Provider | Description | +|---|---| +| Email (SMTP) | Default. Sends HTML-formatted alert emails via SMTP. | +| Webhook | Sends JSON payloads to any HTTP endpoint (ntfy, Matrix, Slack, Gotify, custom APIs). | +| Elasticsearch | Indexes events as ECS-compatible documents into Elasticsearch for Kibana analysis. | + +All providers share the same global settings: +- Enable/disable alerts for bans and unbans independently +- Country-based alert filtering (only alert on selected countries) +- GeoIP provider selection (built-in API or local MaxMind database) +- Maximum log lines included in alert payloads + +Provider-specific settings (SMTP credentials, webhook URL/headers, Elasticsearch URL/auth) are configured in the same UI section and stored in the database. + +For full provider documentation, setup hints, payload formats, and examples, see [`docs/alert-providers.md`](https://github.com/swissmakers/fail2ban-ui/blob/main/docs/alert-providers.md). ## OIDC authentication diff --git a/docs/security.md b/docs/security.md index edd0378..e6648af 100644 --- a/docs/security.md +++ b/docs/security.md @@ -72,6 +72,29 @@ If SELinux is enabled, use the policies provided in (according to your specific Do not disable SELinux as a shortcut. Fix always labeling and policy issues instead. -> Everytime you read "to disable SELinux" you can close that guide :) +## Alert provider security + +Fail2Ban UI supports three alert providers: Email (SMTP), Webhook, and Elasticsearch. Each has specific security considerations. + +### Email (SMTP) + +- Use TLS (`Use TLS` enabled) for all SMTP connections to maximize security here. +- Avoid disabling TLS verification (`Skip TLS Verification`) in production. If you must, ensure the network path to the SMTP server is trusted. +- Use application-specific passwords or OAuth tokens where supported (e.g. Gmail, Office365) instead of primary account passwords. + +### Webhook + +- Use HTTPS endpoints whenever possible. +- If the webhook endpoint requires authentication, use custom headers (e.g. `Authorization: Bearer `) rather than embedding credentials in the URL. +- Avoid disabling TLS verification for production endpoints. The `Skip TLS Verification` option exists for development/self-signed environments only. + +### Elasticsearch + +- Use API key authentication over basic auth when possible. API keys can be scoped to specific indices and rotated independently. +- Restrict the API key to write-only access on the `fail2ban-events-*` index pattern. Avoid cluster-wide or admin-level keys. +- Consider using Elasticsearch's built-in role-based access control to limit what the Fail2Ban UI service account can do. + + ## Audit and operational practices - Back up `/config` (DB + settings) regularly. diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index f1a5595..410ca57 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -249,11 +249,82 @@ Fail2Ban detects intrusion → Fail2ban-UI resolves server (by serverId) → Stores event in SQLite (ban_events table) → Broadcasts via WebSocket to all connected browsers - → Optional: sends email alert, evaluates advanced actions + → Optional: dispatches alert (Email / Webhook / Elasticsearch) + → Optional: evaluates advanced actions (recurring offenders) ``` If any step fails, the chain stops and the event will not appear in the UI. +## Alert provider issues + +### Alerts not being sent (any provider) + +1. Verify that alerts are enabled for the event type (ban and/or unban) in Settings → Alert Settings +2. Check which alert provider is selected - check all your settings for the active provider again. +3. Check country filtering: if specific countries are selected, only IPs geolocated to those countries trigger alerts. Set to `ALL` to alert on every event. +4. Use the Fail2ban-UI logs + (with enabled Debug-logs) to confirm the alert dispatch: + +```bash +podman logs -f fail2ban-ui + +# Successful email alert: +# 📧 sendEmail: Successfully sent email to ... +# Successful webhook: +# ✅ Webhook alert sent successfully +# Successful Elasticsearch: +# ✅ Elasticsearch alert indexed successfully +``` + +### Email alerts: test-email works but ban alerts don't arrive + +This typically happens because the ban-alert message looks differently (having IPs, special characters and maybe also payloads in it) vs. the test emails, that won't trigger a Spam mechanism. +Check: + +- The Fail2ban-UI logs show "Successfully sent email" for the ban event. -> then you know the problem is not on fail2ban-UI +- The email may be landing in spam. Check your spam/junk folder. +- Some SMTP servers (especially Office365) are strict about what is allowed and what not. Fail2ban-UI uses `\r\n` line endings and includes `Message-ID` and `Date` headers for compliance. It should work like this, but it can be always disliked by Microsoft and maybe needs a whitelist in the companies spam-policy. Ensure also you are running the latest version. + +### Webhook: HTTP 400 or connection errors + +Common issues: + +- **ntfy returns 400 "topic invalid"**: ntfy requires the topic in the URL path (e.g. `https://ntfy.sh/fail2ban-alerts`), not just the base URL. When sending JSON payloads, the topic must either be in the URL path or in the JSON body as a `topic` field. +- **Connection refused**: The webhook URL is unreachable from the Fail2ban-UI host. Test with curl from the same host/container. +- **401/403**: The endpoint requires authentication. Add the appropriate header (e.g. `Authorization: Bearer `) in the Custom Headers field. +- **TLS certificate errors**: For self-signed endpoints, enable `Skip TLS Verification` + +Test manually: +```bash +curl -v -X POST https://your-webhook-url \ + -H "Content-Type: application/json" \ + -d '{"event":"test","ip":"203.0.113.1","jail":"sshd","hostname":"testhost","country":"US","failures":"3","timestamp":"2026-02-23T00:00:00Z"}' +``` + +### Elasticsearch: connection or indexing failures + +Common issues: + +- **Connection refused / timeout**: Verify the Elasticsearch URL is reachable from the Fail2ban-UI host +- **401 Unauthorized**: API key or credentials are incorrect. Verify the API key in Kibana → Stack Management → API Keys +- **403 Forbidden**: The API key lacks write permissions on the target index. Create a key with `write` and `create_index` privileges for `fail2ban-events-*` +- **Index template missing**: Without an index template, Elasticsearch uses dynamic mapping which may produce suboptimal field types. Create the template as described in [`alert-providers.md`](https://github.com/swissmakers/fail2ban-ui/blob/main/docs/alert-providers.md) + +Test manually: +```bash +curl -v -X POST "https://your-es-url/fail2ban-events-test/_doc" \ + -H "Content-Type: application/json" \ + -H "Authorization: ApiKey YOUR_BASE64_KEY" \ + -d '{"@timestamp":"2026-02-23T00:00:00Z","event.kind":"alert","event.type":"test","source.ip":"203.0.113.1"}' +``` + +### Switching providers + +When switching alert providers (e.g. from Email to Webhook): + +1. The previous provider's settings are preserved in the database. Switching back restores them. +2. Make sure to save settings after changing the provider. +3. Always use the test button for the new provider before relying on it for real events. + ## Bans fail due to firewall backend (nftables / firewalld) Symptoms often mention `iptables (nf_tables)` or action startup errors.