Real-Time Zeek Analytics

Rule Function

Zeek is an open source network security monitoring tool, consisting of a suite of monitoring apps divided into modules. This rule accepts syslog messages in JSON format comprised of Zeek log messages originating from these various modules. The rule then sets a set of basic user tags (indicated below) and composes a LogZilla event message consisting of certain of the incoming JSON data fields (Zeek data).

The Zeek modules (and whether they are processed by this rule) are:

Vendor Information

• Zeek Script Reference Log Files
• Zeek JSON Format Logs
• Zeek Logs

User Tags

There are many potential user tags available based on the Zeek data fields. The following table indicates both used and potential user tags:

HC Tags

• SrcIP
• DstIP
• Domain
• Status Message

Log Examples

conn entry corresponding to a basic UDP packet communication

Wrap: OnCopyFullscreen
{
  "ts": 1591367999.305988,
  "uid": "CMdzit1AMNsmfAIiQc",
  "id.orig_h": "192.168.4.76",
  "id.orig_p": 36844,
  "id.resp_h": "192.168.4.1",
  "id.resp_p": 53,
  "proto": "udp",
  "service": "dns",
  "duration": 0.06685185432434082,
  "orig_bytes": 62,
  "resp_bytes": 141,
  "conn_state": "SF",
  "missed_bytes": 0,
  "history": "Dd",
  "orig_pkts": 2,
  "orig_ip_bytes": 118,
  "resp_pkts": 2,
  "resp_ip_bytes": 197
}

ssl entry corresponding to a client initiating an SSL connection

Wrap: OnCopyFullscreen
{
  "ts": 1598377391.921726,
  "uid": "CsukF91Bx9mrqdEaH9",
  "id.orig_h": "192.168.4.49",
  "id.orig_p": 56718,
  "id.resp_h": "13.32.202.10",
  "id.resp_p": 443,
  "version": "TLSv12",
  "cipher": "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
  "curve": "secp256r1",
  "server_name": "www.taosecurity.com",
  "resumed": false,
  "next_protocol": "h2",
  "established": true,
  "cert_chain_fuids": [
    "F2XEvj1CahhdhtfvT4",
    "FZ7ygD3ERPfEVVohG9",
    "F7vklpOKI4yX9wmvh",
    "FAnbnR32nIIr2j9XV"
  ],
  "client_cert_chain_fuids": [],
  "subject": "CN=www.taosecurity.com",
  "issuer": "CN=Amazon,OU=Server CA 1B,O=Amazon,C=US"
}

ssh entry corresponding to an inbound SSH connection

Wrap: OnCopyFullscreen
{
  "ts": "2020-09-16T13:29:23.245216Z",
  "uid": "CzEmsljW9ooL0WnBd",
  "id.orig_h": "35.196.195.158",
  "id.orig_p": 53160,
  "id.resp_h": "192.168.4.37",
  "id.resp_p": 22,
  "version": 2,
  "auth_success": true,
  "auth_attempts": 1,
  "direction": "INBOUND",
  "client": "SSH-2.0-OpenSSH_7.9p1 Debian-10+deb10u2",
  "server": "SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.3",
  "cipher_alg": "[email protected]",
  "mac_alg": "[email protected]",
  "compression_alg": "none",
  "kex_alg": "curve25519-sha256",
  "host_key_alg": "ecdsa-sha2-nistp256",
  "host_key": "a3:41:03:32:1f:8c:8e:82:92:9f:62:8c:38:82:d3:74",
  "hasshVersion": "1.0",
  "hassh": "ec7378c1a92f5a8dde7e8b7a1ddf33d1",
  "hasshServer": "b12d2871a1189eff20364cf5333619ee",
  "cshka": "[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa",
  "hasshAlgorithms": "curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c;[email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected];[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1;none,[email protected],zlib",
  "sshka": "ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519",
  "hasshServerAlgorithms": "curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1;[email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected];[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1;none,[email protected]"
}

Zeek Server-side configuration

Zeek configuration

The syslog-ng config in the section below expects the Zeek logs to be sent in JSON format. On the Zeek side, you may need to enable this log type.

• edit /opt/zeek/share/zeek/site/local.zeek and set:

Wrap: OnCopyFullscreen
# Output in JSON format
@load policy/tuning/json-logs.zeek

• Run zeekctl deploy to deploy the change

• check /opt/zeek/logs/current/conn.log to make sure it's in JSON format, for example:

Wrap: OnCopyFullscreen
# tail -1 /opt/zeek/logs/current/conn.log
{"ts":1641949013.6772,"uid":"CX1l7X34hCbGkWGlB6","id.orig_h":"192.168.10.107","id.orig_p":36278,"id.resp_h":"192.168.10.255","id.resp_p":32412,"proto":"udp","conn_state":"OTH","local_orig":true,"local_resp":true,"missed_bytes":0,"history":"CC","orig_pkts":0,"orig_ip_bytes":0,"resp_pkts":0,"resp_ip_bytes":0}

Syslog-ng Configuration on Zeek

For use with Zeek log files this LogZilla rule requires that syslog be configured to read those Zeek log files and forward the log messages to LogZilla.

WARNING: You may also need to enable the syslog-ng-mod-extra package which provides the syslog-ng type output driver used in the zeek2logzilla.conf below. However, newer versions of syslog-ng have the module in the base package.

For example, if installing on a debian based system:

apt install syslog-ng-mod-extra

The syslog-ng configuration for the machine hosting the Zeek log files should be as follows:

Wrap: OnCopyFullscreen
# This is for your *zeek* server (not the LogZilla server)
# filename: /etc/syslog-ng/conf.d/zeek2logzilla.conf
# Zeek log format should look like:
# {"ts":1641946189.335886,"uid":"Ce6ul9J1tSYJNyRga","id.orig_h":"192.168.10.98","id.orig_p":755,"id.resp_h":"192.168.10.99","id.resp_p":2049,"proto":"tcp","conn_state":"OTH","local_orig":true,"local_resp":true,"missed_bytes":0,"history":"CC","orig_pkts":0,"orig_ip_bytes":0,"resp_pkts":0,"resp_ip_bytes":0}
#
# Global Options
options {
  flush_lines(100);
  threaded(yes);
  use_dns(yes);
  use_fqdn (no);
  keep_hostname (yes);
  dns-cache-size(2000);
  dns-cache-expire(87600);
};

# Define log sources
# WARNING: DO NOT USE the zeek symlinked directory
# (/opt/zeek/logs/current by default)
# If you do, then when zeek is restarted
# syslog-ng will try to follow/watch the old files
# and not the new ones
source s_zeek_logs {
    wildcard-file(
        base-dir("/opt/zeek/spool/zeek")
        filename-pattern("*.log")
        flags(no-parse)
    );
};

# Set destination (logzilla)
# REPLACE the host "logzilla" below
# with the actual hostname or IP of your LZ server
# test and make sure you can ping/reach the host
destination d_logzilla {
  syslog-ng(server("logzilla") port(514));
};

log {
  source(s_zeek_logs);
  parser { json-parser (prefix(".JSON.")); };
  rewrite { set("zeek" value(".JSON._source_type")); };
  rewrite { set("$(basename ${FILE_NAME})" value(".JSON._source")); };
  destination(d_logzilla);
  flags(flow-control);
};