IA612: Intrusion Detection and Prevention

St. Cloud State University

LAB-02: The Zeek (Bro) Network Security Monitor-Installation and Configuration

PART-01: How to Install Zeek (formerly Bro) NIDS on CentOS 8

A. Overview

This guide assumes you’ll be installing Zeek on CentOS 8, given how popular CentOS tends to be in the enterprise. However, the guide should work for any RHEL-based flavors of Linux. For Debian-based systems, there will be some modifications required, including using apt-get vs yum for installing Linux packages. Nothing that a search couldn’t help you figure out.

First, we’ll optimize CentOS to efficiently capture packets and then compile Zeek from source to start monitoring network traffic.

In this section, we’ll walkthrough following steps:

1. Enable the “network” service to apply network sniffing optimizations, including disabling NIC offloading functions to ensure Zeek sees full packet data and minimizes packet loss.

2. Setting interfaces to promiscuous mode to ensure all packets are captured and analyzed.

3. Install libmaxminddb to enable IP geolocation capability.

4. Build Zeek from source with optimizations.

5. Create a non-root Zeek user to minimize impact in the event that Zeek is compromised.

6. Deploy and run Zeek to start analyzing traffic.

7. Create a cron job to perform Zeek maintenance tasks.

B. Enable “network” service and disable NIC offloading functions

1. Install the network-scripts package.

sudo yum install network-scripts

2. Use ethtool to determine the maximum ring parameters for your sniffing interfaces. The example below assumes an interface named enp2s0.

sudo ethtool -g enp2s0 Ring parameters for enp2s0:

Pre-set maximums:

RX: 4096 RX Mini: 0

RX Jumbo: 0

TX: 4096

Current hardware settings:

RX: 256 RX Mini: 0

RX Jumbo: 0

TX: 256

3. As root/sudo, edit the /etc/sysconfig/network-scripts/ifcfg-<sniffinginterface> file for each sniffing network interface and change or add the following lines. Respectively, each line will disable control from NetworkManager, disable DHCP, and add appropriate ethtool options. Note that after “rx” you want to enter the maximum ring parameter as determined in the step above.

NM_CONTROLLED=no

BOOTPROTO=none ONBOOT=yes

IPV6INIT=no

ETHTOOL_OPTS="-G ${DEVICE} rx <max ring parameter determined from step 1 above>; -K ${DEVICE} rx off; -K ${DEVICE} tx off; -K ${DEVICE} sg off; -K ${DEVICE} tso off; -K ${DEVICE} ufo off; -K ${DEVICE} gso off; -K ${DEVICE} gro off; -K ${DEVICE} lro off"

4. Your file should now look something like this.

TYPE=Ethernet

BOOTPROTO=none

DEFROUTE=yes

IPV4_FAILURE_FATAL=no IPV6INIT=no

IPV6_AUTOCONF=yes

IPV6_DEFROUTE=yes IPV6_PEERDNS=yes

IPV6_PEERROUTES=yes

IPV6_FAILURE_FATAL=no

NAME=enp2s0

UUID=b22f5d92-3f1e-430b-b660-cb9376d8c0c0 DEVICE=enp2s0 ONBOOT=yes

PEERDNS=yes

PEERROUTES=yes USERS=root

NM_CONTROLLED=no

ETHTOOL_OPTS="-G ${DEVICE} rx 4096; -K ${DEVICE} rx off; -K ${DEVICE} tx off; -K ${DEVICE} sg off; -K ${DEVICE} tso off; -K ${DEVICE} ufo off; -K ${DEVICE} gso off; -K ${DEVICE} gro off; -K ${DEVICE} lro off"

5. Still as root/sudo, enable the “network” service.

sudo systemctl enable network

6. Finally, restart the “network” service.

sudo systemctl restart network

C. Set sniffing network interfaces to promiscuous mode

1. As root/sudo, create /etc/systemd/system/promisc.service in your favorite text editor.

2. Add the following lines, assuming enp2s0 is your sniffing interface.

[Unit]

Description=Makes an interface run in promiscuous mode at boot

After=network.target

[Service]

Type=oneshot

ExecStart=/usr/sbin/ip link set dev enp2s0 promisc on

TimeoutStartSec=0

RemainAfterExit=yes

[Install]

WantedBy=default.target

3. Save the file and run the following commands to make the changes permanent and start on boot.

sudo chmod u+x /etc/systemd/system/promisc.service sudo systemctl start promisc.service sudo systemctl enable promisc.service

Created symlink from

/etc/systemd/system/default.target.wants/promisc.service to /etc/systemd/system/promisc.service.

4. Reboot your system and verify all the changes made thus far have persisted. Verify that PROMISC is listed in the network interface status.

ip a show enp2s0 | grep -i promisc

3: enp2s0: < BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP > mtu 1500 qdisc pfifo_fast state UP group default qlen 1000

D. Install Zeek Dependencies

1. As root/sudo, edit /etc/yum.repos.d/CentOS-PowerTools.repo and set the “enabled” field to 1, to add the PowerTools repository. Your file should look something like this.

# CentOS-PowerTools.repo

#

# The mirror system uses the connecting IP address of the client and the

# update status of each mirror to pick mirrors that are updated to and

# geographically close to the client. You should use this for CentOS updates

# unless you are manually picking other mirrors.

#

# If the mirrorlist= does not work for you, as a fall back you can try the # remarked out baseurl= line instead.

[PowerTools]

name=CentOS-$releasever – PowerTools

mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&r epo=PowerTools&infra=$infra

#baseurl=http://mirror.centos.org/$contentdir/$releasever/PowerTools/$basearc h/os/ gpgcheck=1 enabled=1

gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial

2. Add the EPEL repo.

sudo yum –enablerepo=extras install epel-release

3. Run the following yum command to download the required dependencies.

sudo yum install cmake make gcc gcc-c++ flex bison jemalloc-devel libpcap-devel openssl-devel platform-python-devel swig zlib-devel

4. Ensure all your packages are up to date and reboot your system.

sudo yum update sudo reboot

E. Configure GeoIP Support 1. Install the libmaxminddb development library.

sudo yum install libmaxminddb-devel

2. Sign up for a This is required as of December 2019.

3. and untar the GeoLite2 database.

tar xzvf GeoLite2-City.tar.gz

4. Move the GeoLite2-City.mmdb file in the extracted GeoLite2-City_YYYYMMDD directory to /usr/share/GeoIP.

sudo mv GeoLite2-City_YYYYMMDD/GeoLite2-City.mmdb

/usr/share/GeoIP/GeoLite2-City.mmdb

F. Create the zeek user and directory to install and run Zeek

1. Create the zeek user and add it to the zeek group.

sudo groupadd zeek sudo useradd zeek -g zeek

2. As root/sudo, set a password for the zeek user.

sudo passwd zeek

3. As root/sudo, create the /opt/zeek directory and set ownership to the zeek user.

sudo mkdir /opt/zeek sudo chown -R zeek:zeek /opt/zeek sudo chmod 750 /opt/zeek

G. Download, Compile, and Install Zeek

1. Switch to the zeek user.

su zeek

2. We will download zeek to the /home/zeek directory. Then we will configure Zeek to install in the /opt/zeek directory and enable jemalloc to improve memory and CPU usage. As of this writing, the latest release is version 3.2.1. If the download URL referenced in the wget command below no longer works, you can download the latest stable release directly from the Get Zeek download page.

cd

wget https://download.zeek.org/zeek-3.2.1.tar.gz tar -xzvf zeek-3.2.1.tar.gz cd zeek-3.2.1

./configure –prefix=/opt/zeek –enable-jemalloc make make install

Note: This will take *a while* to compile.

3. Switch back to your normal user by closing the zeek session.

Exit

4. Since the zeek user is not root, give the Zeek binaries permissions to capture packets.

sudo setcap cap_net_raw=eip /opt/zeek/bin/zeek sudo setcap cap_net_raw=eip /opt/zeek/bin/capstats

H. Add Zeek to PATH

1. As root/sudo, create /etc/profile.d/zeek.sh and add the following.

pathmunge /opt/zeek/bin

2. Log out and log back in as the zeek user to update the PATH.

I. Configure Zeek

1. Edit /opt/zeek/etc/node.cfg to configure the number of nodes. It is recommended to use a maximum of one or two less workers than the total number of CPU cores available on your sensor. In the example configuration below we are configuring a total of two workers, analyzing one sniffing interface.

# Below is an example clustered configuration on a single host.

host=localhost

host=localhost

[proxy-1] type=proxy host=localhost

[worker-1] type=worker host=localhost interface=enp2s0

[worker-2] type=worker host=localhost interface=enp2s0

In the event you have two or more sniffing interfaces (e.g. enp2s0 and enp3s0), see the example configuration below which assigns each interface its own worker.

# Below is an example clustered configuration on a single host.

host=localhost

host=localhost

[proxy-1] type=proxy host=localhost

[worker-1] type=worker host=localhost interface=enp2s0

[worker-2] type=worker host=localhost interface=enp3s0

2. Edit /opt/zeek/share/zeek/site/local.zeek to enable or disable scripts as needed.

J. Start Zeek

1. As the zeek user, run zeekctl deploy to apply configurations and run Zeek.

zeekctl deploy checking configurations … installing … removing old policies in /opt/zeek/spool/installed-scripts-do-not-touch/site

…

removing old policies in /opt/zeek/spool/installed-scripts-do-not-touch/auto

… creating policy directories … installing site policies … generating cluster-layout.zeek … generating local-networks.zeek … generating zeekctl-config.zeek … generating zeekctl-config.sh …

stopping …

stopping workers … stopping proxy … stopping manager … stopping logger …

starting … starting logger … starting manager … starting proxy … starting workers …

2. If your output looks similar to what’s shown above, you should be good to go. To verify Zeek is running successfully, you can run zeekctl status. zeekctl status

worker-2-2 worker localhost running 1954 10 Oct 23:15:35

If you see the following errors:

zeekctl deploy

Error: worker-1-1 terminated immediately after starting; check output with "diag"

Error: worker-1-2 terminated immediately after starting; check output with "diag"

Error: worker-2-1 terminated immediately after starting; check output with "diag"

Error: worker-2-2 terminated immediately after starting; check output with "diag" Then try re-running the sudo setcap commands from earlier.

sudo setcap cap_net_raw=eip /opt/zeek/bin/zeek sudo setcap cap_net_raw=eip /opt/zeek/bin/capstats

3. You should now see logs being generated in /opt/zeek/logs/current.

ls -l total 2276

-rw-rw-r–. 1 zeek zeek 1573 Oct 10 23:15 broker.log

-rw-rw-r–. 1 zeek zeek 593 Oct 10 23:45 capture_loss.log

-rw-rw-r–. 1 zeek zeek 1970 Oct 10 23:15 cluster.log

-rw-rw-r–. 1 zeek zeek 673435 Oct 10 23:52 conn.log

-rw-rw-r–. 1 zeek zeek 580865 Oct 10 23:52 dns.log

-rw-rw-r–. 1 zeek zeek 3830 Oct 10 23:49 dpd.log

-rw-rw-r–. 1 zeek zeek 1406 Oct 10 23:47 files.log

-rw-rw-r–. 1 zeek zeek 26108 Oct 10 23:48 http.log

-rw-rw-r–. 1 zeek zeek 24646 Oct 10 23:15 loaded_scripts.log

-rw-rw-r–. 1 zeek zeek 753 Oct 10 23:18 notice.log

-rw-rw-r–. 1 zeek zeek 187 Oct 10 23:15 packet_filter.log

-rw-rw-r–. 1 zeek zeek 743 Oct 10 23:46 software.log

-rw-rw-r–. 1 zeek zeek 86512 Oct 10 23:51 ssl.log

-rw-rw-r–. 1 zeek zeek 5446 Oct 10 23:50 stats.log

-rw-rw-r–. 1 zeek zeek 0 Oct 10 23:15 stderr.log

-rw-rw-r–. 1 zeek zeek 188 Oct 10 23:15 stdout.log

-rw-rw-r–. 1 zeek zeek 240866 Oct 10 23:51 weird.log

4. If you’re running into issues, zeekctl diag can provide more detailed output for troubleshooting purposes.

zeekctl diag

K. ZeekControl Cron

ZeekControl features a cron command to check for and restart crashed nodes and to perform other maintenance tasks. To take advantage of this, let’s set up a cron job.

1. Edit the crontab of the non-root zeek user.

crontab -e

2. Add the following to set up a cron job that runs every five minutes. You can adjust the frequency to your liking.

*/5 * * * * /opt/zeek/bin/zeekctl cron

PART-02: Zeek Package Manager

A. Overview

Now we’ll introduce the to extend Zeek’s functionality with packages contributed by the Zeek community. A full list of available packages can be viewed on the We will focus on configuring Zeek to use AF_PACKET to further optimize packet capture and analysis. We’ll also install additional useful packages.

In this section, we’ll walkthrough following steps:

1. Install Zeek Package Manager to extend Zeek’s functionality.

2. Use Zeek Package Manager to install the AF_PACKET package.

3. Configure Zeek to use the AF_PACKET package to optimize Zeek’s packet capture performance.

4. [Optional] Install additional useful packages including ja3 and HASSH.

5. Update Zeek packages.

B. Install Zeek Package Manager

1. As root/sudo install git to download Zeek packages. sudo yum install git

2. As root/sudo install PIP to download Python packages. sudo yum install python3-pip

3. Switch to the zeek user. su zeek

4. Change directories to the zeek user’s home directory and confirm the change. Doing this ensures that we run zkg from a directory that the zeek user has access to, otherwise zkg will fail.

cd pwd /home/zeek

5. Install the Zeek Package Manager (zkg) as the zeek user. pip3 install zkg –user

6. As the zeek user, configure Zeek Package Manager (zkg).

zkg autoconfig

This will create a configuration file in /home/zeek/.zkg/config. Upon completion it should look something like the following. zeek = https://github.com/zeek/packages

[paths]

state_dir = /home/zeek/.zkg script_dir = /opt/zeek/share/zeek/site plugin_dir = /opt/zeek/lib/zeek/plugins zeek_dist = /home/zeek/zeek-3.1.5

C. Install the AF_PACKET package

1. As the zeek user, stop Zeek if it is currently running.

zeekctl stop

2. Use zkg to install the AF_PACKET package.

zkg install zeek/j-gras/zeek-af_packet-plugin

The following packages will be INSTALLED: zeek/j-gras/zeek-af_packet-plugin (2.0.0)

Proceed? [Y/n] y

Running unit tests for "zeek/j-gras/zeek-af_packet-plugin"

Installing "zeek/j-gras/zeek-af_packet-plugin"…….

Installed "zeek/j-gras/bro-af_packet-plugin" (2.0.0)

D. Configure Zeek to use AF_PACKET

1. Edit /opt/zeek/etc/node.cfg to configure Zeek to use AF_PACKET. In the example configuration below we are configuring one worker, load balanced across two cores, analyzing one sniffing interface.

# Below is an example clustered configuration on a single host.

host=localhost

host=localhost

[proxy-1] type=proxy host=localhost

[worker-1] type=worker host=localhost interface=af_packet::enp2s0 lb_method=custom lb_procs=2 pin_cpus=0,1

In the event you have two or more sniffing interfaces (e.g. enp2s0 and enp3s0), see the example configuration below which assigns each interface its own worker, load balanced across two cores, again using AF_PACKET. Note the addition of unique af_packet_fanout_id values for each of the sniffing interfaces.

# Below is an example clustered configuration on a single host.

host=localhost

host=localhost

[proxy-1] type=proxy host=localhost

[worker-1] type=worker host=localhost interface=af_packet::enp2s0 lb_method=custom lb_procs=2 pin_cpus=0,1 af_packet_fanout_id=2

[worker-2] type=worker host=localhost interface=af_packet::enp3s0 lb_method=custom lb_procs=2 pin_cpus=2,3 af_packet_fanout_id=3

2. As root/sudo, give the Zeek binaries permissions to capture packets. This was previously done in PART-01, however, installing AF_PACKET requires doing this again.

sudo setcap cap_net_raw=eip /opt/zeek/bin/zeek sudo setcap cap_net_raw=eip /opt/zeek/bin/capstats

3. As the zeek user, run zeekctl deploy to apply configurations and run Zeek.

zeekctl deploy

If you see the following errors, try re-running the sudo setcap commands from the previous step.

zeekctl deploy

Error: worker-1-1 terminated immediately after starting; check output with "diag"

Error: worker-1-2 terminated immediately after starting; check output with "diag"

Error: worker-2-1 terminated immediately after starting; check output with "diag"

Error: worker-2-2 terminated immediately after starting; check output with "diag"

E. [Optional] Install Additional Useful Packages (e.g. add-interfaces, ja3, and HASSH) We’ll install additional Zeek packages: add-interfaces, ja3, and HASSH. The install process outlined below should work for installing other packages you may be interested in.

1. As the zeek user, stop Zeek if it is currently running.

zeekctl stop

2. Use zkg to install the add-interfaces package. In situations where you are monitoring multiple network interfaces on one sensor, this adds an “_interface” field to every log file which labels the particular network interface that traffic is coming from. zkg install zeek/j-gras/add-interfaces The following packages will be INSTALLED: zeek/j-gras/add-interfaces (master)

Proceed? [Y/n] y

Installed "zeek/j-gras/add-interfaces" (master) Loaded "zeek/j-gras/add-interfaces"

3. Edit /opt/zeek/share/zeek/site/add-interfaces/add-interfaces.bro and modify the const enable_all_logs and const include_logs: set[Log::ID] fields as shown below. Save the file when you’re finished. export {

## Enables interfaces for all active streams const enable_all_logs = T &redef; ## Streams not to add interfaces for const exclude_logs: set[Log::ID] = { } &redef;

## Streams to add interfaces for

const include_logs: set[Log::ID] = { } &redef; }

4. Use zkg to install the ja3 package. This is used for profiling SSL/TLS clients.

zkg install zeek/salesforce/ja3

The following packages will be INSTALLED: zeek/salesforce/ja3 (master)

Proceed? [Y/n] y

Installing "zeek/salesforce/ja3"

Installed "zeek/salesforce/ja3" (master) Loaded "zeek/salesforce/ja3"

5. Use zkg to install the HASSH package. This is used for profiling SSH clients and servers.

zkg install zeek/salesforce/hassh The following packages will be INSTALLED: zeek/salesforce/hassh (master)

Proceed? [Y/n] y

Installing "zeek/salesforce/hassh"

Installed "zeek/salesforce/hassh" (master)

Loaded "zeek/salesforce/hassh"

6. Edit /opt/zeek/share/zeek/site/local.zeek and add the following lines to the bottom. This will load all packages you’ve installed.

# Load Zeek Packages @load packages

7. As the zeek user, run zeekctl deploy to apply configurations and run Zeek.

zeekctl deploy

F. Update Installed Zeek Packages

1. As the zeek user, stop Zeek if it is currently running. zeekctl stop

2. Use zkg to check for updated packages. zkg refresh

Refresh package source: zeek No changes Refresh installed packages New outdated packages:

zeek/salesforce/hassh (master)

This indicates that zeek/salesforce/hassh needs to be updated.

3. Use zkg to check for updated packages. zkg upgrade

The following packages will be UPGRADED: zeek/salesforce/hassh (master)

Proceed? [Y/n] y

Upgraded "zeek/salesforce/hassh" (master)

4. As the zeek user, run zeekctl deploy to apply configurations and run Zeek. zeekctl deploy

6. Deliverables

1. Read the lab instructions above and finish all the tasks. Take screenshots where necessary (For all important steps) to add your lab report. At each step write what you did in brief in your own words.

2. Configure “Elasticsearch, Logstash, and Kibana (ELK)” data visualization tools to visualize the sample log files such as syslog.log, dns.log, ssl.log, x509.log, and conn.log generated by the “ZEEK” in this lab, whne you surfe web while ZEEK is monitoring your system. After that add the clear and well organized steps into the lab report that how you configure the ELK to view “ZEEK” log files.

What is the ELK Stack?

The ELK Stack is a collection of three open-source products — Elasticsearch,Logstash, and Kibana — all developed, managed and maintained by Elastic. Elasticsearch is a NoSQL database that is based on the Lucene search engine. Logstash is a log pipeline tool that accepts inputs from various sources, executes different transformations, and exports the data to various targets. Kibana is a visualization layer that works on top of Elasticsearch. The stack also includes a family of log shippers called Beats, which led Elastic to rename ELK as the Elastic Stack.

Together, these different open source products are most commonly used for centralized logging in IT environments (though there are many more use cases for the ELK Stack including business intelligence, security and compliance, and web analytics). Logstash collects and parses logs, and then Elasticsearch indexes and stores the information. Kibana then presents the data in visualizations that provide actionable insights into one’s environment.

7. Submission Instruction:

1. Complete all the tasks assigned in the deliverable section.

2. Take all necessary and appropriate screenshots.

3. Past those new screenshots to a MS-Word file with the step number.

4. Write a brief description bellow or above each screenshot for each step to make a good report related to this lab.

5. Submit your report to the D2L “LAB-02” drop-box by only one member of each group within one week from the assigned date.

NOTE-1: Please add all group member’s names (LAST_NAME, FIRST_NAME), the group number and page numbers of the report.

NOTE-2: Please add all group member’s contributions to complete and submit this lab as a percentage as shown below at the end of the report. (Before submitting to D2L all of the group members must know or aware their reported contribution as a percentage in the lab report)

Example:

Member-01: 100% Member-02: 75% Member-03: 100%

References

[1]. [2].

[3].

[5].

1

1

1