Sam Stelfox

Thoughts from a software engineer, systems hacker and Linux gubernāre.

Hardening

PLEASE NOTE: This guide was developed for Red Had Based architectures, specifically CentOS 5, and Fedora 16+. A lot of the information here is generally solid principles but you may need to adapt it to your distribution.

Hardening of all my servers is the largest piece IMHO to the defence in depth. It brings to them each a strength to stand on their own. I want to be able to plug each and everyone on a raw internet connection and feel safe knowing that they won't be compromised.

In the event that any section of my defence fails, I'll know that the rest of my systems are strong enough to hopefully be protected from whatever the weak link was.

This section of my documentation provides a quick overview of procedures to harden a system to make sure I never miss a step.

This hardening guide started out as a summary and a few updates to the NSA RedHat 5 Operating System Security Guidelings. Since then it has grown to include a lot of services that were not mentioned anywhere in there, and a few best practices that I've developed myself over the years of managing Linux boxes.

I've started including information from National Vulnerability Database (NVD) National Checklist Program Repository hosted by NIST. They aggregate some useful information from a few sites including the Center for Internet Security (CIS) Security Benchmarks. One more site I've got some information from is the SANS Information Security Reading Room.

These have all been listed here to give them some credit towards the following security guidelines. If you doubt anything I've mentioned here I'd recommend you go look through these sites.

General Principles

Review of the Security Notes

This section is mostly a TODO for myself. To keep this as up to date and relevant as possible I need to annually review the following information:

Hardware

Hardware and BIOS's change from machine to machine but there are some quick best practices that should be followed. Follow the BIOS notes and ensure the hardware is physically secure.

Installation Notes

Firewall

A strong firewall does not just protect against incoming attacks but also prevents a machine from connecting to things it shouldn't. In the event of a partially compromised system, an egress ruleset can help prevent an attacker from using the machine as a springboard to launch further attacks on the internal networks.

IPv4 and IPv6 traffic have separate firewalls. Strong examples for configuring IPTables and IP6Tables can be found on their respective configuration pages.

Networking

Follow the Network guide to configure a static address and gateway.

Update /etc/resolv.conf to include the proper domain and nameserver. It will look something like this:

search internal.example.org
nameserver 4.2.2.2
nameserver 8.8.8.8

Verify hostname / domain in /etc/sysconfig/network. An example is given below:

NETWORKING=yes
HOSTNAME=testing.internal.example.org

Edit /etc/hosts. Remove all the IPv6 loopback entries with the exception of localhost6 and all of the IPv4 addresses with the exception of localhost. 127.0.0.1 should also have entries for the FQDN of this server and just the hostname. Other extraneous entries should be removed unless absolutely necessary.

127.0.0.1       testing.internal.example.org testing localhost localhost4
::1             testing.internal.example.org testing localhost localhost6

Logging

You'll want to make sure you configure RSyslog, Logrotate, and Logwatch according to their appropriate guides. Logwatch is not installed in the minimum install and should be installed.

Configure Root's Mail Recipient

Add the line to following line to /etc/aliases, replacing [email protected] with an administrator's email address. This is where logs will be emailed.

root:     [email protected]

As root run the newaliases command which will update the binary database of mail recipients.

Services

Review the list of services running on the machine with the following command:

systemctl --type=service

Pay special attention to any that is listed as "running", these are the always on services that start up on boot. If it says "exited" it has started up on boot finished whatever part in the boot process it took care of and then properly exited. The latter need to be audited as well, if they're not needed they should not be executing on boot.

Whenever you find a service that shouldn't be started on boot run the following command replacing "rpcbind.service" with the service you would like to disable.

systemctl disable rpcbind.service
systemctl stop rpcbind.service

If it's particularily pesky about not going away you may need to mask it. This will essentially create a symlink of the service to /dev/null, allowing it to attempt to execute without actually accomplishing anything.

systemctl mask rpcbind.service

To re-enable a service you'll want to execute the following commands:

system enable rpcbind.service
system start rpcbind.service

If you masked the service you'll need to unmask it before re-enabling it.

systemctl unmask rpcbind.service

You need to be aware that the mask / unmask command is a stronger version of "disable", disable just prevents systemd from starting the service on boot. "mask" links the service file to /dev/null preventing manual activation as well as other services attempting to start it as requirements.

If you mask a critical service that the machine need to boot it will fail and you'll need to fix this by hand in the debug shell.

All Server Services

On all of my servers I configure:

Updates

Updates regardless of whether KSplice is being used should be done as soon as possible. For the most part there shouldn't be any reason not to update automatically unless you are using untrusted repositories.

There is a daemon available yum-updatesd in the Fedora repositories which can be configured to notify administrators of new packages and optionally install them automatically.

Updating Manually

Updating the system manually can be performed using the following command:

yum update

Init Process

Ensure that root logins are only permitted on appropriate terminals in /etc/securetty. An example used by me is the following:

tty1
tty2
tty3

If I'm going to access the system of the serial console I'd want to add "console" to that as well.

The init process by default starts with that pretty plymouth graphical boot loader, to change it back to the default linux boot sequence that is actually verbose and useful you need to make a few changes to the /etc/sysconfig/init file. I use the following:

BOOTUP=verbose
AUTOSWAP=no
ACTIVE_CONSOLES=/dev/tty[1-3]
SINGLE=/sbin/sulogin

The changes actually made reduces the number of active TTYs to 3 from 6, requires a root login to get into single user mode, and disables the pretty color status display when in text mode during boot up.

The other change that needs to be made is in the default grub options in /etc/default/grub. Remove rhgb and quiet from the GRUB_CMDLINE_LINUX variable. Strictly speaking you don't need to remove quiet, however, it allows the kernel to print out it's boot messages which can help debug some lower level issues and I generally find it useful. After making the change you'll need to run grub2-mkconfig -o /boot/grub2/grub.cfg as root.

One thing that I have noticed is that as of grub2 if a boot password is defined you need to add --unrestricted to the menu entry if you want to boot the menu entry without a super user password. I've done this in /etc/grub.d/10_linux by adding --unrestricted to the CLASS environment variable around line 29 and rebuilding the grub config.

Sudo

Sudo is one of those things that I need to go through and properly restrict. Since I'm usually the lone administrator or am very close with the few others on servers and I trust the people given sudo permissions with global root like permissions.

This definitely has larger security implications but here is how to configure that. This grants unlimited sudo privileges to any user in the sudoers group. They still need to authenticate however.

Edit /etc/sudoers adding the line:

%sudoers                ALL=(ALL)        ALL

Users

Password Requirements

Update the minimum user password to 10 characters in /etc/login.defs

Creation

If local authentication is used, create any users that will need access to the system using the following command:

useradd <username>
passwd <username>

If the user isn't available locally to enter their own password, a hash from another system can be extracted and dropped into place or alternatively a very strong temporary password can be set and provided to the user through secure means. You can then run the following command to force the user to change their password the next time they login:

chage -d 0 <username>

Securing System Users

Review /etc/passwd to ensure all non-user accounts have their account shells set to /sbin/nologin with the exception of halt, shutdown, sync which should be /sbin/halt, /sbin/shutdown, and /bin/sync respectively.

Mysql creates it's user with /bin/bash as the shell which shouldn't be (this is just an example).

System Wide Settings

Edit /etc/profile, find HISTSIZE replace that line with these couple of lines.

readonly HISTSIZE=50
export readonly TMOUT=900
export readonly HISTFILE="$HOME/.bash_history"

Groups

Creation

Create the sudoers and sshers group

groupadd sudoers -g 400
groupadd sshers -g 401

Add Users

useradd <username>
passwd <username>

Any users that are going to be SSH'ing into the server should be added to the sshers group. This includes root if necessary (which by default is only available to pubkey based authentication if you follow this guide).

usermod -a -G sshers <username>

Any users that NEED sudo permission should be added to the sudoers group like so:

usermod -a -G sudoers <username>

You can do both at the same time like so:

usermod -a -G sshers,sudoers <username>

System Hardening

You'll want to verify that that there are not more SUID or GUID flags set than absolutely necessary. You can find all of the files with these flags set with the following command:

[[email protected] ~]# find / -type f \( -perm -04000 -o -perm -02000 \)

The same goes for unowned files as there should never be any of these:

[[email protected] ~]# find / \( -nouser -o -nogroup \) -print

Something that should be considered is security limits per user. This can not only be used to prevent users from abusing the system but to prevent a user accidentally fork bombing a system or causing other kinds of havoc. These settings can be found in /etc/security/limits.conf

Checklist

Updating Software

Permission Verification

Restrict Dangerous Execution Patterns

Restrict Password Based Login

Account Security

SELinux

Network/Firewall Configuration

Logging and Auditing

Misc

Services