Enterprise Linux System Hardening & Performance Tuning Guide

Enterprise Linux System Hardening & Performance Tuning

Securing and optimizing Linux production servers requires deliberate kernel parameters, memory management, and process limits.

NOTE
Tested on Ubuntu 22.04/24.04 LTS and AlmaLinux / Rocky Linux 9. Always verify parameters in a staging environment before applying to core database or high-traffic web nodes.

1. Network & Kernel Security (/etc/sysctl.d/99-sysadmin-hardening.conf)

Apply these optimized TCP stack parameters to resist SYN floods, IP spoofing, and maximize throughput:

ini
# Disable IP Packet Forwarding (unless node is a router/VPN gateway)
net.ipv4.ip_forward = 0

# Ignore ICMP Echo Broadcasts (Smurf attack protection)
net.ipv4.icmp_echo_ignore_broadcasts = 1

# Enable TCP SYN Cookies against SYN flood attacks
net.ipv4.tcp_syncookies = 1

# Disable ICMP Redirect Acceptance (prevent MITM route injection)
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.all.secure_redirects = 0

# Log Spoofed Packets, Source Routed Packets, Redirect Packets
net.ipv4.conf.all.log_martians = 1

# TCP High-Performance Window Scaling & Fast Open
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_fastopen = 3
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 8192

# Virtual Memory Swappiness (keep cached pages in RAM)
vm.swappiness = 10
vm.dirty_ratio = 15
vm.dirty_background_ratio = 5

Apply immediately:

bash
sudo sysctl --system


2. File Descriptor & Process Limits (/etc/security/limits.d/99-nofile.conf)

High-concurrency servers (Nginx, Postfix, Redis) can exhaust standard file descriptors:

ini
*          soft    nofile     65535
*          hard    nofile     65535
root       soft    nofile     65535
root       hard    nofile     65535

3. SSH Daemon Hardening (/etc/ssh/sshd_config.d/hardening.conf)

CAUTION
Ensure your public SSH key is installed in ~/.ssh/authorized_keys before restarting sshd!
ini
Port 22
PermitRootLogin prohibit-password
PasswordAuthentication no
ChallengeResponseAuthentication no
MaxAuthTries 3
ClientAliveInterval 300
ClientAliveCountMax 2
X11Forwarding no

Validate and reload:

bash
sudo sshd -t && sudo systemctl reload ssh


Architecture Flow Diagram

graph TD
    A[Client Request] -->|TCP SYN| B[Firewall UFW / nftables]
    B -->|SYN Cookies Verified| C[Nginx Reverse Proxy]
    C -->|Unix Socket / FastCGI| D[PHP 8.3-FPM Backend]
    D -->|Persistent WAL Read/Write| E[(SQLite 3 Embedded DB)]
Tags: #linux #sysctl #security #kernel #performance