Back to tutorials
Tutorial

Linux VPS Network Security Configuration Tutorial: Complete UFW, iptables, and Port Scanning Protection Setup for 2026

Complete Linux VPS network security configuration tutorial. Set up UFW, iptables, port scanning protection, and advanced firewall rules for 2026.

By Anurag Singh
Updated on May 10, 2026
Category: Tutorial
Share article
Linux VPS Network Security Configuration Tutorial: Complete UFW, iptables, and Port Scanning Protection Setup for 2026

Understanding VPS Network Security Fundamentals

Network security forms the first line of defense for your VPS infrastructure. Without proper configuration, your server remains vulnerable to port scans, brute force attacks, and unauthorized access attempts. These threats can compromise your entire hosting environment.

This Linux VPS network security configuration tutorial covers comprehensive hardening for Ubuntu 24.04 and AlmaLinux 9 instances. You'll implement multiple security layers including UFW firewall rules, advanced iptables configurations, and automated port scanning protection.

The configuration methods shown here work across all major VPS hosting environments. HostMyCode VPS hosting provides root access on all plans. This allows you to implement these security measures without restrictions.

Prerequisites and Initial Server Preparation

Before starting the network security configuration, ensure your VPS meets these requirements:

  • Fresh Ubuntu 24.04 LTS or AlmaLinux 9 installation
  • Root or sudo access to the server
  • SSH access configured with key-based authentication
  • Backup of current network configuration files

Connect to your VPS via SSH and update the system packages:

# Ubuntu 24.04
sudo apt update && sudo apt upgrade -y

# AlmaLinux 9
sudo dnf update -y

Verify your current network interfaces and IP configuration:

ip addr show
ss -tuln

This shows active network interfaces and listening ports. Document these details before making changes.

UFW Firewall Configuration and Rule Management

UFW (Uncomplicated Firewall) provides an intuitive interface for managing iptables rules. Install and configure UFW with security-focused defaults.

Installing and Enabling UFW

Install UFW on Ubuntu (typically pre-installed) or AlmaLinux:

# Ubuntu 24.04
sudo apt install ufw -y

# AlmaLinux 9
sudo dnf install ufw -y

Configure default policies before enabling UFW:

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw default deny forward

Add essential service rules before enabling the firewall:

# Allow SSH (critical - configure this first)
sudo ufw allow 22/tcp comment 'SSH access'

# Allow HTTP and HTTPS
sudo ufw allow 80/tcp comment 'HTTP web server'
sudo ufw allow 443/tcp comment 'HTTPS web server'

# Allow specific management ports if needed
sudo ufw allow 2083/tcp comment 'cPanel HTTPS'
sudo ufw allow 8443/tcp comment 'Plesk HTTPS'

Enable UFW after configuring essential rules:

sudo ufw --force enable
sudo ufw status verbose

Advanced UFW Rule Configuration

Create specific rules for database and application servers:

# MySQL/MariaDB (restrict to specific IPs)
sudo ufw allow from 192.168.1.0/24 to any port 3306 comment 'MySQL internal'

# PostgreSQL (application server access only)
sudo ufw allow from 10.0.0.100 to any port 5432 comment 'PostgreSQL app server'

# Redis (local connections only)
sudo ufw allow from 127.0.0.1 to any port 6379 comment 'Redis local'

Implement rate limiting for SSH connections:

sudo ufw limit ssh comment 'SSH rate limiting'
sudo ufw delete allow 22/tcp

This automatically blocks IPs with more than 6 connection attempts in 30 seconds.

Advanced iptables Security Configuration

UFW handles most scenarios. However, direct iptables configuration provides granular control over network traffic. Implement advanced rules for enhanced protection.

Creating Custom iptables Chains

Create specialized chains for different traffic types:

# Create custom chains
sudo iptables -N SCAN_PROTECT
sudo iptables -N RATE_LIMIT
sudo iptables -N LOG_DROP

Configure the scan protection chain:

# Detect and block port scans
sudo iptables -A SCAN_PROTECT -p tcp --tcp-flags SYN,ACK,FIN,RST RST -m limit --limit 1/s --limit-burst 2 -j RETURN
sudo iptables -A SCAN_PROTECT -p tcp --tcp-flags SYN,ACK,FIN,RST RST -j LOG_DROP

# Block NULL packets
sudo iptables -A SCAN_PROTECT -p tcp --tcp-flags ALL NONE -j LOG_DROP

# Block XMAS packets
sudo iptables -A SCAN_PROTECT -p tcp --tcp-flags ALL ALL -j LOG_DROP

Implementing Connection State Tracking

Configure stateful connection tracking for better security:

# Allow established and related connections
sudo iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

# Drop invalid connections
sudo iptables -A INPUT -m conntrack --ctstate INVALID -j LOG_DROP

# Limit new connections per IP
sudo iptables -A INPUT -p tcp --dport 80 -m conntrack --ctstate NEW -m limit --limit 25/minute --limit-burst 20 -j ACCEPT

Add geographic blocking if needed (requires geoip module):

# Install geoip database
sudo apt install xtables-addons-common libtext-csv-xs-perl -y

# Block specific countries (example: block CN, RU)
sudo iptables -A INPUT -m geoip --src-cc CN,RU -j LOG_DROP

Port Scanning Detection and Prevention

Automated detection and blocking of port scanning attempts protects against reconnaissance attacks. These attacks precede more serious intrusion attempts.

Installing and Configuring psad

psad (Port Scan Attack Detector) analyzes iptables logs and automatically blocks scanning IPs:

# Ubuntu installation
sudo apt install psad -y

# AlmaLinux installation
sudo dnf install epel-release -y
sudo dnf install psad -y

Configure psad settings in /etc/psad/psad.conf:

sudo nano /etc/psad/psad.conf

Key configuration parameters:

# Email alerts
EMAIL_ADDRESSES         admin@yourdomain.com;
HOSTNAME                your-vps-hostname;

# Blocking thresholds
PORT_RANGE_SCAN_THRESHOLD    1;
SCAN_TIMEOUT                 3600;

# Automatic blocking
ENABLE_AUTO_IDS              Y;
AUTO_IDS_DANGER_LEVEL        2;
AUTO_BLOCK_TIMEOUT           86400;

Start and enable psad:

sudo systemctl enable psad
sudo systemctl start psad
sudo psad --sig-update

Custom Port Scan Detection Script

Create a custom script for enhanced port scan detection:

sudo nano /usr/local/bin/scan_detector.sh
#!/bin/bash

# Custom port scan detector
LOG_FILE="/var/log/scan_attempts.log"
BLOCK_TIME=3600  # Block for 1 hour
THRESHOLD=10     # Connections per minute

# Monitor connection attempts
while read line; do
    IP=$(echo $line | awk '{print $1}')
    COUNT=$(grep -c "$IP" $LOG_FILE)
    
    if [ $COUNT -gt $THRESHOLD ]; then
        # Block IP with iptables
        iptables -I INPUT -s $IP -j DROP
        
        # Schedule unblock
        echo "iptables -D INPUT -s $IP -j DROP" | at now + $BLOCK_TIME seconds
        
        logger "Blocked scanning IP: $IP"
    fi
done < <(tail -f /var/log/auth.log | grep "Failed password")

Make the script executable and create a systemd service:

sudo chmod +x /usr/local/bin/scan_detector.sh

sudo nano /etc/systemd/system/scan-detector.service
[Unit]
Description=Port Scan Detector
After=network.target

[Service]
Type=simple
ExecStart=/usr/local/bin/scan_detector.sh
Restart=always
User=root

[Install]
WantedBy=multi-user.target

Network Interface Security Hardening

Harden network interface parameters to prevent various network-based attacks. This improves overall security posture.

Kernel Parameter Optimization

Configure kernel parameters for enhanced network security:

sudo nano /etc/sysctl.d/99-network-security.conf
# Disable IP forwarding (unless needed for routing)
net.ipv4.ip_forward = 0
net.ipv6.conf.all.forwarding = 0

# Disable source routing
net.ipv4.conf.all.accept_source_route = 0
net.ipv6.conf.all.accept_source_route = 0

# Disable redirects
net.ipv4.conf.all.accept_redirects = 0
net.ipv6.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0

# Enable reverse path filtering
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1

# Log martian packets
net.ipv4.conf.all.log_martians = 1

# Ignore ping requests
net.ipv4.icmp_echo_ignore_all = 1

# TCP SYN flood protection
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 2048
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_syn_retries = 5

Apply the new settings:

sudo sysctl -p /etc/sysctl.d/99-network-security.conf

Network Interface Monitoring

Set up continuous monitoring of network interfaces:

sudo nano /usr/local/bin/network_monitor.sh
#!/bin/bash

# Network interface monitoring script
INTERFACE="eth0"  # Adjust to your interface
LOG_FILE="/var/log/network_monitor.log"

while true; do
    # Monitor interface statistics
    RX_PACKETS=$(cat /sys/class/net/$INTERFACE/statistics/rx_packets)
    TX_PACKETS=$(cat /sys/class/net/$INTERFACE/statistics/tx_packets)
    RX_ERRORS=$(cat /sys/class/net/$INTERFACE/statistics/rx_errors)
    TX_ERRORS=$(cat /sys/class/net/$INTERFACE/statistics/tx_errors)
    
    TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')
    echo "$TIMESTAMP - RX: $RX_PACKETS, TX: $TX_PACKETS, RX_ERR: $RX_ERRORS, TX_ERR: $TX_ERRORS" >> $LOG_FILE
    
    # Alert on high error rates
    if [ $RX_ERRORS -gt 100 ] || [ $TX_ERRORS -gt 100 ]; then
        logger "High network error rate detected on $INTERFACE"
    fi
    
    sleep 300  # Check every 5 minutes
done

Intrusion Detection System Integration

Integrate network security with intrusion detection for comprehensive monitoring. This provides real-time alerts and automated responses to security events.

Install and configure AIDE (Advanced Intrusion Detection Environment):

# Ubuntu installation
sudo apt install aide -y

# AlmaLinux installation
sudo dnf install aide -y

Initialize AIDE database:

sudo aideinit
sudo mv /var/lib/aide/aide.db.new /var/lib/aide/aide.db

Create automated AIDE checks:

sudo nano /etc/cron.daily/aide-check
#!/bin/bash

# Run AIDE check and email results
AIDE_REPORT="/tmp/aide-report-$(date +%Y%m%d).txt"

aide --check > $AIDE_REPORT 2>&1

if [ $? -ne 0 ]; then
    # Send alert email
    mail -s "AIDE Security Alert - $(hostname)" admin@yourdomain.com < $AIDE_REPORT
    
    # Log to syslog
    logger -p security.warning "AIDE detected file system changes"
fi

# Clean up old reports
find /tmp -name "aide-report-*.txt" -mtime +7 -delete

For production environments requiring managed security monitoring, HostMyCode managed VPS hosting includes 24/7 security monitoring and automated threat response.

Secure your VPS with professional network hardening and monitoring. HostMyCode VPS hosting provides root access for complete security control, while our managed VPS plans include expert security configuration and 24/7 monitoring.

Testing and Validation

Verify your network security configuration with comprehensive testing. This ensures all security measures function correctly without blocking legitimate traffic.

Port Scan Testing

Test your configuration against common scanning techniques:

# Install nmap for testing
sudo apt install nmap -y

# Test from external source (use another server)
nmap -sS -O your-vps-ip
nmap -sU -p 1-1000 your-vps-ip

# Check for stealth scan detection
nmap -sF -T4 your-vps-ip

Verify UFW and iptables rules:

# Check UFW status
sudo ufw status numbered

# Review iptables rules
sudo iptables -L -n -v
sudo iptables -L SCAN_PROTECT -n -v

Performance Impact Assessment

Measure the performance impact of security configurations:

# Network performance testing
iperf3 -s  # On test server
iperf3 -c your-vps-ip -t 60  # From client

# Connection handling test
ab -n 10000 -c 100 http://your-vps-ip/

Monitor system resources during testing:

# Monitor CPU and memory usage
top -b -n 1 | head -20

# Check network interface statistics
cat /proc/net/dev

# Review connection tracking table
cat /proc/net/nf_conntrack | wc -l

Maintenance and Monitoring Procedures

Establish regular maintenance procedures to keep your network security configuration effective and up-to-date.

Automated Security Updates

Configure automatic security updates while maintaining stability:

# Ubuntu unattended upgrades
sudo apt install unattended-upgrades -y
sudo nano /etc/apt/apt.conf.d/50unattended-upgrades

Enable security-only updates:

Unattended-Upgrade::Allowed-Origins {
    "${distro_id}:${distro_codename}-security";
    "${distro_id}ESMApps:${distro_codename}-apps-security";
    "${distro_id}ESM:${distro_codename}-infra-security";
};

For detailed security automation guidance, review our automated security patching tutorial. This guide covers comprehensive update management strategies.

Log Analysis and Alerting

Set up centralized log analysis for security events:

sudo nano /usr/local/bin/security_log_analyzer.sh
#!/bin/bash

# Security log analyzer
LOG_DIR="/var/log"
ALERT_EMAIL="admin@yourdomain.com"
TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')

# Check for failed login attempts
FAILED_LOGINS=$(grep "Failed password" $LOG_DIR/auth.log | wc -l)
if [ $FAILED_LOGINS -gt 50 ]; then
    echo "High number of failed login attempts: $FAILED_LOGINS" | \
    mail -s "Security Alert - Failed Logins" $ALERT_EMAIL
fi

# Check for blocked IPs
BLOCKED_IPS=$(iptables -L INPUT -n | grep DROP | wc -l)
if [ $BLOCKED_IPS -gt 20 ]; then
    echo "High number of blocked IPs: $BLOCKED_IPS" | \
    mail -s "Security Alert - Blocked IPs" $ALERT_EMAIL
fi

# Check system load
LOAD=$(uptime | awk '{print $10}' | cut -d',' -f1)
if (( $(echo "$LOAD > 5.0" | bc -l) )); then
    echo "High system load detected: $LOAD" | \
    mail -s "Performance Alert - High Load" $ALERT_EMAIL
fi

Schedule regular security analysis:

# Add to crontab
echo "*/15 * * * * /usr/local/bin/security_log_analyzer.sh" | sudo crontab -

Troubleshooting Common Issues

Address frequent network security configuration problems and their solutions.

Connection Timeout Issues

If legitimate connections are being blocked:

# Check current connections
sudo netstat -tuln
sudo ss -tulpn

# Review recent blocks
sudo iptables -L INPUT -n -v | grep DROP

# Check UFW logs
sudo tail -f /var/log/ufw.log

Temporarily disable rate limiting for troubleshooting:

# Remove rate limiting rules
sudo ufw delete limit ssh
sudo ufw allow ssh

# Test connections, then re-enable
sudo ufw delete allow ssh
sudo ufw limit ssh

Performance Degradation

If network security rules cause performance issues:

# Check connection tracking table size
cat /proc/sys/net/netfilter/nf_conntrack_count
cat /proc/sys/net/netfilter/nf_conntrack_max

# Increase connection tracking table if needed
echo 'net.netfilter.nf_conntrack_max = 131072' >> /etc/sysctl.d/99-network-security.conf
sudo sysctl -p

Monitor rule processing performance:

# Check iptables rule hit counts
sudo iptables -L INPUT -n -v --line-numbers

# Optimize rule order (most frequently matched first)
sudo iptables -I INPUT 1 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

Frequently Asked Questions

How often should I review and update network security rules?

Review network security configurations monthly and after any infrastructure changes. Update rules immediately when new vulnerabilities are discovered or attack patterns change. Monitor security logs daily for unusual activity patterns.

What's the difference between UFW and direct iptables configuration?

UFW provides a simplified interface for common firewall tasks and is ideal for standard configurations. Direct iptables offers granular control for advanced rules like custom chains, connection limiting, and complex traffic filtering. Use UFW for basic protection and iptables for advanced security requirements.

How can I prevent legitimate traffic from being blocked by security rules?

Implement whitelist rules for known good IPs and use appropriate rate limiting thresholds. Monitor logs for false positives. Create exception rules for your management IP addresses and regularly test connectivity from different locations.

Should I disable ICMP completely for security?

Disabling ICMP ping responses reduces reconnaissance opportunities. However, completely blocking ICMP can break network diagnostics. It also breaks Path MTU Discovery. Instead, limit ICMP rate and disable only ping responses while allowing necessary ICMP types like destination unreachable.

How do I backup and restore iptables rules?

Use iptables-save > /etc/iptables/rules.v4 to backup current rules. Use iptables-restore < /etc/iptables/rules.v4 to restore them. For UFW, backup /etc/ufw/ directory contents. Always test restored rules in a controlled environment before applying to production servers.