VPS Experience (Part 2): Configuring Fedora for VPS
Since I rented my current VPS web last year, I had been using CentOS 7.4 for its operating system. It is derived from Red Hat Enterprise Linux (RHEL) project which offers similar stability to an enterprise-grade server. Unfortunately, a few days ago I discovered that some packages in CentOS were really outdated, even on the latest version (7.6). Those packages are related to software development, such as GCC, Clang toolchains, as well as Boost library. The available GCC on the repository cannot even compile C++14! At that time, I was thinking to use my VPS to compile some codes I wrote, which depends on several libraries that I have to also compile manually. Thus, I could not manage to do my experiment and promised myself to destroy the VPS and rebuild from scratch.
My VPS provider provides several operating systems, which are OpenSUSE, Fedora, Ubuntu, CentOS, Scientific, and Debian. I am not really a fan of Debian and its derivatives, such as Ubuntu, despite that now I am heavily using Ubuntu on my Windows Subsystem for Linux (WSL) platform. I thought that Ubuntu is too mainstream and configured mostly for workstation system. I might be wrong and this is purely a taste, so don’t start any distro war here :P. On the other hand, I used OpenSUSE also quite extensively as my desktop workstation. But my VPS providers only provide OpenSUSE Leap rather than rolling-release, which I expect (and experienced) it also has slow-moving repositories. So, I picked Fedora as my new distro for my VPS.
My previous choice on CentOS was also influenced by my experience in using Fedora as my main office workstation. I am quite accustomed to its commands and file system structures, and I have rarely got a big issue when using it as my daily driver (except for installing Tizen Studio, of course, :P). And it has a quite frequent update cycle, which therefore I will not miss any new software update or utilizing the latest feature. But apparently, my VPS only provides an installer for Fedora 27 (currently is 29), and it is not working. The working version that is provided by my provider is Fedora 21, which is way too old. Fortunately, my provider offers a way to upload my own ISO, and I can install directly. My provider also supports web-based display console, which allows me to install the OS easily without the need for an automated script to perform the installation. I will describe some server configurations in this post.
Partitioning and Mount Point
Partition Scheme
I usually tried to separate between system files and user content into separate partitions. This allows me to perform an operating system reinstallation without backing up my personal data. In my regular Linux configuration, I put my user data in my /home folder. But for server configuration, /home folder is not adequate for storing other user content, such as web sites. Web sites usually are stored under /var/www folder and have its own permission model. Therefore, we need to also consider files under /var folder on our partition design.
I own 100GB of space from my VPS provider and want to allocate the storage into two part: system and data. I am thinking to reserve 70% for data, and the remaining for the system. 30GB for the system is fairly adequate considering that this is a headless VPS that does not require workstation application and window manager. The data space will be shared for both home and var. I don’t want to split the data space into two different partitions and prefer to combine both folders to reside in the same space.
Mount Point and fstab
Putting two root-folders (/home and /var) into one partition is not a trivial task. Basically, Linux only allows a partition to be mounted to one specific point. For example, partition /dev/sda1 is mounted to / (root), while /dev/sda2 is mounted to /home. It is not possible to mount /dev/sda2 to two different points such as /home and /var altogether. However, there is a workaround for this which therefore allows us to put /home and /var folder into one partition.
We can utilize Bind mount to remount an existing folder to a new mount point so the new mount point will act similar to a symbolic link to an existing folder. To apply this method, we need to mount the partition to a specific folder first as a temporary mount point, then remount using bind to /var and /home folder. The partition should consist of home and var folder only, which will be accessible from the temporary mount point. Below is the fstab configuration:
/dev/sda2 /mnt/data xfs defaults 0 0 /mnt/data/home /home none bind /mnt/data/var /var none bind
In fstab configuration above, we mount our second partition temporarily at /mnt/data point, then remount the /mnt/data/home and /mnt/data/var to /home and /var respectively. The remounted folder will behave like a regular mount point and it will work with SELinux policy system which will be discussed at later part.
Software Packages
Nginx
I used Nginx (spelled eNGINe X) for
Configuring Nginx with PHP-FPM to serve WordPress site is also quite straightforward. Nginx act as a reverse proxy, delegating request processing for PHP pages to PFP-FPM service. Additionally, my WordPress installation is configured with Polylang, where I handled different language for the site under different subdomain. All these settings are simply need to be put on the Nginx configuration script without additional hassle. Below is the example of my Nginx configuration script for my installation:
server {
server_name www.heliosky.com heliosky.com id.heliosky.com en.heliosky.com;
root /var/www/wordpress;
index index.php index.html index.htm;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
location / {
try_files $uri $uri/ /index.php?$args;
}
location ~ \.php$ {
try_files $uri =404;
fastcgi_pass unix:/var/run/php-fpm.sock;
fastcgi_index index.php;
fastcgi_param PHP_VALUE "upload_max_filesize=8M \n post_max_size=8M";
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi_params;
}
listen [::]:443 ssl; # managed by Certbot
listen 443 ssl; # managed by Certbot
# Some ssl config by Certbot...
}
MariaDB
MariaDB is a fork of MySQL by the creators of MySQL themselves. It is intended to be a direct replacement of MySQL, that will remain free and open source, in case of Oracle run amok with their stuff :P. So, MariaDB is completely compatible with MySQL and applications that are built to work with MySQL, including WordPress. Moreover, all necessary tools and commands are still backward compatible with MySQL, including the command line client and additional installation tools. The decision of using MariaDB is because WordPress requires it and WordPress cannot work with other RDBMS. If you are thinking to use other RDBMS, such as PostgreSQL, no you can’t.
Not many mysql_secure_installation. Run this tool to preconfigure your MariaDB (or MySQL) installation with
PHP-FPM
The server is going to use PHP7 for a more up-to-date and secure platform. As the server is going to use Nginx to serve HTTP requests, PHP-FPM is the only choice to serve PHP content as there is no PHP module similar to mod_php as in Apache stack. PHP-FPM, or FastCGI Process Manager, is a process manager that pools PHP handler to serve PHP using FastCGI mechanism. In old era, CGI is outperformed by module-based handlers like mod_php or ISAPI because the cost of interprocess communication (IPC) between the web server and CGI handler, which both of them run on separate processes, is too high. But nowadays, FastCGI is an improvement on CGI-based stack which provides greater speed comparable with module-based handlers.
PHP-FPM hosts a pool of handlers which are ready to be dispatched when a PHP content is requested by the web server. Web server communicates with it via UNIX socket and FastCGI protocol, and wait until the handler completes the processing. The handler runs on a separate process and can also be configured to run as a different user. Unlike traditional CGI, which shuts down after a request is processed, FastCGI allows the process to keep running, eliminating the overhead of restarting the process on each request. Moreover, it also enables the handler to cache the execution thus making the processing faster.
Various discussions have compared Apache + mod_php and Nginx + PHP-FPM stacks, and it is concluded that PHP-FPM stack can perform with similar performance as mod_php in real life application. The overhead that is caused by the interprocess communication between Nginx and PHP-FPM does not significantly affect the performance. It rather improves security and reliability altogether, for example, the failure in PHP system does not propagated to the web server, avoiding domino effect of server crash.
Configuring Security
SSH Configuration
SSH is the main access control to the server. Although many VPS providers provide VNC remote desktop, SSH remains the most important command control as it is lightweight and secure, if configured properly. Therefore, it is compulsory to perform specific configuration for SSH server and not using default settings provided by the distro installation. The settings include: using SSH key, disable password-based login, disable root login and use non-default port.
First, we need to create SSH key and install it on server. You can generate the key in your local computer, or directly on the server. To generate the key, we can use ssh-keygen tool: ssh-keygen -t rsa -b 4096. This command generates a 4096-bit RSA keypair, which is considered quite secure at this moment. The key will be stored at .ssh folder as id_rsa and id_rsa.pub. Don’t forget to use passphrase to ensure the key can only be accessed by you.
To install the key, run ssh-copy-id command: ssh-copy-id -i <keyfile> <username>@<host>. This will install the key into the server. The command will request for your password prior to the key installation. After you install it, make sure to secure your key and remove it from the server if you are generating it there.
The next one is to configure the sshd on several settings: disable password-based login, disable root login, and change the default port. Here’s the configuration
Port <PORTNUMBER> PermitRootLogin no PasswordAuthentication no
The reason for changing the port number is to avoid brute force attack from botnet which continuously tries to authenticate on normal SSH port 22. When I installed my VPS for the first time, I noticed a large amount of entry in my /var/log/secure log. Apparently, many botnet tries to attack the SSH connection by doing brute force attack. I tried to trace back the origin IP address, and most of them came from China. I think it is fairly common that botnet continuously tries to attack insecure server to gain access, so you have to be careful with that.
Changing the port does not necessarily makes everything secure, where an attacker practically can perform port scanning against your server. However, changing the port number can eliminate passive attack like that and interested attackers need to analyze your server manually if they want to gain access. Make sure you also use a port that is not used in your system, and avoid popular port number altogether. Also, it is a good idea if the number is something that you can memorize yourself, so it will similarly acts like a passphrase to access your server. Finally, changing the port number will also require you to change firewall and SELinux configuration, which will be discussed later on.
Firewall Configuration
Next important thing is configuring firewall on your system. Never ever think to disable firewall just because you are lazy to configure it. Fedora provides firewall-cmd to handle firewall configuration easier than using iptables. Yes, I agree that iptables is very hard to configure and can be painful just to make it work correctly. With firewall-cmd, configuring the firewall is easier and more straightforward as it is organized logically using zones and services. For simple hosting server, this is more than adequate.
Some firewall settings are automatically configured by installation script, such as when installing web server. So, no action needed to configure those server. Manual configuration will be required if you make nonstandard changes, such as the previous SSH port change. To configure that, use this example command: sudo firewall-cmd --permanent --service=ssh --add-port=<PORT>/tcp.
DAC Configuration for PHP-FPM
PHP-FPM can be configured to run under a specific user, which can benefit from Discretionary Access Control (DAC) configuration and privilege separation. To configure that, we need to create a new user account first using useradd command. The user should has no login access, and no home directory. Here’s the example command: sudo useradd -M -s /sbin/nologin php-user. It is a good idea to make multiple users if you have multiple PHP application in your server.
After creating the user, we need to configure PHP-FPM pool. The pool configuration is stored in /etc/php-fpm.d/. There is a default configuration www.conf which can be used as template. Each configuration file stored in this folder will spawn a separate pool. So, you can create multiple pool for different PHP application as you need, and each of them will not interfere with each other. I suggest you copy www.conf file and creates your own new file, and rename the default www.conf so an unused PHP-FPM pool will not be spawned. Below is some changes to the default configuration:
[php-poolname] user = php-user group = php-user listen = /var/run/php-myapp.sock listen.owner = nginx listen.group = nginx listen.mode = 0660 listen.allowed_clients = 127.0.0.1 chdir = /var/www/myphpapp
The Nginx is also have to be configured to access the correct socket. Change the fastcgi_pass entry as shown in the example Nginx configuration above with the correct socket file path.
DAC for MariaDB
MariaDB can also be configured to benefit Linux DAC system for its user permission. Normally, MariaDBuses its own user database to perform authentication. By configuring the DAC, MariaDB can authenticate database user using Linux user instead, eliminating the requirement to specify a password in configuration file. This is similar concept with Windows style DAC where the authentication is performed using Windows User Accounts or Active Directory, although less powerful in Linux.
Since the PHP-FPM now has its own user account to run its worker process, we can set MariaDB to authenticate this user to the database using Unix socket, when the user is trying to establish the connection using the socket file. The required library has to be installed first, by issuing this command on MariaDB console: INSTALL SONAME 'auth_socket';. Then, we create a user in MariaDB with exact same name and authenticate it using auth_socket: CREATE USER 'php-user'@localhost IDENTIFIED VIA unix_socket;. Finally, you can grant database access to this user as required.
In this way, the PHP-FPM worker, which runs as php-user, will be able to authenticate itself to MariaDB without supplying a password. And other process which runs as different user cannot access or impersonate php-user when accessing the database. According to the PHP documentation, mysqli_connect function still requires a username to be passed to the function, therefore, you are still required to fill the username field in WordPress config and set password to null.
SELinux Policies
The final part is configuring SELinux policies. This is an important security feature which imposes a Mandatory Access Control (MAC) on
You need to ensure policycoreutils-pythonsemanagerestorecon
First, we need to add the custom port for SSH to SELinux policy. Although we have opened the port on semanage port -a -t ssh_port_t -p tcp <your-new-port>
The next part is configuring our web application root folder. SELinux /var/www/html/* folder, SELinux has a preconfigured file context for that kind of installation, and you can skip this step. Otherwise, you will also have to modify the file context to ensure your WordPress installation works correctly.
To modify the file context policy, you can either modify the file (/etc/selinux/targeted/contexts/files/file_contexts) yourself using semanagefile_contexts
/var/www/bloggue/wp-content(/.*)? system_u:object_r:httpd_sys_rw_content_t:s0 /var/www/bloggue/wp_backups(/.*)? system_u:object_r:httpd_sys_rw_content_t:s0
The policy describes var/www/bloggue/wp-content/var/www/bloggue/wp_backups folders and all httpd_sys_rw_content_t/var/wwwhttpd_sys_content_t
Then finally, to apply the file context rules, you have to use restorecon command under the folder that you want to be relabeled. If you are applying custom mount point and partition as I did, you will have to relabel the /home/varmnt_trestorecon -R <folder-to-relabel>
If you have issues with your application, for example, your web application is not properly loaded, it is worth to read the log file /var/log/securehttpd_can_network_connectsetsebool httpd_can_network_connect truegetsebool -a
Conclusion
This explanation roughly covers everything I do to set up my simple VPS for my blog. You can try to apply it on your own setup especially if you want to have fully functional VPS server with basic security in mind. This far from perfect settings, but at least it is sufficient for hosting a small web application
Should you also have any idea, discussion, or challenge the explanation, feel free to drop a comment :).
References
- https://www.chriswiegman.com/2011/10/fastcgi-vs-suphp-vs-cgi-vs-mod_php-dso/
- https://www.ssh.com/ssh/copy-id
- https://www.tecmint.com/add-users-in-linux/
- https://mariadb.com/kb/en/library/authentication-plugin-unix-socket/
- https://wiki.centos.org/HowTos/SELinux#head-ad837f60830442ae77a81aedd10c20305a811388
- Image by Victorgrigas – Creative Common Attribution-Share Alike 3.0 Unported
