HOWTO Ubuntu 18.04 Virtual Web Hosting With Apache, PHP, vsFTPD and Let’s Encrypt

Introduction

The focus of this howto is for those users who need to host their own domains and maybe a few customer domains. This is not aimed at being used for mass web hosting.

There are many ways to do virtual websites under linux. The technique I am going to use is multiple domains on one ip address. I’m using standard linux users to log into the virtual domains.

Setting Up The Base Server

For a dedicated server start with the base server setup:

HOWTO Ubuntu 18.04 LTS Base Server Setup

Introduction All of our servers will start with this install. This base server is based on Ubuntu 18.04 LTS Server.  I don’t explain much in the howto so if you have a question leave a comment or use Google. Downloading … Continue reading

NOTE: If you don’t follow the base server setup then you may run into problems with this howto.

Install Software

We need to install an FTP server and Let’s Encrypt. So type the following:
> sudo add-apt-repository ppa:certbot/certbot
> sudo apt update
> sudo apt install vsftpd python-certbot-apache

Setup Default User Directory

A new user’s directory needs to have some files and folders created for them.  We will modify the user skel directory so when a new user is created the required folder structure will be there.

Type the following.
> sudo mkdir -p /etc/skel/{website,logs,cgi-bin,backup}
> sudo echo “HELLO WORLD” > /etc/skel/website/index.html

Configuring vsftpd

Lets create the configuration file.  Replace the contents of /etc/vsftpd.conf with the text below.

listen=NO
listen_ipv6=YES
anonymous_enable=NO
local_enable=YES
write_enable=YES
#local_umask=022
dirmessage_enable=YES
use_localtime=YES
xferlog_enable=YES
connect_from_port_20=YES
chroot_local_user=YES
secure_chroot_dir=/var/run/vsftpd/empty
pam_service_name=vsftpd
rsa_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
rsa_private_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
ssl_enable=NO
allow_writeable_chroot=YES
pasv_enable=Yes
pasv_min_port=40000
pasv_max_port=40100

Start vsftpd.
> sudo systemctl enable vsftpd
> sudo systemctl start vsftpd.service

Configuring Apache

Most of the apache configuration is already done.  We are going to do some changes to make managing websites easier.

Create the virtual host config file. I defined macros to make virtual host creation easier.  I also turn on compression. Create /etc/apache2/conf-available/virtual.conf with the following:

# Go ahead and accept connections for these vhosts
# from non-SNI clients
SSLStrictSNIVHostCheck off

# define a macro for the virtual hosts
# the user's directory should be setup as follows:
# |- cgi-bin
# |- logs
# |- website
# |- ssl
#
LoadModule macro_module modules/mod_macro.so

<Macro virtHost $type $user $host>
    use $type $host

    ServerName $host
    ServerAlias www.$host 
    DocumentRoot /home/$user/website
    ScriptAlias "/cgi-bin/" "/home/$user/cgi-bin"
    LogFormat "%h %l %u %t \"%r\" %>s %b" common
    CustomLog /home/$user/logs/access_log common
    ErrorLog /home/$user/logs/error_log
    <Directory /home/$user/website>
      DirectoryIndex index.html index.php
      Options Indexes FollowSymLinks
      AllowOverride All
      Require all granted

      # setup file compression
      use CompressFiles

      # setup browser caching
      use BrowserCache

      # disable hotlinking for some files
      use DisableHotLink $host
    </Directory>
  </VirtualHost>
</Macro>

<Macro BrowserCache>
  # Enable expires cache
  <IfModule mod_expires.c>
    ExpiresActive On
    ExpiresByType text/css "access 1 month"
    ExpiresByType text/html "access 1 month"
    ExpiresByType image/gif "access 1 year"
    ExpiresByType image/png "access 1 year"
    ExpiresByType image/jpg "access 1 year"
    ExpiresByType image/jpeg "access 1 year"
    ExpiresByType image/x-icon "access 1 year"
    ExpiresByType application/pdf "access 1 month"
    ExpiresByType application/javascript "access 1 month"
    ExpiresByType text/x-javascript "access 1 month"
    ExpiresDefault "access 1 month"
  </IfModule>

  # Cache-Control Headers
  <ifModule mod_headers.c>
    <filesMatch "\.(ico|jpe?g|png|gif|swf)$">
      Header set Cache-Control "public"
    </filesMatch>
    <filesMatch "\.(css)$">
      Header set Cache-Control "public"
    </filesMatch>
    <filesMatch "\.(js)$">
      Header set Cache-Control "private"
    </filesMatch>
    <filesMatch "\.(x?html?|php)$">
      Header set Cache-Control "private, must-revalidate"
    </filesMatch>
  </ifModule>
</Macro>

<Macro CompressFiles>
  # enable compression
  <IfModule mod_deflate.c>
    AddOutputFilterByType DEFLATE "application/atom+xml"
    AddOutputFilterByType DEFLATE "application/javascript"
    AddOutputFilterByType DEFLATE "application/json"
    AddOutputFilterByType DEFLATE "application/ld+json"
    AddOutputFilterByType DEFLATE "application/manifest+json"
    AddOutputFilterByType DEFLATE "application/rdf+xml"
    AddOutputFilterByType DEFLATE "application/rss+xml"
    AddOutputFilterByType DEFLATE "application/schema+json"
    AddOutputFilterByType DEFLATE "application/vnd.geo+json"
    AddOutputFilterByType DEFLATE "application/vnd.ms-fontobject"
    AddOutputFilterByType DEFLATE "application/x-font"
    AddOutputFilterByType DEFLATE "application/x-font-opentype"
    AddOutputFilterByType DEFLATE "application/x-font-otf"
    AddOutputFilterByType DEFLATE "application/x-font-truetype"
    AddOutputFilterByType DEFLATE "application/x-font-ttf"
    AddOutputFilterByType DEFLATE "application/x-javascript"
    AddOutputFilterByType DEFLATE "application/x-web-app-manifest+json"
    AddOutputFilterByType DEFLATE "application/xhtml+xml"
    AddOutputFilterByType DEFLATE "application/xml"
    AddOutputFilterByType DEFLATE "font/eot"
    AddOutputFilterByType DEFLATE "font/otf"
    AddOutputFilterByType DEFLATE "font/ttf"
    AddOutputFilterByType DEFLATE "font/opentype"
    AddOutputFilterByType DEFLATE "image/bmp"
    AddOutputFilterByType DEFLATE "image/svg+xml"
    AddOutputFilterByType DEFLATE "image/vnd.microsoft.icon"
    AddOutputFilterByType DEFLATE "image/x-icon"
    AddOutputFilterByType DEFLATE "text/cache-manifest"
    AddOutputFilterByType DEFLATE "text/css"
    AddOutputFilterByType DEFLATE "text/html"
    AddOutputFilterByType DEFLATE "text/javascript"
    AddOutputFilterByType DEFLATE "text/plain"
    AddOutputFilterByType DEFLATE "text/vcard"
    AddOutputFilterByType DEFLATE "text/vnd.rim.location.xloc"
    AddOutputFilterByType DEFLATE "text/vtt"
    AddOutputFilterByType DEFLATE "text/x-component"
    AddOutputFilterByType DEFLATE "text/x-cross-domain-policy"
    AddOutputFilterByType DEFLATE "text/xml"
  </IfModule>
</Macro>

<Macro DisableHotLink $host >
  # Disable file hotlinking - jpg jpeg png gif pdf
  <IfModule mod_rewrite.c>
    RewriteEngine on
    RewriteCond %{HTTP_REFERER} !^$
    RewriteCond %{HTTP_REFERER} !^http(s)?://(www\.)?$host [NC]
    RewriteCond %{HTTP_REFERER} !^http(s)?://(www\.)?google.com [NC]
    RewriteRule \.(jpg|jpeg|png|gif|pdf)$ – [NC,F,L]
  </IfModule>
</macro>

<Macro VHost443 $host >
  <VirtualHost *:443>
    SSLEngine on
    SSLProtocol all -SSLv2
    SSLCipherSuite HIGH:MEDIUM:!aNULL:!MD5
    SSLCertificateFile /etc/letsencrypt/live/$host/cert.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/$host/privkey.pem
    SSLCertificateChainFile /etc/letsencrypt/live/$host/fullchain.pem
</Macro>

<Macro VHost80 $host >
  <VirtualHost *:80>
</Macro>

Enable the configuration.
> sudo a2enconf virtual

Enable macros and ssl.
> sudo a2enmod macro
> sudo a2enmod ssl

Restart apache
> sudo service apache2 restart

Configuring Let’s Encrypt

Let’s Encrypt needs to be configured to auto renew certs. Lets create a daily cron job
> sudo nano -w /etc/cron.daily/letsencrypt

#!/usr/bin/bash
# letsencrypt auto renew
/usr/bin/certbot renew --no-self-upgrade >> /var/log/le-renew.log

Adding a Default Website

Now we will create a default website.  This site will be used when no other website can be found.

Setup a DNS record for the new domain. I won’t cover this here.

Add a user. This user will be associated with the new domain name. Type the following.
> sudo useradd -m -U -s /bin/bash -c “default website” defaultweb
> sudo passwd defaultweb

Add the apache user to the new user’s group.
> sudo usermod -a -G defaultweb www-data

Update directory permissions.
> sudo chmod g+rwx /home/defaultweb
> sudo chown -R defaultweb:defaultweb /home/defaultweb

Create the virtual host file.  For the default server we will use port 80. Past the text below into the file.  Type:
> sudo nano -w /etc/apache2/sites-available/00-default.conf

# Virtual host config file
#
# MACRO FORMAT
# virtHost [type] [user] [host]
#  type = VHost80 or VHost443
#  user = the username of the website
#  host = domain name or virtual host name
#
# Use the line below to configure a site on port 80
use virtHost VHost80 defaultweb myserver.mydomain.tld

# Uncomment the line below once lets encrypt is setup
# use virtHost VHost443 defaultweb myserver.mydomain.tld

Disable the old default site and enable our default site.
> sudo a2dissite 000-default
> sudo a2ensite 00-default

Reload apache config
> sudo service apache2 reload

Test out the new website.  You should get a page that says ‘Hello World’.

Now we will setup lets encrypt for the default website.  The website must be reachable from the internet.  So lets get the cert:
> sudo certbot certonly –webroot -w /home/defaultweb/website/ -d <YOUR_DOMAIN> –email <YOUR_EMAIL_ADDRESS> –agree-tos

Edit /etc/httpd/virtualHosts.d/00-default.conf
Uncomment the last line to enable ssl connections for the virtual host.

Reload apache.
> sudo service apache2 reload

Test it out.  Connect to your default host via https.

Setup Additional Virtual Hosts

Adding a new virtual host is like adding the default virtual host.  Lets go through the steps.

Be sure DNS is configured for the new virtual host.

Setup a new user. This user will be associated with the new domain name. Type the following. Change ‘NEWUSER’ to the username you want.
> sudo useradd -m -U -s /bin/bash -c “Virtual Website User” NEWUSER
> sudo passwd NEWUSER
> sudo usermod -a -G NEWUSER www-data
> sudo chmod g+rwx /home/NEWUSER
> sudo chown -R NEWUSER:NEWUSER /home/NEWUSER

Create the virtual host file.  For the virtual server we will use port 80. Past the text below into the file.  Replace ‘NEWUSER’ with your user name.  Replace NEWVHOST with your hostname.
> sudo nano -w /etc/apache2/sites-available/NEWUSER.conf

# Virtual host config file
#
# MACRO FORMAT
# virtHost [type] [user] [host]
#  type = VHost80 or VHost443
#  user = the username of the website
#  host = domain name or virtual host name
#
# Use the line below to configure a site on port 80
use virtHost VHost80 NEWUSER NEWVHOST

# Uncomment the line below once lets encrypt is setup
# use virtHost VHost443 NEWUSER NEWVHOST

Enable the new site and reload apache config
> sudo a2ensite NEWUSER
> sudo service apache2 reload

Now we will setup lets encrypt for the new website.  The website must be reachable from the internet.  Replace NEWUSER and NEWVHOST with the info you have.  So lets get the cert.
> sudo certbot certonly –webroot -w /home/NEWUSER/website/ -d NEWHOST -d www.NEWHOST –email YOUR_EMAIL_ADDRESS –agree-tos

Edit /etc/httpd/virtualHosts.d/NEWUSER.conf
Uncomment the last line to enable ssl connections for the virtual host.

Reload apache.
> sudo service apache2 reload

Test it out.  Connect to your new host via https.

Conclusion

That’s the complete setup.

HOWTO Ubuntu 18.04 LTS Base Server Setup

Introduction

All of our servers will start with this install. This base server is based on Ubuntu 18.04 LTS Server.  I don’t explain much in the howto so if you have a question leave a comment or use Google.

Downloading the ISO

Visit the Ubuntu website and download the ubuntu 18.04 server ISO.

Initial Install

The install screens are straight forward.  I’m not going to cover them in much detail.  Unless you need to customize a setting, just choose the default settings.  Boot the install DVD.

Click ‘Reboot’ when it appears.

First boot

Reboot the machine when the install finishes.
The OS will boot. Log in. All the commands need to be run as root so lets start a shell with root privilleges.
> sudo bash

Get everything updated and install a couple of items.
> sudo apt update
> sudo apt dist-upgrade

WARNING: My server isn’t directly connected to the internet. The firewall is disabled to help with installation, configuration and testing easier. Once everything is working, turn on the firewall and configure it. I wil remind you to secure your server at the end of this howto.

now reboot the server.

The Second Boot – Installing Additional Packages

We need quite a few other packages. In this howto I’m installing packages regardless if they were already installed by another dependency. This guards against package changes that could cause a package to not be installed. Once again log in to your server.

We need the webmin repo. Create webmin.repo with the text below.
> sudo nano -w /etc/apt/sources.list.d/webmin.list

deb http://download.webmin.com/download/repository sarge contrib

Install the webmin key.
> wget http://www.webmin.com/jcameron-key.asc
> sudo apt-key add jcameron-key.asc

Let’s add a 3rd party repo for Apache:
> add-apt-repository ppa:ondrej/apache2

Now bring everything up to date.
> sudo apt update

Install the following packages. Multiple lines to make cut and paste easier.
> sudo apt install make screen snmp composer libcurl3 unzip
> sudo apt install apache2 php7.2-fpm libapache2-mod-php7.2 mysql-server mysql-client
> sudo apt install libapache2-mod-fcgid php7.2-opcache php-apcu

Install some extra PHP libraries.
> sudo apt install php7.2-gd php7.2-snmp php7.2-mbstring php7.2-mysql
> sudo apt install php7.2-odbc php7.2-imap
> sudo apt install php7.2-xmlrpc php7.2-dba
> sudo apt install php7.2-soap php7.2-zip php7.2-intl php7.2-curl

Now lets install webmin.
> sudo apt install webmin
> sudo systemctl enable webmin
> sudo service webmin start

Configure Apache and PHP

Enable the rewrite module.
> sudo a2enmod rewrite actions fcgid alias proxy_fcgi expires headers

Enable Apache.
> sudo systemctl enable apache2.service

Reload apache.
> sudo systemctl restart apache2.service

Configuring MySQL

Configure mysql.
> sudo mysql_secure_installation

Installing and Configuring phpMyAdmin

I prefer to phpMyAdmin to manage my MySQL databases.

Now install phpMyAdmin.
> sudo apt install phpmyadmin

Restart Apache.
> sudo systemctl restart apache2.service

Installing Postfix

Lets install postfix.
> sudo apt install postfix

When prompted select internet site. Next set the mail server name.

Installing cockpit

I’m trying cockpit as my server admin tool.  Do the following to set it up.
> sudo apt install cockpit
> sudo systemctl start cockpit
> sudo systemctl enable cockpit.socket

You can now login to https://yourserver.tld:9090 to administer your server.

Getting root’s and other’s mail

You need to get some local system user’s mail. We’ll use postfix’s virtual file to get the emails to the right place.

Add the following to /etc/postfix/virtual

root admin@yourdomain.tld
postmaster admin@yourdomain.tld
abuse admin@yourdomain.tld

Now add the configuration option to main.cf
> sudo postconf -e “virtual_alias_maps = hash:/etc/postfix/virtual”
Just a couple commands to wrap everything up.
> sudo postmap /etc/postfix/virtual
> sudo systemctl restart postfix

Remove Cloud Init

Our server won’t be using cloud-init.  To keep from causing you trouble in the future we will remove it now.

Type:
> sudo apt remove cloud-init cloud-initramfs-copymods cloud-initramfs-dyn-netconf

Final Settings

You may want to enable the linux firewall.
Set your timezone in /etc/php.ini

Conclusion

That’s it for the basic server setup. This is an example of a standard linux server setup. Be sure to use setup or webmin to set which services you want to start at boot time. See the other pages for info on configuring servers for virtual webhosting or virtual email hosting. Remember to configure the firewall on the server.

TIP: Working with Disk Images Under Linux

Introduction

At times I need to work with disk images under linux.  Now I’m not talking about an image file that is a single partition. I’m talking about a disk image that represents a hard drive with an MBR and multiple partition in it.

Example 1:  Mounting A Disk Image

We need to start by mounting the image file to a loop back device.
> losetup /dev/loop0 yourimage.imgReplace yourimage.img with the path to your image file. Now lets use fdisk to

see the partions on the disk image.
> fdisk -ul /dev/loop0

Fdisk showed that my image file had 2 partitions. One was a linux partition and the other was a linux lvm partition. Let’s mount the linux partition.
> mkdir /mnt/diskimg_p1
> lomount -diskimage /dev/loop0 -partition 1 /mnt/diskimg_p1

This partition turned out to be /boot. You can unmount it just as easy.
> umount /mnt/diskimg_p1

Now unmount the disk image
> losetup -d /dev/loop0

Example 2:  Mount A Disk Image Partition Using Offsets

Here’s another way to mount the partitions. We will use losetup with offsets. So start with the following:
> losetup /dev/loop0 yourimage.img
> fdisk -ul /dev/loop0

Which gives:

Disk /dev/loop0: 4294 MB, 4294967296 bytes
 255 heads, 63 sectors/track, 522 cylinders, total 8388608 sectors
 Units = sectors of 1 * 512 = 512 bytes

Device Boot Start End Blocks Id System
 /dev/loop0p1 * 63 208844 104391 83 Linux
 /dev/loop0p2 208845 8385929 4088542+ 8e Linux

Now the important part is the start blocks. Your start blocks will look different. My starts are 63 and 208845. The sector size is 512. So do 63*512 and 208845*512. That will give you the offset numbers to use in loset. For my setup I do the following.
> losetup -o 32256 /dev/loop1 /dev/loop0
> losetup -o 106928640 /dev/loop2 /dev/loop0
> mkdir /mnt/p1
> mkdir /mnt/p2
> mount /dev/loop1 /mnt/p1
> mount /dev/loop2 /mnt/p2

Now you have the partitions mounted.

Now lets unmount the partitions and cleanup.
> umount /mnt/p1
> umount /mnt/p2
> rmdir /mnt/p1
> rmdir /mnt/p2
> losetup -d /dev/loop1
> losetup -d /dev/loop2
> losetup -d /dev/loop0

Conclusion

Hopefully this tip will help you master the disk image.