Tech Chorus

Run Your Own OpenVPN Server

written by Sudheer Satyanarayana on 2019-06-30

Introduction

The article explains how to run your own OpenVPN server. We will setup one Certificate Authority Server and an OpenVPN server. We will also generate certificates for the clients. We will also learn how to manage revocation of client certificates using the Ansible roles.

Use the Ansible roles gavika.openvpn and gavika.easy_rsa to install and configure your OpenVPN server.

You can install the OpenVPN server on any public cloud or hosting provider or on-premise servers. The Ansible roles are designed to install the OpenVPN server and a Certificate Authority server.

At the moment these Ansible roles support Ubuntu 18.04 and CentOS 7.

System Architecture And Requirements

In order to run your OpenVPN server via these Ansible roles, you will need three machines:

  1. Controller machine. This is the machine from which you execute the Ansible playbooks. This could be your laptop or a machine in the cloud. You will designate a directory on this machine as a temporary pool of files.
  2. Certificate Authority server. You will create your own CA machine that signs the certificate requests. You will need SSH access to this machine from the controller machine. You will only need to turn this server on when required. It is recommended to shut down the CA server when not in use to improve the security. Also, saves cost.
  3. OpenVPN server. You will create your OpenVPN server on this machine. You will need SSH access to this machine from the controller machine. You will also need to ensure that UDP port 1194 is open on this machine. The Ansible playbook takes care of enabling the port on the machine itself. You are responsible to open the ports on the network firewall(such as AWS Security Groups, on-premise hardware or software firewall). You will have adjust your network firewalls too in case you change the defaults in Ansible playbook or inventory.

In addition to SSH access, the servers require a user with administrative privileges via sudo. Typically, cloud images of servers provide such user accounts on the server. On AWS, for the Ubuntu images, the user is typically called ubuntu. On CentOS the user is typically called centos. If you do not have such a username, create one. There's an Ansible role to create administrative user accounts too.

Once you have provisioned the servers, proceed to create the Ansible playbooks.

Installing The Ansible Roles

Our roles require Ansible 2.8 or higher. Ensure that the required version of Ansible is installed. If not, follow the instructions to install Ansible.

Create a directory to store the playbooks and inventory.

mkdir my-openvpn-server-orchestration
cd my-openvpn-server-orchestration

I create a directory called my-openvpn-server-orchestration. You can name it whatever you want.

Next step is to install the Ansible roles from Ansible Galaxy.

ansible-galaxy install gavika.easy_rsa
ansible-galaxy install gavika.openvpn

If your target OS is CentOS, install the centos_base role too:

ansible-galaxy install bngsudheer.centos_base

Preparing Ansible Inventory

Create the file inventory.yml and add the following contents:

all:
  hosts:
    placeholder
  children:
    ca_server:
      hosts:
        dev-ca-01.example.com:
          ansible_become: true
          ansible_user: ubuntu
          ansible_host: 192.168.1.10
          easy_rsa_ca_server_mode: true
          ansible_python_interpreter: /usr/bin/python3
    openvpn_server:
      hosts:
        dev-vpn-01.example.com:
          ansible_python_interpreter: /usr/bin/python3
          ansible_become: true
          ansible_user: ubuntu
          ansible_host: 192.168.1.11
          openvpn_server_ip_address: 192.168.1.11

I prefer to use YAML formatted Ansible inventory file. Your mileage may vary. If you are using INI format for your inventory file, make sure to port the format as required.

dev-ca-01.example.com is our CA server and dev-vpn-01.example.com. We are specifying the IP addresses of these hosts, in case the DNS is not setup yet. If the DNS resolves to the correct IP addresses, you can remove the ansible_host key. Specifying ansible_host is especially useful in test environments where there is no proper DNS system.

In this example we are using Ubuntu 18.04 for both the CA and OpenVPN servers. Ansible connects to these servers with the username ubuntu. We also tell Ansible to use the Python interpreter from the location usr/bin/python3. If Python 2 is installed on the servers, you don't have to mention the interpreter path. We also mention in our inventory that Ansible should use sudo via ansible_become.

If your OS has another administratnor user, adjust the value of ansible_user. If the target host has Python 2 installed, remove the key ansible_python_interpreter from your inventory.

Notice that the IP address of the OpenVPN server is mentioned in both ansible_host and openvpn_server_ip_address. ansible_host is used to connect to the server via SSH by Ansible. openvpn_server_ip_address is used to generate the client certificate.

Preparing The OpenVPN Server

Create the file openvpn-server.yml with the following contents if your target host is Ubuntu 18.04:

---
- hosts: openvpn_server
  vars:
    openvpn_client_users:
      - janedoe
      - johndoe
    openvpn_generated_configurations_local_pool: true
    easy_rsa_req_country: "IN"
    easy_rsa_req_province: "KA"
    easy_rsa_req_city: "Bangalore"
    easy_rsa_req_org: "My Organization"
    easy_rsa_req_email: "admin@example.com"
    easy_rsa_req_ou: "My Organization Unit"
    easy_rsa_local_pool_directory: /tmp/ca_openvpn_pool_example
  roles:
    - role: gavika.easy_rsa
    - role: gavika.openvpn

If your target host is CentOS, ensure EPEL is enabled. Edit your openvpn-server.yml like below

---
- hosts: openvpn_server
  vars:
    centos_base_enable_epel: true
    openvpn_client_users:
      - janedoe
      - johndoe
    openvpn_generated_configurations_local_pool: true
    easy_rsa_req_country: "IN"
    easy_rsa_req_province: "KA"
    easy_rsa_req_city: "Bangalore"
    easy_rsa_req_org: "My Organization"
    easy_rsa_req_email: "admin@example.com"
    easy_rsa_req_ou: "My Organization Unit"
    easy_rsa_local_pool_directory: /tmp/ca_openvpn_pool_example
  roles:
    - role: bngsudheer.centos_base
    - role: gavika.easy_rsa
    - role: gavika.openvpn

We are specifying that we want to create two client users janedoe and johndoe. We also specify the variables for the EasyRSA Public Key Infrastructure. On the OpenVPN server, we will also setup PKI but not in the CA mode. We use the PKI on this server to generate certificate requests and to store the client configurations. Certificate signing is done on the CA server.

Setting openvpn_generated_configurations_local_pool to true causes the generated client configurations to be copied to the local pool. We also ensure that easy_rsa_local_pool_directory is set to same value as in our ca-server.yml playbook.

In this playbook, we are executing two roles. gavika.easy_rsa to setup PKI and gavika.openvpn to setup OpenVPN server.

Run the playbook:

ansible-playbook -i inventory.yml openvpn-server.yml --private-key /path/to/my/private/key

At this point, you should see the file server.req in the path /tmp/ca_openvpn_pool_example/server/ in the local pool. You should also see janedoe.req and johndoe.req in /tmp/ca_openvpn_pool_example/client/ in the local pool.

Preparing The CA Server

Create the file: ca-server.yml

---
- hosts: ca_server
  vars:
    easy_rsa_req_country: "IN"
    easy_rsa_req_province: "KA"
    easy_rsa_req_city: "Bangalore"
    easy_rsa_req_org: "Example"
    easy_rsa_req_email: "admin@example.com"
    easy_rsa_req_ou: "Example"
    easy_rsa_local_pool_directory: /tmp/ca_openvpn_pool_example
    easy_rsa_ca_server_mode: true
  roles:
    - role: gavika.easy_rsa

We want to run the playbook on the hosts group: ca_server. This is exactly what we have in our inventory. The vars section has a series of variables used in certificates. Adjust them to your liking. Some files have to be transferred between the CA server and the OpenVPN server. For this purpose, we use a directory on the controller machine(the machine on which you execute the Ansible playbooks, probably your laptop or a bastion host or a management host). In our example we use /tmp/ca_openvpn_pool_example as the pool. You are free to choose a different directory.

Setting easy_rsa_ca_server_mode to true ensures we want to make this server a Certificate Authority.

Just like we did for OpenVPN playbook, adjust the CA playbook for CentOS 7:

---
- hosts: ca_server
  vars:
    centos_base_enable_epel: true
    easy_rsa_req_country: "IN"
    easy_rsa_req_province: "KA"
    easy_rsa_req_city: "Bangalore"
    easy_rsa_req_org: "Example"
    easy_rsa_req_email: "admin@example.com"
    easy_rsa_req_ou: "Example"
    easy_rsa_local_pool_directory: /tmp/ca_openvpn_pool_example
    easy_rsa_ca_server_mode: true
  roles:
    - role: bngsudheer.centos_base
    - role: gavika.easy_rsa

Execute the playbook:

ansible-playbook -i inventory.yml ca-server.yml --private-key /path/to/my/private/key

/path/to/my/private/key is your SSH private key used to connect to the CA server.

If the playbook ran successfully, your CA server is setup. At this point you should see the file ca.crt in /tmp/ca_openvpn_pool_example/.

The certificate signing request for the server - server.req will be uploaded to the CA server. The CA server imports the request and signs it. The signed certificate will be copied to the local pool. You should be able to see server.crt in /tmp/ca_openvpn_pool_example/issued/server/ local pool.

Execute the openvpn-server.yml playbook again:

ansible-playbook -i inventory.yml openvpn-server.yml --private-key /path/to/my/private/key

This time, openvpn service will be started. The playbook execution will also copy the generated client configuration files in /tmp/ca_openvpn_pool_example/generated/.

Connect To The OpenVPN Server

The gavika.openvpn role generates three files for each user.

Install the openvpn package on the client machine:

Fedora:

sudo dnf install openvpn

Ubuntu:

sudo apt install openvpn

Example command to connect to the OpennVPN server on a Fedora client:

 sudo openvpn --config /tmp/ca_openvpn_pool_example/generated/janedoe/janedoe-el.ovpn

Example command to connect to the OpennVPN server on an Ubuntu client:

 sudo openvpn --config /tmp/ca_openvpn_pool_example/generated/janedoe/janedoe.ovpn

If you see a message like:

Tue Jul  2 00:34:37 2019 Initialization Sequence Completed

then you have connected successfully. Try browsing the Internet from your browser. Or just check your Internet routed IP address from the command line:

curl http://api.ipify.org

The output should show your OpenVPN server's IP address.

Revoking Certificates

If you want to revoke access to a client, edit your ca-server.yml playbook and include the list of clients to be revoked:

---
- hosts: ca_server
  vars:
    ...
    easy_rsa_revoke_clients:
      - janedoe
  roles:
    - role: gavika.easy_rsa

In this example, we are revoking the certificate for the client janedoe. Next step is to run the CA playbook:

ansible-playbook -i inventory.yml ca-server.yml --private-key /path/to/my/private/key

When the playbook finishes executing, you should see the file crl.pem in /tmp/ca_openvpn_pool_example/crl/ directory of the local pool.

Next, we run the OpenVPN playbook to update the Certificate Revocation List:

ansible-playbook -i inventory.yml openvpn-server.yml --private-key /path/to/my/private/key

After the playbook executes successfully, the client janedoe won't be able to connect to the OpenVPN server.

Routing

You can configure your OpenVPN server to:

  1. route all traffic via the OpenVPN server
  2. route traffic via OpenVPN server to specific IP addresses or networks.

If you want to route traffic to specific networks, change the Ansible variables like below:

openvpn_route_all_traffic: false
openvpn_additional_configs:
   - push: "topology subnet"
   - push: "route 192.168.4.5 255.255.255.255"
   - push: "route 192.168.4.6 255.255.255.255"

Setting openvpn_route_all_traffic to false removes the redirect-gateway field and that def1 and bypass-dhcp flags in the OpenVPN server configuration. openvpn_additional_configs allows you to write additional OpenVPN server configuration. In our example, we set two such additional configuration lines. Each push line ensures that the client uses the OpenVPN connection to reach out to the corresponding IP address. In this example, when the client tries to reach the IP addresses 192.168.4.5 or 192.168.4.6, it uses the OpenVPN connection.

Creating Administrative Linux User Accounts: gavika.administrators

written by Sudheer Satyanarayana on 2019-06-10

We are pleased to announce gavika.administrators.

The Ansible role provides a declarative method to create Linux user accounts with administrative privileges. In other words, the these users have sudo access without password and are empowered to run all commands on the system.

You might be wondering, why you would need a role when you can write a couple tasks yourselves in an Ansible playbook. The reason is, Do Not Repeat Yourself(DRY ). Instead of writing such playbook tasks over and over, use the abstraction provided by the role. You just have to write some YAML declaration and be done with it. Moreover, the maintenance is outsourced to an Apache licensed open source software. The role has Molecule tests to boost your confidence.

Here's an example:

  - hosts: servers  
    vars:
      - administrators_names: ['admin01', 'admin02']
      - administrators_keys:
          - username: admin01
            key: /path/to/id_rsa_pub_admin01
    roles:
       - role: gavika.administrators

This playbook will create the users admin01 and admin02. After creating the users, sudoers configuraion is added to empower these users to run any command with sudo and without password. In addition the public key from the file /path/to/id_rsa_pub_admin01 is added to autorized_keys file of admin01 .

How To Determine Your Public IP Address Programmatically

written by Sudheer Satyanarayana on 2019-03-30

Short answer: use ipify

ipify provides a simple public address API.

Using the tool, you can determine your public IP address programmatically. If you are using shell:

curl 'https://api.ipify.org'

Using it in a shell script:

my_ip=$(curl 'https://api.ipify.org' -s)
echo $my_ip

Using the Ansible ipify module:

- hosts: localhost
  vars:
  tasks:
    - name: Get my public IP
      ipify_facts:
        timeout: 20
      delegate_to: localhost
      register: public_ip
    - name: output
      debug: msg="{{ ipify_public_ip }}"

Sample output of Ansible playbook execution:

ansible-playbook ipify.yml 
 [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'


PLAY [localhost] **************************************************************************************************************************************************************************************************

TASK [Gathering Facts] ********************************************************************************************************************************************************************************************
ok: [localhost]

TASK [Get my public IP] *******************************************************************************************************************************************************************************************
ok: [localhost -> localhost]

TASK [output] *****************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": "49.206.13.205"
}

PLAY RECAP ********************************************************************************************************************************************************************************************************
localhost                  : ok=3    changed=0    unreachable=0    failed=0

Gavika Ansible Roles

written by Sudheer Satyanarayana on 2019-03-27

Yesterday, we announced the launch of Ansible role to install and configure AWS CloudWatch Agent.

You might have seen my other open source Ansible roles on Ansible Galaxy and Github.

In the same spirit, the company, Gavika Information Technologies Pvt. Ltd. Bangalore, has started publishing open source projects on Github. Ansible role to install and configure AWS CloudWatch Agent is the first project. Expect more projects in future.

These are some of the guidelines for the Ansible role projects that Gavika follows.

Installing AWS CloudWatchAgent On EC2 Instance Via Ansible

written by Sudheer Satyanarayana on 2019-03-26

Install the Ansible role gavika.aws_cloudwatchagent via Galaxy.

ansible-galaxy install gavika.aws_cloudwatchagent

Create The Playbook File - cw-play.yml :

---
- hosts: all
  become: true
  vars:
  roles:
    - role: gavika.aws_cloudwatchagent

Prepare the AWS CloudWatch Agent configuration - aws-cw-config.json:

{
    "metrics": {
        "namespace": "gavika",
        "metrics_collected": {
            "cpu": {
                "measurement": [
                    "cpu_usage_idle",
                    "cpu_usage_iowait",
                    "cpu_usage_user",
                    "cpu_usage_system"
                ],
                "metrics_collection_interval": 360,
                "resources": [
                    "*"
                ],
                "totalcpu": false
            },
            "disk": {
                "measurement": [
                    "used_percent",
                    "inodes_free"
                ],
                "metrics_collection_interval": 360,
                "resources": [
                    "*"
                ]
            },
            "diskio": {
                "measurement": [
                    "io_time"
                ],
                "metrics_collection_interval": 360,
                "resources": [
                    "*"
                ]
            },
            "mem": {
                "measurement": [
                    "mem_used_percent"
                ],
                "metrics_collection_interval": 360
            },
            "swap": {
                "measurement": [
                    "swap_used_percent"
                ],
                "metrics_collection_interval": 360
            }
        }
    }
}

In this example, I am using the namespace, gavika. Feel free to change it. We collect the cpu, disk, diskio, mem and swap metrics. The agent will send these metrics once in 360 seconds.

Run The Playbook (CentOS):

ansible-playbook -i centos@myserver.example.com, cw-play.yml

The target machine is a CentOS server. Hence you see centos username. I am passing the server name inline. In a production system, you might have a well-defined inventory file. Change the command to suit your needs.

Run The Playbook (Ubuntu)

ansible-playbook -i ubuntu@myserver.example.com, cw-play.yml -e ansible_python_interpreter=/usr/bin/python3 -vvv

The Ubuntu server has Python3 by default and Ansible expects Python2 by default. Therefore, I pass the ansible_python_interpreter extra variable from the command line.

After the playbook executes successfully, you should see the metrics in AWS CloudWatch under the namespace specified.

Fedora 29 And Overheating Lenovo W540 Laptop

written by Sudheer Satyanarayana on 2019-03-06

I installed Fedora 29 on a Lenono W540 laptop. The laptop started overheating within a few minutes of booting up.

The output of lspci:

0:00.0 Host bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor DRAM Controller (rev 06)
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06)
00:02.0 VGA compatible controller: Intel Corporation 4th Gen Core Processor Integrated Graphics Controller (rev 06)
00:03.0 Audio device: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller (rev 06)
00:14.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI (rev 04)
00:16.0 Communication controller: Intel Corporation 8 Series/C220 Series Chipset Family MEI Controller #1 (rev 04)
00:16.3 Serial controller: Intel Corporation 8 Series/C220 Series Chipset Family KT Controller (rev 04)
00:19.0 Ethernet controller: Intel Corporation Ethernet Connection I217-LM (rev 04)
00:1a.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #2 (rev 04)
00:1b.0 Audio device: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller (rev 04)
00:1c.0 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 (rev d4)
00:1c.1 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #2 (rev d4)
00:1c.2 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #3 (rev d4)
00:1c.4 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #5 (rev d4)
00:1d.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 (rev 04)
00:1f.0 ISA bridge: Intel Corporation QM87 Express LPC Controller (rev 04)
00:1f.2 SATA controller: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] (rev 04)
00:1f.3 SMBus: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller (rev 04)
01:00.0 VGA compatible controller: NVIDIA Corporation GK106GLM [Quadro K2100M] (rev a1)
02:00.0 SD Host controller: O2 Micro, Inc. SD/MMC Card Reader Controller (rev 01)
03:00.0 Network controller: Intel Corporation Wireless 7260 (rev 83)

The laptop has an NVIDIA Quadro K2100M graphics card.

The solution was to install the NVIDIA proprietary driver via RPMFusion.

First, enable RPMFusion.

sudo dnf install https://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm https://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm

Then, install the NVIDIA drivers.

dnf install xorg-x11-drv-nvidia akmod-nvidia
dnf install xorg-x11-drv-nvidia-cuda 
dnf update -y

Reboot and wait for five minutes till the kernel module is built.

Simple Password Vault With Ansible

written by Sudheer Satyanarayana on 2019-02-17

Ansible comes with a vault feature. It is meant to be used in the context of configuration management. But you can also use it as a standalone simple password vault for your personal or organization's use.

Initial setup of password vault:

At this point, your password vault is setup and pushed to your central Git repository. The next time you want to use it on another or same device, here's the flow you could use:

Keep in mind that Git will treat your password vault as a binary file.

Installing Specific Version Of Ruby On CentOS

written by Sudheer S on 2019-01-08

How do you install a specific version of Ruby on CentOS 7?

The answer: via SCL.

The Ansible role bngsudheer.ruby makes it even more easier.

Create the file ruby_playbook.yml and insert the following contents:

- hosts: my_ruby_servers
  vars:
    ruby_version: 2.3
  roles:
     - bngsudheer.ruby

Once done, run the Ansible playbook:

ansible-playbook ruby_playbook.yml

What if I want to install Ruby 2.4 on CentOS?

Just set ruby_version to 2.4. Similarly, if you want to install Ruby 2.5, set ruby_version to 2.5.

It is also possible to enable particular version of Ruby on a per-user basis. For example, to enable Ruby 2.5 for the user myproject user, set ruby_enable_users to ['myproject'].

Systemd Unit File Just To Create A Directory Under /run

written by Sudheer Satyanarayana on 2018-01-19

On recent versions of Fedora, CentOS and RHEL, /var/run and /var/lock are mounted as /tmpfs. Also, /var/run is a symlink to /run. Because of this, if you create a directory under /run it won't stay there after a reboot.

If you are starting to migrate your service to systemd or just want to create a directory under /run using systemd, use this simple unit file. The contents of:

/etc/systemd/system/my.service
[Unit]
Description=My Service

[Service]
# We just want to create the myservice run directory
Type=oneshot
RuntimeDirectory=myservice
User=myuser
Group=myuser
ExecStart=/bin/true
RemainAfterExit=yes
[Install]
WantedBy=multi-user.target

Replace my.service with an appropriate name for your service. Replace the description, User and Group values to your needs.

Start the unit:

systemctl start my.service

Enable the unit so that it starts when the system boots:

systemctl enable my.service

What is wrong in using ExecStartPre=-/bin/mkdir /var/run/myservice?

Systemd can handle directory creation and setting up its attributes for you. You just specifiy the name of the directory in the RuntimeDirectory= directory. When the unit starts, systemd creates the directory and sets its owner and group to the values you specify in User= and Group= directives. So, the counter question is, when systemd can do it for you using a simple directive, why do you want to do it yourself? Also, note that the - prefix just ignores the error when the directory already exists and another attempt is made to create it. Although the error is ignored, it still shows up in red when you check the status of the service.

What is Type=oneshot?

Type=oneshot means the process is expected to exit before systemd starts follow-up units. Along with this, we also specify RemainAfterExit=yes. We tell systemd that my.service shall be considered active even when all its processes exited. When you query the status of the service, you will see that the unit is active because of this directive.

What is ExecStart=/bin/true?

Typically, you would put the path to your service executable in ExecStart= directive. In our example, we're not doing that yet. We're simply using the sytemd unit file to create the /run/myservice directory. So, we use the placeholder command /bin/true. /bin/true simply returns 0 indicating that the command was executed successfully.

References


journalctl Cheatsheet

written by Sudheer Satyanarayana on 2017-10-26

Between date ranges

journalctl --since "2017-10-20" --until "2017-10-21"

Between date and time ranges

journalctl --since "2017-10-20 10:00:00" --until "2017-10-21 11:00:00"

Friendlier unit values of since and until

journalctl --since "2017-10-20 10:00:00" --until "1 hour ago"
journalctl  --since "1 hour ago"
journalctl  --since "2 hour ago"  --until "1 hour ago"

View particular unit

journalctl -u alertmanager

where alertnamaner is the systemd unit name.

Kernel messages

journalctl -k

By priority

journalctl -p crit

Continuously watch like tail -f

journalctl -fu prometheus

Catalogue and jumping to end in the pager

journalctl -xe

This is particularly useful when your last command results in an error and you want to the most recent errors. The -x flag adds useful descriptive messages to errors when they are available.