How to set up a WordPress cluster using Docker and Google Cloud Platform


Setting up a scalable and highly available WordPress cluster using Docker and Google Cloud Platform can be quite some work. But with this guide, it will be easier!

Introduction

In this guide, we will see how to create and deploy a scalable WordPress cluster hosted on the Google Cloud Platform. The cluster is made of:

  • A Cloud SQL database containing the WordPress data.
  • A Cloud Storage bucket containing the media content uploaded from WordPress – it is shared by all the worker instances.
  • An auto-scaling instance group which contains all the worker instances.
  • A HTTP Load Balancer which handles all the requests and forwards the traffic to the worker instances.

 

wordpress-cluster

Docker image

In order to ease the development and deployment of the WordPress site we are running it inside a Docker container. All the site content is packaged in a custom WordPress image (loosely based on the official one), automatically built and hosted in a private Docker Hub repository.

Our custom image contains a typical Apache/PHP/WordPress stack, adds the site content and declares a volume mounting point on the WordPress media upload directory: /var/www/html/wp-content/uploads.

WordPress media upload folder

In a single node configuration, the media files uploaded to WordPress are stored directly on the server filesystem. In a clustered configuration with multiple servers, this cannot work anymore, as these files must be accessible by all instances.

Unfortunately, WordPress does not provide any native solution to this problem, so we had to find a way to have all the instances media repository synchronized. Several blog articles cover the deployment of a WordPress cluster with different solutions to this problem:

In our case the site is deployed on the Google Cloud Platform which allows a more elegant way to achieve this: creating a mount point on the local file system to a shared Cloud Storage bucket with Cloud Storage FUSE. As per the documentation “Cloud Storage FUSE provides a tool for applications to upload and download Google Cloud Storage objects using standard file system semantics”. The mounting point content is automatically synchronized between all the instances without having to setup a cron periodic synchronization which implies replication delays or using another third party products like GlusterFS.

Warning : Cloud Storage FUSE is currently in Beta and has no guarantee of long-term maintenance.

Here are the required steps to achieve media synchronization between all the WordPress instances:

  1. Create a new Cloud Storage bucket.
  2. Create a new directory on the worker’s file system.
  3. Install and configure Cloud Storage Fuse to use this directory as a mounting point for the bucket.
  4. Mount this directory as a volume when running WordPress in a Docker container.

Info : The use of Cloud Storage Fuse requires to activate full access to the Storage API in the worker instance (or instance template) Access scopes configuration. For security reason, API access authorizations can’t be modified after the creation of a compute instance and must be set at creation

First we have to create a new bucket named wp-uploads-bucket.

We then have to create a new directory that will be used as the bucket mounting point:

sudo mkdir /mnt/wp-uploads
sudo chmod a+w /mnt/wp-uploads

The installation of Cloud Storage Fuse is simple and well described in the documentation.

As we will create the mounting before running the Docker container, and access it from inside the container, we need to authorize access for all users. To do so we first need to allow the allow_other option to be used in the mount command: open the /etc/fuse.conf file and uncomment the line containing user_allow_other.

The mounting of the bucket can then be performed with the following command:

gcsfuse --dir-mode "777" -o allow_other wp-uploads-bucket /mnt/wp-uploads

The final step is to use this directory as volume mounting point when running a container from our image:

docker run [...] -v /mnt/wp-uploads:/var/www/html/wp-content/uploads -p 80:80 sfeir/custom-wordpress

Cloud SQL proxy

WordPress data are stored in a Cloud SQL database, which is a fully managed and cloud hosted MySQL instance. Access control on Cloud SQL database is achieved by exhaustively whitelisting all the IP addresses that are allowed to access the base. As we are working with a dynamic number of worker instances which are automatically created depending on the site traffic, we don’t know in advance the number of instances nor their IP address.

Fortunately, the GCP provides a tool to cover this use case: the Cloud SQL Proxy. Once installed and configured on a GCE instance, the proxy will handle the connection to the Cloud SQL database with the provided credentials using the Cloud SQL API which does not require IP whitelisting.

Info : To be able to use the Cloud SQL Proxy you must enable the Cloud SQL API when creating your GCE instance or instance template.

The Cloud SQL Proxy can be configured in many ways depending on which credentials you want to use and how to provide them, the explicit specification of the Cloud SQL database or the automatic instance discovery… Please refer to the documentation to select the appropriate configuration for your project.

The simplest configuration is Cloud SDK authentication and automatic instance discovery, which will use the credentials of the service account associated with the GCE instance and create a socket for all the Cloud SQL databases of the project.

For example, here are the commands allowing you to use the proxy and configure it to create Unix sockets instead of TCP sockets.

Create a new directory that will contain the socket files:

sudo mkdir /cloudsql
sudo chmod 777 /cloudsql

Start the proxy using Cloud SDK authentication and automatic instance discovery (this has to be done every time the GCE instance is started):

./cloud_sql_proxy -dir=/cloudsql &

Once the connection to the database is performed and the Unix socket is created, the WordPress container can be run with the appropriate configuration:

docker run [...] \
-v /cloudsql:/cloudsql \
-e "WORDPRESS_DB_HOST=localhost:/cloudsql/SOCKET_TO_USE" \
-e "WORDPRESS_DB_USER=USER" \
-e "WORDPRESS_DB_PASSWORD=PASSWORD” \
-e "WORDPRESS_DB_NAME=DATABASE_NAME" \
-e "WORDPRESS_TABLE_PREFIX=prefix_” \
 -p 80:80 sfeir/custom-wordpress

Instance image and startup scripts

The initial configuration of the worker instance (installation of Cloud Storage FUSE / Cloud SQL Proxy and the creation of the mounting point directories) can be performed only once and then committed to a new image containing all the static system configuration. An instance template can then be created, based on this image, and will be used when creating the worker instances.

The startup scripts are executed every time a new worker instance is created. They perform the actions required to initialize the instance and run a new docker container with our WordPress. Some actions need to be executed as root and some others using an account created to access Docker.

These scripts can be hosted on Cloud Storage, on a dedicated bucket.

Here is an example of a startup script executed as root when a worker instance is started:

# !/bin/bash
#
# Initialize Cloud SQL Proxy
mkdir /cloudsql
chmod 777 /cloudsql
cloud_sql_proxy -dir=/cloudsql &
#
# Retrieve the next startup script and execute it as Docker user
gsutil cp gs://CONFIG_BUCKET/startup-docker.sh startup-docker.sh
chmod +x startup-docker.sh
sudo -u DOCKER_USER ./startup-docker.sh%

And an example of the next startup script, executed as Docker user:

# !/bin/bash
#
# Perform Cloud Storage FUSE mounting
gcsfuse --dir-mode "777" -o allow_other wp-uploads-bucket /mnt/wp-uploads
#
# Pull our custom WordPress image
gsutil cp gs://CONFIG_BUCKET/docker-config/.dockercfg ~/.dockercfg
docker pull sfeir/custom-wordpress
#
# Create a new container to run WordPress
 docker run -d \
-v /mnt/wp-uploads:/var/www/html/wp-content/uploads \
-v /cloudsql:/cloudsql \
-e "WORDPRESS_DB_HOST=localhost:/cloudsql/SOCKET_TO_USE" \
-e "WORDPRESS_DB_USER=USER" \
-e "WORDPRESS_DB_PASSWORD=PASSWORD” \
-e "WORDPRESS_DB_NAME=DATABASE_NAME" \
-e "WORDPRESS_TABLE_PREFIX=prefix_” \
-p 80:80 \
sfeir/monimage-wordpress:latest

Instance group and load balancer

The final step is to set up an instance group for the workers and to configure the load balancer. The instance group is based on the instance template and handles automatically the scaling of the worker pool depending on the instances load. An instance group can be configured to create new instances in one zone/datacenter or in a region (multiple datacenters) to ensure higher availability.

The worker instances do not have a public IP address and thus cannot be accessed directly from the Internet.

The load balancer is the only system in the cluster having a public IP address which shall be the one registered in the DNS. It handles the incoming requests and distributes the traffic evenly across the worker instances of the cluster. The load balancer is a global system on the GCP, it is not associated to a datacenter or a region.

Both the instance group and the load balancer use a configurable health check that reports the status of an instance. Depending on the load of the workers, the instance group will determine if new instances must be created or if some instances can be stopped. The load balancer uses the load report to distribute the requests evenly between the workers.

Conclusion

To this day, WordPress does not provide any straightforward solution to run it in a clustered mode. Fortunately the Google Cloud Platform is a powerful infrastructure and offers many tools we can use to set up the desired cluster. The hosting of the WordPress data and media content can be performed using Cloud SQL and Cloud Storage that are fully managed GCP products, which means that we don’t have to care about their availability or scalability.

The use of a load balancer backed by an instance group allows us to dynamically size the worker pool according to the number of visitors. We also don’t have to worry of a hypothetical worker instance failure, as a new one would automatically be spawned.

All these mechanisms working together allow us to build a scalable and highly available WordPress cluster able to handle the traffic peaks and decreases with a constant level of performance while limiting the costs associated with the number of instances running.

Vous aimerez aussi...

  • Jon

    How do you SFTP to adjust the Word Press files on the server? I have everything set up correctly, but when I try to SFTP using winSCP or FileZilla it gives me errors!

    Or, are you storing all wordpress files on the google bucket? I have WP deployed using a load-balancer setup but I can’t SFTP and it’s screwing me up becuase I need to adjust the wordpress files to build the site of course!

    Thanks a lot for your help.

    • Pascal Casteran

      Hi Jon,

      We don’t need to access the running WordPress instances by SFTP to change the site content.
      In our case, the site content is managed in a Git repository from which we build the Docker image that is deployed on the cluster’s instances as described above.
      So releasing a new version of the site means building a new version of the Docker image, which embeds the site content, and publishing it to our private Docker Hub repository.

      This is how we manage the WordPress content, the Cloud Storage bucket is only for hosting the uploaded media files (stored in /var/www/html/wp-content/uploads) so that they can be accessible by all the running WordPress instances.

      If I understand correctly, you also have several WordPress instances running behind a Load Balancer. In that case, you should not try to update directly the WordPress content on these instances as you would have to do it for all of them…
      It would be better to package the site content in the Docker image as we do or you can also have everything (site content + media files) in the Cloud Storage bucket.

      Regarding the connection errors you got, are you trying to SFTP directly to the worker instances or to the Load Balancer?

      I hope this answers your question.

      Pascal

      • Jon

        Pascal,
        First off, thanks for the prompt response! Your post and reply has allowed me to gain a lot of insight into what the final setup should look like. I believe that I’m close to getting everything to work. Please allow me to show you my current thought process so that you can stop me where I’m going wrong:

        1. I created an instance template from a custom image that I created after installing LEMP and WordPress onto the base Ubuntu 16.04 image.
        2. I created an instance group using the instance template.
        3. I created a load balancer using the simple-to-use Google Cloud GUI (under the networking section)
        4. The bare-bones wordpress install is up and working at the static IP, along with the ephemeral instance IPs. Additionally, SSL works fine as intended

        That’s what I have so far.

        But, I know that’s wrong? I don’t know how I’m supposed to change the WordPress Files, and furthermore, Google Cloud doesn’t allow SSH into custom images.

        I’m a bit lost on the docker aspect of what you’re doing. Is it the same thing as a custom image template? Is it possible for me to use Google storage to host all of the word press files? Then I could adjust just the google cloud storage files and the ephemeral instances would pull from that, right? (just confirming this is what you’re saying). If so, could you please provide resources or quick steps to change the directory to point to the cloud bucket?

        The cloud SQL database seems pretty straight forward, but resources or quick tips on how to “point” wordpress as this DB instead would be very much appreciated as well.

        I know this is a lot, but I truly appreciate your knowledge and expert help!

        • Pascal Casteran

          Hi Jon,

          It is not clear to me how you manage the database in your configuration.
          As you explained you have set up an auto-scaling architecture (instance template + instance group + load balancer), the same than the one I describe, which is correct and implies that multiple compute engine (GCE) instances are running concurrently.
          However you also say that the LEMP stack is installed in the GCE instance image, so you will have one MySQL server per GCE instance. Is it intentional? If so, how do you manage database synchronization between all the instances? Same for backup?
          This is why we are using a CloudSQL database, which is an external (ie not part of the instance image) and fully managed service : we have only one database server for all the WordPress instances.

          So our GCE instances only run the Apache/PHP/WordPress stack (no database). We use a Docker image for this to avoid installing (and having to update) that stack directly to the GCE instance image. Using Docker is not required, it’s up to you to decide how you want to manage the administration of the cluster.

          The same goes for the WordPress files: you don’t want them to be part of the GCE instance image as you can’t access them and you would have to create a whole new image every time you update them. Here you have two solutions:
          1) either have them in a space accessible by all the WordPress instances (Cloud Storage bucket, NFS, …);
          2) or have them in the Docker image (this is what we chose).

          To answer your questions, yes you can use Google storage to host all of the word press files (this is solution 1) and the ephemeral instances would pull from that so the changes done on these files on the bucket will be reflected on every instance.
          To do this you will have to mount the bucket on the file system of your GCE instances using Cloud Storage FUSE.

          To use the CloudSQL database with an arbitrary number of clients you will have to set-up a CloudSQL Proxy on each GCE instance (on the image actually). You can find the related documentation here and here.

          Once installed and configured, the CloudSQL Proxy will be accessible on localhost:3306 which means that you won’t need to update the WordPress configuration as the variable DB_HOST is by default set to ‘localhost’.

          I hope it will help you.

      • Jon

        Pascal,

        I appreciate the in-depth and informative response. I understand the necessity for a « common » DB and single location for the WP files.

        Currently, I do not use docker (I read through the documentation thoroughly after your post and it makes sense, but I’m unsure of how to actually execute it– store WP files on gitlab and then pull them into a container and then call that container in the google cloud); what would a startup script look like in my case where I just want to use google cloud storage for simplicity (the #1 option in your reply)?

        I just want to make sure I mount it correctly and any configuration is taken care of (I use nginx on ubuntu 16.04).

        Thanks again for your help.

        • Pascal Casteran

          Hi Jon,

          In your case I think you should just have to perform the mounting of the bucket that will contain the WordPress files with the gcsfuse command (adapt the /path/to/mount path according to your LEMP stack set-up):

          gcsfuse YOUR_BUCKET /path/to/mount

          • Jon

            Pascal,

            Happy New Year! I have continued to learn a bit and I have some questions regarding your setup. Here is what I am planning now:

            1. store the WordPress files locally, and on gitlab so that team members can access the code and commit and changes to it

            2. create a docker image locally and install LEMP along with the WordPress files from gitlab

            3. push that « complete » docker image to the private docker hub that you referred to

            4. Create a base Ubuntu 16.04 instance on GCE and setup SSL with let’s encrypt

            5. Make a startup script to initiate the cloud SQL proxy and pull the website docker image

            6. Make a template from the created GCE instance and then deploy an instance group with the appropriate load balancing

            Is this correct? Please let me know if this is wrong.

            Thanks again!

        • Pascal Casteran

          Hi Jon. Thank you and happy new year to you too :)

          The solution you describe is fine and almost exactly matches the one we set up.
          However, if you build the Docker image locally, you even don’t need a private Docker Hub repository. You can push it to a Google Container Registry directly on your GCP project in step 3.

  • Hi Pascal, I’ve been stuck with this for a few days now. It appears SQL is working fine, but whenever I attempt to mount my bucket via gcsfuse inside my container, I get the following error:
    fusermount: failed to open /dev/fuse: Operation not permitted

    It would appear I can establish a connection with google but not mount. Have you any idea why this might be? Its currently prohibiting me from building my image (docker build -t ic/spm .).

    • Pascal Casteran

      Hi Gareth,

      I run into the same issue when trying to perform a similar mounting from inside a container. To have it working you will have to add the following options to your docker run command to authorize the mounting operation : –cap-add SYS_ADMIN –device=/dev/fuse

      See this thread for more details.

      I hope this answers your question.

      Pascal

      • That goes some way to explaining how I can get it to work initially, but I’m wondering how I might build the image to utilise on google cloud as a container. The build command does not allow me to pass the desired params.

        • Pascal Casteran

          Do you really need to perform the fuse mounting when building the image? It would be better to do it in a startup script embedded in the image and launched when a container is started.

          If you absolutely want to have it inside the image, you will have to do it in a running container then commit this container to a new image.