Automatically scaling Drone CI with Gitea

Published on 17 December 2022, updated on 30 December 2022 - ci development rust technology

Why

For a while now, I have been working on a personal project called newsletter2web, which I run on a Raspberry PI. As the title already indicates, it is written in Rust.

Since compiling the project takes a while, especially on a Raspberry PI, I decided to investigate how to make it build automatically.

That sounds like a job for continuous integration solution. But these tend to use quite a bit of resources, even when not in use. That is wasteful and I didn't want that.

So put together, I got the following requirements:

A continuous integration solution, that
Would use little resources when not in use,
Integrates with Gitea, and
Is capable of cross-compiling Rust

After a bit of digging, I found Drone CI to be a nice fit: it is fairly lightweight, and easy to get running. Making it work well enough to cross compile Rust took a while though.

This post describes the Drone and Gitea parts, the Rust part is in a followup post.

The components

In order to build everything, we need a few moving parts:

Gitea for hosting the code.
Drone CI for running the builds.
Drone autoscaler to ensure it only uses resources when necessary.
cross: a cross compilation solution for Rust.

Setting up Drone with Gitea

Setting up Drone is not that hard, and the installation instructions are clear.

Then, run it with docker using the drone/drone:2 image, and ensure that all these environment variables are passed through. Take care that the Drone API is reachable from Gitea as well.

For me, the configuration came down to this:

# Drone host configuration
DRONE_SERVER_HOST=drone-ci.kiers.eu
DRONE_LOGS_DEBUG=true
DRONE_LOGS_PRETTY=true

# RPC secret for build agents
DRONE_RPC_SECRET=<redacted>

# DB configuration
DRONE_DATABASE_DATASOURCE=postgres://drone:drone-password@db:5432/drone-db
DRONE_DATABASE_SECRET=<redacted>

# Gitea integration
DRONE_GITEA_SERVER=https://code.kiers.eu
DRONE_GITEA_CLIENT_ID=<redacted>
DRONE_GITEA_CLIENT_SECRET=<redacted>

# Enable initial admin user
# This needs to be the same username as in Gitea
DRONE_USER_CREATE=username:my-username,admin:true

# Enable Jsonnet support
DRONE_JSONNET_ENABLED=true

Setting up Drone autoscaler

During my research, I tried multiple approaches: first I used a simple Docker runner on a static build server. But I wasn't happy to incur the costs of something that is only used sporadically. Then I tried multiple different solutions, such as the VM runner, the (deprecated) Digitalocean runner and the autoscaler. In the end, the last one was what I settled on.

It works by polling the Drone server api to see whether there are builds pending and whether there is build capacity available. Then, it will scale up or down the numer of servers that are required to handle these builds. Crucially for me, it is possible to set minimum number of build servers to zero, to ensure that they will all be destroyed when there is no need for them anymore.

This does mean that one needs a place to host these build servers. Luckily, the autoscaler has support for a number of different cloud providers. I ended up using Digital Ocean.

First, a machine user with administrative credentials needs to be created. This needs to be done with the Drone CLI. Check the documentation for how to do so.

The setup is fairly easy again, and also builds on environment variables:

#!/usr/bin/bash

# Communication with the drone server
DRONE_RPC_HOST=drone-ci.kiers.eu
DRONE_RPC_PROTO=https
DRONE_AUTOSCALER_USER_TOKEN=<redacted> # Machine user token

# RPC Secret for the build agents that are installed on the build servers
DRONE_RPC_SECRET=<redacted>

# Digitalocean settings
# Get these from the DO Droplet Launch UI, in the URL bar
DRONE_DIGITALOCEAN_SIZE='s-4vcpu-8gb-amd' 
DRONE_DIGITALOCEAN_IMAGE='docker-20-04'
DRONE_DIGITALOCEAN_REGION='fra1'
DRONE_DIGITALOCEAN_TOKEN=<redacted>
DRONE_DIGITALOCEAN_SSHKEY=<redacted> # To log in to the build servers

# How often should the autoscaler check whether to scale up or down
# This then also is the time that it takes before a build server will
# be started when none is available, and therefore has a big influence
# on total build time. 
DRONE_INTERVAL=30s

# Pool settings
DRONE_POOL_MIN=0 # This allows the pool to turn off everything
DRONE_POOL_MAX=4 # The maximum number of build servers to launch

docker run -d \
  --volume=$(pwd)/data:/data \
  --env=DRONE_SERVER_PROTO=${DRONE_RPC_PROTO} \
  --env=DRONE_SERVER_HOST=${DRONE_RPC_HOST} \
  --env=DRONE_SERVER_TOKEN=${DRONE_AUTOSCALER_USER_TOKEN} \
  --env=DRONE_AGENT_TOKEN=${DRONE_RPC_SECRET} \
  --env=DRONE_POOL_MIN=${DRONE_POOL_MIN} \
  --env=DRONE_POOL_MAX=${DRONE_POOL_MAX} \
  --env=DRONE_DIGITALOCEAN_REGION=${DRONE_DIGITALOCEAN_REGION} \
  --env=DRONE_DIGITALOCEAN_TOKEN=${DRONE_DIGITALOCEAN_TOKEN} \
  --env=DRONE_DIGITALOCEAN_SIZE=${DRONE_DIGITALOCEAN_SIZE} \
  --env=DRONE_DIGITALOCEAN_SSHKEY=${DRONE_DIGITALOCEAN_SSHKEY} \
  --env=DRONE_DIGITALOCEAN_IMAGE=${DRONE_DIGITALOCEAN_IMAGE} \
  --env=DRONE_LOGS_TRACE=true \
  --env=DRONE_LOGS_DEBUG=true \
  --env=DRONE_LOGS_PRETTY=true \
  --env=DRONE_LOGS_COLOR=true \
  --env=DRONE_INTERVAL=${DRONE_INTERVAL} \
  --publish=8080:8080 \
  --restart always \
  --name drone-autoscaler \
  drone/autoscaler:latest

And that's it. Now it will scale up and down the build servers, as new builds come in.

Also, take a look at all the possible settings for the autoscaler, as there may be some options that are beneficial as well.