Using Bots to Solve Developer Drudgery

Scaling an engineering organization has a lot of moving parts. One of the parts we discuss rarely is reducing friction and reducing time spent on menial tasks to the absolute minimum.

Figuring out “What is hard?” and “Should it be this hard or time-consuming?” is critical to the success of your organization.

In this post, I’ll discuss flows we automated with bots.

Pull request review

We have an internal library that many our backend services share. That internal library has JSON files that are swagger definitions of services.

Let’s take the NLP service and the analysis service. NLP Service wants to call the analysis service and report the results. To do that, we have an internal library that generates an “SDK” based on a JSON swagger definition. That definition often needs to update when the analysis service changes its API.

Our engineers create a PR against that library with the updated JSON. However, there is little value in reviewing it since it’s been generated by the swagger output. It is the definition of the service, and there are no manual edits.

While working on services, engineers were often held back by that review and could not continue their work without coming up with creative solutions.

This made the problem even worse when the engineer is in another location and another timezone.

Mergebot

Introducing mergebot. Mergebot watches for PRs on repositories; Once a PR is posted, it looks for a configuration file that looks like this:

{
  "automerge": false,
  "paths": [
    "swagger/(.*).JSON"
  ],
  "changed_files": 1
}

• automerge -> Should I automatically merge the PR or just approve it?
• paths -> list of paths that the bot can approve
• changed_files -> How many changed files can be in the PR.

If an engineer changes the file under the swagger directory and it’s only a single file, the bot will automatically approve the PR.

This automation saved quite a lot of time for our engineers, reduced frustration and allowed us to move faster.

Mergebot acts on every repository that has this configuration; You can configure users, merging, deleting branches and others.

Here’s what it looks like

Internal dependency management

We use microservices; Microservices often share libraries and those libraries are often internally developed.

nodule-config for example; it is used in all of our GraphQL based JS services.

We also like to work with the latest version of every library as described in How we keep dependencies fresh.

Knowing those facts, you can assume that when we release a new version of a library, we want to auto bump it on every repository that uses it.

Commitbot

Commitbot is a bot that searches github and bumps dependency of any library (supports both Python and Javascript).

Once you release a new version you call:

globality-build dependency-update \
    --package-name alethia \
    --version $version \
    --bump-type EQUALS \
    --language python

Commitbot crawls our github, searches for alethia in every python dependency (setup.py) and bump it to $version with the bump type of EQUALS.

Here’s what this looks like:

Productivity improvement galore

Mergebot and Commitbot work together. Mergebot automatically merges PRs by commitbot (if configured).

When you release a new library, you assume it is automatically bumped in all services using it. If you think of the entire flow when our engineers change a client configuration, they merge it and they’re done.

What used to be 3-4 steps with wait times between them is a single step with no wait and automation nicely wrapped around it.