Creating Homebrew Formulas with GoReleaser

We chose to use GoReleaser at ObservIQ for our distro of the OpenTelemetry Collector to simplify how we build and support many operating systems and architectures. GoReleaser enables us to build targeting a matrix of GOOS and GOARCH targets as well as automate creating a wide range of deliverables. The ones we have utilized are building tarballs, nfpm packages, docker images, and Homebrew formula.

For this article, the focus is on the Homebrew Taps capabilities in GoReleaser and our journey using it. Our goal was to make it easy for users to install our software on macOS so that they could easily try it out. We went with Homebrew as it’s familiar to many macOS users and would allow a user to try out our software and remove it just as easily when they are finished.

When setting up Homebrew in GoReleaser, we found that documentation around creating a Homebrew formula, in general, was lacking. Also, it wasn’t easy to search for solutions when we encountered a problem. Homebrew provides a Formula Cookbook, but it can be a bit confusing if you aren’t already familiar with building formulas.

We went through several iterations of our Homebrew formula as we learned more and more about the correct way to do things.

First, we created a public repo that was to be our Homebrew formula. We would specify this as where GoReleaser would send formula updates. We created [https://github.com/observIQ/homebrew-observiq-otel-collector].

As we started setting up GoReleaser, we originally used the caveats, install, and plist blocks of the brews section in GoReleaser to create our formula.

Caveats Block

The caveats block can be used to relay textual information to the user after a homebrew installation. We use it to:

1. Give info on how to start/stop/restart the homebrew service that is installed
2. How to properly uninstall the entire formula
3. And give info on where certain configuration files live

Install Block

Inside the install block, you can use the same brew shortcuts for installation used in the formula file itself. This ultimately will copy these same lines to the ruby formula file’s install block. For example, we use:

• prefix.install to copy files and directories to homebrew’s versioned formula directory
• prefix.install_symlink to create symlinks in homebrew’s versioned formula directory
• etc.install to copy files and directories to homebrew’s shared etc directory
• bin.install to copy binary executable files to homebrew’s versioned formula’s “bin” directory
• lib.install to copy library files to homebrew’s versioned formula’s “lib” directory

Service Blocks

The plist block was where we defined a plist file to allow our software to be run as a launchd service. The service block wasn’t supported in GoReleaser when we started, once it was, we shifted to using that as it was easier to define for us and allowed a more brew native way of managing our service.

Our original plist block looked like the XML below:

plist: |
    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
    <plist version="1.0">
    <dict>
        <key>Label</key>
        <string>com.observiq.collector</string>
        <key>ProgramArguments</key>
        <array>
        <string>#{bin}/observiq-otel-collector</string>
        <string>--config</string>
        <string>#{prefix}/config.yaml</string>
        </array>
        <key>RunAtLoad</key>
        <true/>
        <key>StandardErrorPath</key>
        <string>/tmp/observiq-otel-collector.log</string>
        <key>StandardOutPath</key>
        <string>/tmp/observiq-otel-collector.log</string>
    </dict>
    </plist>

Once we saw that GoReleaser supported the service block we were able to simplify it to the following:

service: |
    run [opt_bin/"observiq-otel-collector", "--config", opt_prefix/"config.yaml"]
    environment_variables OIQ_OTEL_COLLECTOR_HOME: opt_prefix
    keep_alive true

We did have some trouble creating the service originally as there were some ‘magic’ words that correspond to special directories in a brew installation. The cookbook documentation used these magic words in examples but did not list them as it does in the install section. We had to search the brew source code to find a list of the support ‘magic’ words.

Here are a few of the common ones we used:

Initial Config

Here is the brews block we originally generated that created a working formula for us.

brews:
  - name: observiq-otel-collector
    tap:
      owner: observIQ
      name: homebrew-observiq-otel-collector
      branch: main
    download_strategy: CurlDownloadStrategy
    folder: Formula
    url_template: https://github.com/observIQ/observiq-otel-collector/releases/download/{{ .Tag }}/{{ .ArtifactName }}
    commit_author:
      name: observiq
      email: [email protected]
    homepage: "https://github.com/observIQ/observiq-otel-collector"
    license: "Apache 2.0"
    caveats: |
      ****************************************************************
      The configuration file that is run by the service is located at #{prefix}/config.yaml.
      If you are configuring a logreceiver the plugin directory is located at #{prefix}/plugins.
      ****************************************************************
      ****************************************************************
      Below are services commands to run to manage the collector service.
      If you wish to run the collector at boot prefix these commands with sudo
      otherwise the service will run at login.
      To start the collector service run:
        brew services start observiq/observiq-otel-collector/observiq-otel-collector
      To stop the collector service run:
        brew services stop observiq/observiq-otel-collector/observiq-otel-collector
      To restart the collector service run:
        brew services restart observiq/observiq-otel-collector/observiq-otel-collector
      ****************************************************************
      ****************************************************************
      To uninstall the collector and its dependencies run the following commands:
          brew services stop observiq/observiq-otel-collector/observiq-otel-collector
          brew uninstall observiq/observiq-otel-collector/observiq-otel-collector
          launchctl remove com.observiq.collector
          # If you moved the opentelemetry-java-contrib-jmx-metrics.jar
          sudo rm /opt/opentelemetry-java-contrib-jmx-metrics.jar
      ****************************************************************
    install: |
      bin.install "observiq-otel-collector"
      prefix.install "LICENSE", "config.yaml"
      prefix.install Dir["plugins"]
      lib.install "opentelemetry-java-contrib-jmx-metrics.jar"
    service: |
      run [opt_bin/"observiq-otel-collector", "--config", opt_prefix/"config.yaml"]
      environment_variables OIQ_OTEL_COLLECTOR_HOME: opt_prefix
      keep_alive true

Versioning Brew Formulas

One issue we eventually stumbled upon was versioning our software releases with Homebrew. We found after every release, GoReleaser would update the Formula repo by overwriting the previous formula. A user could update the formula and run brew upgrade to easily get the latest version. The issue we ran into was what if you wanted a specific version of the Collector with a specific brew formula? You would have to know which commit in the Formula corresponds to the release you want. That’s not very user-friendly.

This also made it hard for us to test pre-releases as we wanted GoReleaser to generate formulas for release candidates but not overwrite the production one.

It wasn’t easy to find out how to version Homebrew Formulas. We ended up looking at the homebrew-core repo for examples of how other formulas do it.

To version a formula there are a few special things to do. The formula name needs to be of the format [email protected]. The added @major.minor.patch allows Homebrew to know which formula to get when specified in the brew command. Inside the formula the class name must have special formatting too. It must be of the format FormulaNameAT{Major}{Minor}{Patch}. So an example filename and corresponding class name for our Collector is [email protected] and ObserviqOtelCollectorAT060, respectively.

That formula file will exist in the Formula directory of your repo next to the current main formula, the formula you get if you just run brew install X. You can also add a version to the main formula so users can get it by version or by the basic brew command. To do this, you create an Aliases directory on the same level as your Formula directory. Inside that directory create a symlink to the main formula with a versioned name.

If that’s confusing, here’s the command we run to create the symlink:

cd Aliases && ln -s ../Formula/observiq-otel-collector.rb [email protected]

Now that we know how to create a versioned formula we need to update our GoReleaser config to generate versioned formulas for us. This should be simple since the formula and class names are taken from the name field under the brews block. We changed our name to observiq-otel-collector@{{ .Major }}.{{ .Minor }}.{{ .Patch }}. When we ran a test release with GoReleaser though we saw the class for the formula wasn’t quite right. GoReleaser was generating the class name as ObserviqOtelCollectorAT0_6_0. One quick pull request to GoReleaser and we fixed that.

Here’s what our brews block of our GoReleaser config now looks like to support versions.

We also made some changes to the configuration of the Collector so there are some additional flags and files in the install and service blocks.

brews:
  - name: observiq-otel-collector@{{ .Major }}.{{ .Minor }}.{{ .Patch }}
    description: "observIQ's distribution of the OpenTelemetry Collector"
    tap:
      owner: observIQ
      name: homebrew-observiq-otel-collector
      branch: main
    download_strategy: CurlDownloadStrategy
    folder: Formula
    url_template: https://github.com/observIQ/observiq-otel-collector/releases/download/{{ .Tag }}/{{ .ArtifactName }}
    commit_author:
      name: observiq
      email: [email protected]
    homepage: "https://github.com/observIQ/observiq-otel-collector"
    license: "Apache 2.0"
    caveats: |
      ****************************************************************
      The configuration file that is run by the service is located at #{prefix}/config.yaml.
      If you are configuring a logreceiver the plugin directory is located at #{prefix}/plugins.
      ****************************************************************
      ****************************************************************
      Below are services commands to run to manage the collector service.
      If you wish to run the collector at boot prefix these commands with sudo
      otherwise the service will run at login.
      To start the collector service run:
        brew services start observiq/observiq-otel-collector/observiq-otel-collector@{{ .Major }}.{{ .Minor }}.{{ .Patch }}
      To stop the collector service run:
        brew services stop observiq/observiq-otel-collector/observiq-otel-collector@{{ .Major }}.{{ .Minor }}.{{ .Patch }}
      To restart the collector service run:
        brew services restart observiq/observiq-otel-collector/observiq-otel-collector@{{ .Major }}.{{ .Minor }}.{{ .Patch }}
      ****************************************************************
      ****************************************************************
      To uninstall the collector and its dependencies run the following commands:
          brew services stop observiq/observiq-otel-collector/observiq-otel-collector@{{ .Major }}.{{ .Minor }}.{{ .Patch }}
          brew uninstall observiq/observiq-otel-collector/observiq-otel-collector@{{ .Major }}.{{ .Minor }}.{{ .Patch }}
          # If you moved the opentelemetry-java-contrib-jmx-metrics.jar
          sudo rm /opt/opentelemetry-java-contrib-jmx-metrics.jar
      ****************************************************************
    install: |
      bin.install "observiq-otel-collector"
      prefix.install "LICENSE", "config.yaml", "VERSION.txt", "logging.yaml"
      prefix.install Dir["plugins"]
      lib.install "opentelemetry-java-contrib-jmx-metrics.jar"
    service: |
      run [opt_bin/"observiq-otel-collector", "--config", opt_prefix/"config.yaml", "--logging", opt_prefix/"logging.yaml", "--manager", opt_prefix/"manager.yaml"]
      environment_variables OIQ_OTEL_COLLECTOR_HOME: opt_prefix
      keep_alive true

Persisting Configuration Files

Initially, in our install block of the GoReleaser config, we were using prefix.install to place our configuration file in the main install directory of our formula.

install: |
    …
    prefix.install "LICENSE", "VERSION.txt", "config.yaml"
    …

We found that after reinstalling this formula, our configuration file would be replaced with fresh defaults, and any user changes would be lost. This wasn’t ideal, so we had to figure out how to ensure that this file would persist between installations.

Ultimately, the solution was to use Homebrew’s etc directory. This is a shared directory amongst all formulas, so we had to make an extra effort to ensure our configuration file would be uniquely named. Now our GoReleaser install block looks something like this:

install: |
    …
    prefix.install "LICENSE", "VERSION.txt"
    etc.install "config.yaml" => "observiq_config.yaml"
    …

The problem was almost solved at this point, but we still had a preference to have this configuration file ‘exist’ in the base formula install directory. We also wanted to have this configuration file have its original name without the ‘observiq_’ prefix. Luckily, using a symlink was a simple solution to this. Our final install block related to the configuration looked similar to this:

install: |
    …
    prefix.install "LICENSE", "VERSION.txt"
    etc.install "config.yaml" => "observiq_config.yaml"
    prefix.install_symlink etc/"observiq_config.yaml" => "config.yaml"

With this solution, our configuration lived safely in Homebrew’s etc directory with a special prefix and where it would never be automatically overwritten. At the same time, it would appear to also exist in the base installation directory without any naming prefix.

There is one more thing to note here. When there is a new installation on top of our formula, homebrew automatically adds a new version of our configuration file to its etc directory. In our case, the file is named something like observiq_config.yaml.default. This will contain a clean config with default settings. This is built-in behavior for Homebrew, and we haven’t found any way to change this.

Conclusion

GoReleaser provides a great way to distribute your Go program via Homebrew. It allows you to focus on the installation part of your application while taking care of all the formatting and setup of your formula file. Hopefully, we’ve given some good insight into the pitfalls we encountered to help simplify using GoReleaser and Homebrew for you.