2 Error-Free Options for Decimal handling in Golang

January 28th 2021

@vijaysavanthVijay Savanth

Lead Engineer. Based in Tāmaki Makaurau, Aotearoa (Auckland, New Zealand).

Motivation

Early on in my Go journey, I was building a billing engine. I needed to handle decimal calculations accurately. Unfortunately, Go does not have a native decimal type. After about an hour of research I found 2 error free solutions.

Fast forward to Jan 2021 and a Google search for

decimal handling in golang

retrieves the following results:

In my view, most people who contribute to open source projects are amazing and brave. It takes a lot of courage to put your work and ideas out there. My intention here is not to criticise their work. At the time this post was penned, the library mentioned in the first search result is a poor choice if you want accurate results.

There are numerous open issues clearly indicating incorrect results. The second search result has an open issue for incorrect results with very large values. All the other search results are discussing various options to handle decimal operations.

The motivation for this post is to discuss 2 error free options with the hope that this post reaches more gophers.

Option 1 - https://github.com/cockroachdb/apd

CockroachDB is a popular open source DB written in Go. I use it for most of my projects. I was using it for the billing engine I was building and my investigation led me to the library they were maintaining.

Here is some example code for decimal division. Note: make sure you are using the latest version of the library -

v2

at the time of this post.

package main

import (
	"fmt"
	"log"

	"github.com/cockroachdb/apd/v2"
)

func main() {
	// Division needs a rounding specified.
	// Precision here is the total number of digits 
	// to the left and right of the decimal place.
	context := apd.BaseContext.WithPrecision(10)

	// x = 1.234
	xDecimal, _, err := apd.NewFromString("1.234")
	if err != nil {
		log.Fatalln(err)
	}

	// y = 3
	yDecimal := apd.New(3, 0)

	result := apd.New(0, 0)

	_, err = context.Quo(result, xDecimal, yDecimal)
	if err != nil {
		log.Fatalln(err)
	}

	fmt.Println(result)
}

Produces the following result:

0.4113333333

Here is some example code based on discussion here to control rounding.

package main

import (
	"errors"
	"fmt"
	"log"
	"math"

	"github.com/cockroachdb/apd/v2"
)

var (
	// DecimalCtx is the default context for decimal operations. Any change
	// in the exponent limits must still guarantee a safe conversion to the
	// postgres binary decimal format in the wire protocol, which uses an
	// int16. See pgwire/types.go.
	DecimalCtx = &apd.Context{
		Precision:   20,
		Rounding:    apd.RoundHalfUp,
		MaxExponent: 2000,
		MinExponent: -2000,
		// Don't error on invalid operation, return NaN instead.
		Traps: apd.DefaultTraps &^ apd.InvalidOperation,
	}
)

func main() {
	xDecimal, _, _ := apd.NewFromString("14.955")
	// Max 4 digits to the left and right of decimal place. Max 1 digit to right of decimal place.
	err := LimitDecimalWidth(xDecimal, 4, 1)
	if err != nil {
		log.Println(err)
	}
	fmt.Printf("x= %v\n", xDecimal)

	// Max 4 digits to the left and right of decimal place. Max 2 digits to right of decimal place.
	yDecimal, _, _ := apd.NewFromString("14.955")
	err = LimitDecimalWidth(yDecimal, 4, 2)
	if err != nil {
		log.Println(err)
	}
	fmt.Printf("y= %v\n", yDecimal)
}

func LimitDecimalWidth(d *apd.Decimal, precision, scale int) error {
	if d.Form != apd.Finite || precision <= 0 {
		return nil
	}
	// Use +1 here because it is inverted later.
	if scale < math.MinInt32+1 || scale > math.MaxInt32 {
		return errors.New("out of range")
	}
	if scale > precision {
		return fmt.Errorf("scale (%d) must be between 0 and precision (%d)", scale, precision)
	}

	// http://www.postgresql.org/docs/9.5/static/datatype-numeric.html
	// "If the scale of a value to be stored is greater than
	// the declared scale of the column, the system will round the
	// value to the specified number of fractional digits. Then,
	// if the number of digits to the left of the decimal point
	// exceeds the declared precision minus the declared scale, an
	// error is raised."

	c := DecimalCtx.WithPrecision(uint32(precision))
	c.Traps = apd.InvalidOperation

	if _, err := c.Quantize(d, d, -int32(scale)); err != nil {
		var lt string
		switch v := precision - scale; v {
		case 0:
			lt = "1"
		default:
			lt = fmt.Sprintf("10^%d", v)
		}
		return fmt.Errorf("value with precision %d, scale %d must round to an absolute value less than %s", precision, scale, lt)
	}
	return nil
}

Output

x= 15.0
y= 14.96

Option 2 - Use a database

Most popular databases have native support for decimal type. If your app involves the use of a database and you need to perform decimal operations on values which are already stored in the DB, then it may be convenient to use SQL to perform decimal operations instead of having to retrieve data and then perform decimal operations in Go.

In the event data is not already in the DB, you can construct a SQL statement to perform decimal operations with the understanding that there is a network hop.

Example SQL:

// PostgreSQL
SELECT ('1.234'::numeric/'3'::numeric)::numeric(20,10);

// CockroachDB
SELECT ('1.234'::decimal/'3'::decimal)::decimal(20,10);

Gotchas for Option 1

You will need to spend a couple of hours learning the library.

Gotchas for Option 2

A network hop is involved if decimal operations cannot be performed directly on data in the DB. It may pay to spend some time to measure this delay and then assess if option 2 fits your needs for data not already in the DB.
The network hop results in a less performant solution when compared to Option 1.
Setting up a DB to perform decimal operations will increase cost and complexity if your app does not need a DB.

Summary

2 error free options for handling decimal types in Go have been presented. Always prototype new options to measure and understand if they can fit your needs.

Trivia

Did you know that the co-founder and current CEO of CockroachDB was one of the original authors of the popular tool Gimp?

Disclaimer

I am not an employee of CockroachDB.

Win $2000 by building a web app on Velo by Wix