How to Send Millions of Push Notifications with Go and Firebase Cloud Messaging

Hey everybody! My name is Vladislav Gukasov. I am a software engineer in a fintech company in the communications team. One of the most effective communication tools that we use today is the push notification service that dispatches tens of millions of messages per day.

If you develop a push notification system the first time, it can be tricky. I’ll tell you how to configure a service, send push messages and avoid some mistakes that can cause poor performance.

Push notifications

Firstly, we need to be clear about what’s push notifications are. Push notifications are short messages that a web resource sends to its subscribers on computers and mobile devices. Notifications like that return the user to the site (or application) that he has visited already. The technical capabilities of push notifications allow developing a more productive and “responsive” marketing strategy for a product.

Why you should use push notifications in your application:

• a user will see your message most likely
• it's free (unlike SMS and emails)
• privacy: the user does not give you any personal data. You communicate through an impersonal push token

Service architecture

Let's move on to the implementation of the service for sending push notifications. We will use Golang and Firebase Cloud Messaging. A little more detail why:

Go

• notification service is usually a microservice, and Go suits greatly for that
• works concurrently, utilizing the CPU as much as possible, which gives high performance
• has built-in profiling / benchmarking tools

Firebase Cloud Messaging

• free solution from google
• you can send notifications to Android via Firebase only
• there are SDKs for most programming languages

Integration with Firebase takes place on both the client and server-side. All client platforms must install due SDK and generate a push token. Finished tokens are sent to the server—the server stores tokens in storage.

When any service needs to send a notification, we get push tokens from the storage and send a message.

SDK Installation and initialization

The first thing we need to do is install the Firebase package

go get firebase.google.com/go

Before we can use the SDK, we need to get the authorization credits. To do this, you need to create a new project in the Firebase console and get the auth credentials from the service account.

Now we can initialize the client in our code.

import (
  "context"

  firebase "firebase.google.com/go"
 "firebase.google.com/go/messaging"
 "google.golang.org/api/option"
)

// There are different ways to add credentials on init.
// if we have a path to the JSON credentials file, we use the GOOGLE_APPLICATION_CREDENTIALS env var
os.Setenv("GOOGLE_APPLICATION_CREDENTIALS", c.Firebase.Credentials)
// or pass the file path directly
opts := []option.ClientOption{option.WithCredentialsFile("creds.json")}

// if we have a raw JSON credentials value, we use the FIREBASE_CONFIG env var
os.Setenv("FIREBASE_CONFIG", "{...}")

// or we can pass the raw JSON value directly as an option
opts := []option.ClientOption{option.WithCredentialsJSON([]byte("{...}"))}


app, err := firebase.NewApp(ctx, nil, opts...)
if err != nil {
    log.Fatalf("new firebase app: %s", err)
}

fcmClient, err := app.Messaging(context.TODO())
if err != nil {
    log.Fatalf("messaging: %s", err) 
}

How to use Firebase with proxy

In some projects, the mailing service is located on the internal network, and it has no access to the outside world. To access external APIs, you have to use a proxy. FCM client makes 2 types of HTTP requests: receiving OAuth access tokens and sending notifications. This is how we can initialize a client with a proxy.

import (
  "context"

  firebase "firebase.google.com/go"
 "firebase.google.com/go/messaging"
 "google.golang.org/api/option"
)

proxyURL := "http://localhost:10100" // insert you proxy here 

// The SDK makes 2 different types of calls:
// 1. To the Google OAuth2 service to fetch the refresh and access tokens.
// 2. To Firebase to send the pushes.
// Each type uses its own HTTP Client and we need to insert our custom HTTP Client with proxy everywhere.
cl := &http.Client{
	Transport: &http.Transport{Proxy: http.ProxyURL(proxyURL)},
}
ctxWithClient := context.WithValue(ctx, oauth2.HTTPClient, cl)

// This is how we insert our custom HTTP Client in the Google OAuth2 service:
// by context with specific value.
creds, err := google.CredentialsFromJSON(ctxWithClient, []byte(c.Firebase.Credentials), firebaseScopes...)
if err != nil {
    log.Fatalf("google credentials from JSON: %s", err)
}

// And this is how we insert proxy for the Firebase calls. Initialize base transport with our proxy.
tr := &oauth2.Transport{
	Source: creds.TokenSource,
	Base:   &http.Transport{Proxy: http.ProxyURL(proxyURL)},
}

hCl := &http.Client{
	Transport: tr,
	Timeout:   10 * time.Second,
}

opts := []option.ClientOption{option.WithHTTPClient(hCl)}

app, err := firebase.NewApp(ctx, nil, opts...)
if err != nil {
    log.Fatalf("new firebase app: %s", err)
}

fcmClient, err := app.Messaging(context.TODO())
if err != nil {
    log.Fatalf("messaging: %s", err)
}

Let’s send some push notifications

Common push dispatch is very simple. We pass the title, body, and client push token, and the notification is sent.

response, err := fcmClient.Send(ctx, &messaging.Message{
	Notification: &messaging.Notification{
		Title:    "A nice notification title",
		Body:     "A nice notification body",
	},
	Token: "client-push-token", // a token that you received from a client
})

Let's take a look at the performance of the implementation using Go's built-in benchmarking tool.

Sent 590 push messages by service
Benchmark_Service_SendPush-8   	     590	   2099685 ns/op

We see that each call takes an average of 2.1 ms, and we sent only 590 notifications in 1.24s. Therefore, the implementation may be suitable for you if you have few users and just started to send notifications.

Performance improvement

However, the implementation described above is not enough to send millions of notifications. We don't want to make N calls because each message imposes an overhead in milliseconds. We need to group messages into batches and send hundreds of messages at once. For this purpose, we will implement a simple buffer. Firebase provides a batch API for such cases. In response, you will receive an array, the indices of which are the same as the array of messages sent by you.

import (
	"context"
	"log"
	"sync"
	"time"

	"firebase.google.com/go/messaging"
)

// Buffer batches all incoming push messages and send them periodically.
type Buffer struct {
	fcmClient *messaging.Client

	dispatchInterval time.Duration
	batchCh          chan *messaging.Message
	wg               sync.WaitGroup
}

func (b *Buffer) SendPush(msg *messaging.Message) {
	b.batchCh <- msg
}

func (b *Buffer) sender() {
	defer b.wg.Done()

	// set your interval
	t := time.NewTicker(b.dispatchInterval)

	// we can send up to 500 messages per call to Firebase
	messages := make([]*messaging.Message, 0, 500)

	defer func() {
		t.Stop()

		// send all buffered messages before quit
		b.sendMessages(messages)

		log.Println("batch sender finished")
	}()

	for {
		select {
		case m, ok := <-b.batchCh:
			if !ok {
				return
			}

			messages = append(messages, m)
		case <-t.C:
			b.sendMessages(messages)
			messages = messages[:0]
		}
	}
}

func (b *Buffer) Run() {
	b.wg.Add(1)
	go b.sender()
}

func (b *Buffer) Stop() {
	close(b.batchCh)
	b.wg.Wait()
}

func (b *Buffer) sendMessages(messages []*messaging.Message) {
	if len(messages) == 0 {
		return
	}

	batchResp, err := b.fcmClient.SendAll(context.TODO(), messages)

	log.Printf("batch response: %+v, err: %s \n", batchResp, err)
}

Let's take a look at the performance of the buffer implementation

Sent 1513794 push messages by buffer
Benchmark_Buffer_SendPush-8    	 1513794	       677.9 ns/op

We see that each call takes on average 0.00068 ms, and we sent ~ 1.5 million notifications in 1.02s. That’s a great boost that can handle a lot of push messages fastly. For further improvements, we can just horizontally scale our service.

Conclusion

The performance of sending push notifications depends on many factors that are unique to each project. We've considered examples of improvements from the code side, but there are many growth points beyond that.

Anyway, it is worth focusing first and foremost on users and their needs. If a simple implementation of "N API calls" works successfully in your project, then you should not prematurely optimize and complicate the code. The simpler your system is, the cheaper it is to maintain it, and the faster your users will see results.