Mat Evans

@matzhouse

How fast is yes in Go?

I saw this link on Hackernews this morning.
It’s a small experiment on the speed of writing from memory using the yes command.

$ yes
y
y
y
y
y
y
y
y
y

That’s the entirety of the the command. It prints the letter y, forever.

From the article, we can use the pv command to test the bandwidth used to write to /dev/null. From the pv man page..

pv allows a user to see the progress of data through a pipeline, by giving information such as time elapsed, percentage completed (with progress bar), current throughput rate, total data transferred, and ETA.

Results on my old mac mini running Ubuntu

$ yes | pv > /dev/null
... [3.59GiB/s] [

Results on my macbook pro running OSX

$ yes | pv > /dev/null
... [26.2MiB/s]

Well.. that’s a bit of a difference. It looks like the yes command is OSX is shit. Who knew? Let’s see if we can make it faster.

GOPHERS ASSEMBLE!

Yes, we’re going to try and make a faster yes in Go. How fast can we get?

Round 1

package main
import (
"fmt"
)
func main() {
  for {
fmt.Println("y")
}
}

erm.. slow on OSX

./yes | pv > /dev/null
... [1.69MiB/s]

and Linux

$ ./yes | pv > /dev/null
... [1.99MiB/s]

What if we use a more direct method of writing to stdout?

y := []byte("y\n")
for {
os.Stdout.Write(y)
}

a bit better on OSX

./yes | pv > /dev/null
... [1.99MiB/s]

and a bit better on Linux

$ ./yes | pv > /dev/null
... [3.05MiB/s]

Round 2

What happens if we use the same trick as the article. The main way it speeds things up is to create a buffer of output in an array in memory. This can then be very quickly written to Stdout.

Something like this.

package main
import (
"os"
)
func main() {
  // create a buffer that has space equal to the page size
buf := make([]byte, 4096)
  y := []byte("y\n")
used := 0
  for {
    buf[used] = y[0]
buf[(used)+1] = y[1]
    used += 2
    if used == 4096 {
break
}
}
  // Print out the entire buffer at once - writing direct to
// stdout is the fastest way of doing this.
// It's the size of a standard linux pagefile.
for {
os.Stdout.Write(buf)
}
}

and the results?

On OSX

$ ./yes | pv > /dev/null
... [1.88GiB/s]

On Linux

$ ./yes | pv > /dev/null
... [2.75GiB/s]

Well that looks ok! On OSX we’ve managed nearly 105x speed increase. On Linux, slightly less good results but I’m not surprised there — the Linux version seems to be the the most efficient anyway.

Can we do better?

One more thing to try, what if we use a bigger buffer?

buf := make([]byte, 16384)

OSX

$ ./yes | pv > /dev/null
... [2.98GiB/s]

Linux

$ ./yes | pv > /dev/null
... [ 3.19GiB/s]

I think i’ll leave it there :)

If you have any questions please feel free to comment here or tweet me — matzhouse

Hit the heard if you enjoyed reading this!

More by Mat Evans

Topics of interest

More Related Stories