Billion-Dollar Mistake in Go?

The following sample code is from Go's standard library documentation:

data := make([]byte, 100)
count, err := file.Read(data)
if err != nil {
	log.Fatal(err)
}
fmt.Printf("read %d bytes: %q\n", count, data[:count])

It seems to be ok. It must be correct because it's from the official documentation of the standard library, right?

Let's spend a few seconds to figure out what's wrong with it before reading the documentation of the

io.Reader

Read

The

if

if err != nil && err != io.EOF {

Did I trick you (and my self)? Why didn’t we check the 

File.Read

What good comes with interfaces if we really cannot hide the implementation details with them? The interface should set its semantics, not the implementer as 

File.Read

File

io.Reader

io.EOF

io.Reader

Interface vs Implementer

In Go, you don’t need to mark an implementer of the interface explicitly. It’s a powerful feature. But does it mean that we should always use interface semantics according to the static type? For example, should the following 

Copy

io.Reader

func Copy(dst Writer, src Reader) (written int64, err error) {
	src.Read() // now read semantics come from io.Reader?
	...
}

But should this version use only 

os.File

func Copy(dst os.File, src os.File) (written int64, err error) {
	src.Read() // and now read semantics come from os.File's Read function?
	...
}

The practice has thought it’s always better to use interface semantics instead of the binding yourself to the implementation—the famous loose coupling.

Problems with io.Reader

The interface has the following problems :

• You cannot safely use any implementation of the 

  Read

   function without studying documentation of the 

  io.Reader

  .
• You cannot implement the 

  Read

   function without closely studying documentation of the 

  io.Reader

  .
• The interface is not intuitive, complete, and idiomatic because of missing the error distinction.

The previous problems multiply because of 

io.Reader

io.Reader

Read

There are many other examples in the standard library itself where callers of the 

io.Reader

According to this issue, the standard library and especially its tests are tight to the 

if err != nil

Read

For instance, you cannot return 

io.EOF

When Read encounters an error or end-of-file condition after successfully reading n > 0 bytes, it returns the number of bytes read. It may return the (non-nil) error from the same call or return the error (and n == 0) from a subsequent call.

Interfaces should be intuitive and formally defined with the programming language itself that you cannot implement or misuse them. You should not need to read the documentation to be able to do necessary error propagation.

It’s problematic that multiple (two in this case) different explicit behaviour of the interface function is allowed. The whole idea of the interfaces is that they hide the implementation details and enable loose coupling.

The most obvious problem is that the 

io.Reader

EOF is the error returned by Read when no more input is available. Functions should return EOF only to signal a graceful end of input. If the EOF occurs unexpectedly in a structured data stream, the appropriate error is either ErrUnexpectedEOF or some other error giving more detail.

Errors as Discriminated Unions

The 

io.Reader

io.EOF

Herb Shutter conveniently put in his C++ proposal, Zero-overhead deterministic exceptions: Throwing values:

“Normal” vs. “error” [control flow] is a fundamental semantic distinction, and probably the most important distinction in any programming language even though this is commonly underappreciated.

Solution

Go’s current 

io.Reader

Adding The Semantic Distinction

First, we stop using error return for something which isn’t an error by declaring a new interface function.

Read(b []byte) (n int, left bool, err error)

Allowing Only Obvious Behaviour

Second, to avoid confusion and prevent clear errors we have guided to use the following helper wrapper to handle both of the allowed EOF behaviours. The wrapper offers only one explicit conduct to process the end of the data. Because the documentation says that returning zero bytes without any error (including EOF) must be allowed (“discouraged from returning a zero byte count with a nil error“) we cannot use zero bytes read as a mark of the EOF. Of course, the wrapper also maintains the error distinction.

type Reader struct {
	r   io.Reader
	eof bool
}

func (mr *MyReader) Read(b []byte) (n int, left bool, err error) {
	if mr.eof {
		return 0, !mr.eof, nil
	}
	n, err = mr.r.Read(b)
	mr.eof = err == io.EOF
	left = !mr.eof
	if mr.eof {
		err = nil
		left = true
	}
	return
}

We made an error distinction rule where error and success results are exclusive. We have used the distinction for the 

left

left

for n, left, err := src.Read(dst); err == nil && left; n, left, err = src.Read(dst) {
	fmt.Printf("read: %d, data left: %v, err: %v\n", n, left, err)
}

As the sample code shows, it allows a happy path and error control flows to be separated, which makes program reasoning much easier. The solution we showed here isn’t perfect because Go’s multiple return values aren’t distinctive.

In our case, they all should be. However, we have learned that every newcomer (also them who are new with Go) can use our new

Read

Conclusion

Can we say that 

io.EOF

Go’s error handling practice is still missing language features to help the semantic distinction. Luckily, most of us already treat errors in the distinct control flow.