When working with distributed systems generally the more logs, the better — the complication lies in being liberal for visibility while being conservative to keep sensitive information private and secure. It’s not easy, just ask GitHub and Twitter who found themselves logging raw passwords recently.
The details of GitHub’s and Twitter’s incidents weren’t made public but I’d guess an engineer changed a log level config or logged the request bodies without realizing the consequences.
The fact is this is the reality of the job — with complex systems like these no one can know the consequences of every change, especially so with the fast tech turnover rate. If we’re gonna solve problems like these we need to go deeper.
Issues like this arise by Murphy’s law: whatever can go wrong, will go wrong — if you leave a trap behind someone will walk into it. Let’s not do that. When writing Go, here’s what I do to leave no such trap behind and prevent the logging of secrets.
Let’s say you have a sign up request like so:
type CreateUserRequest struct {Credentials Credentials `json:”credentials”`}
type Credentials struct {Email string `json:”email”`Password string `json:”password”`}
And let’s say you’re logging to JSON for Elasticsearch:
// logger we’re usinglogger := log.NewJSONLogger(os.Stdout)
// request we decodedrequest := CreateUserRequest{Credentials: Credentials{Email: “[email protected]”,Password: “theonering”,},}
// our call to log the requestlogger.Log(“request”, request)
// the output{“request”:{“credentials” {“email”:”[email protected]”,”password”:”theonering”}}}
The result is the user’s password is logged. Not good.
To solve the problem we’ll implement our own MarshalJSON that’ll redact the user’s password.
func (co Credentials) MarshalJSON() ([]byte, error) {type credentials Credentialscn := credentials(co)cn.Password = "[REDACTED]"return json.Marshal((*credentials)(&cn))}
Now when we log the same request the user’s password is not logged:
// the output{“request”:{“credentials”:{“email”:”[email protected]”,”password”:”[REDACTED]”}}}
There is now no possibility of logging the user’s password, regardless of who logs, nor where or when — at least if you’re logging JSON with this implementation. If you’re logging in another format, the same technique applies and there should be a corresponding marshal method for you to implement.
–
Please say hi at @travisjeffery.
Hit the 👏 and share if you found this useful.
Thanks for reading.