145 reads

Go-CoNLLU Introduction: OSS Tool For Machine Learning Support in Go

by Lane WagnerAugust 14th, 2020

Too Long; Didn't Read

Go-Conllu is an OSS tool for Machine Learning Support in Go. It is a simple and reliable way to import conllu data into your application as Go structs. GoDoc can be found here with the specifics of the GoDoc. The Universal Dependency Project hosts many annotations of textual data. In order to use these corpora, we need a parser that makes it simple for developers to utilize the data. Go-conllu parses Conllu and prints sentences and tokens to the console.

Company Mentioned

featured image - Go-CoNLLU Introduction: OSS Tool For Machine Learning Support in Go

Python is commonly seen as the AI/ML language, but is often a dull blade due to unsafe typing and being slow, like really slow. Many popular natural language processing toolkits only have Python APIs, and we want to see that change. At Nuvi, we use Go for the majority of our data processing tasks because we can write simple and fast code. Today we are open-sourcing a tool that has helped make our ML lives easier in Go. Say hello to go-conllu.

What is CoNLL-U?

The Conference on Natural Language Learning (CoNNL) has created multiple file-formats for storing natural language annotations. CoNLL-U is one such format and is used by the Universal Dependency Project, which hosts many annotations of textual data. In order to use these corpora, we need a parser that makes it simple for developers to utilize the data.

How Does Go-Conllu Help?

Go-conllu parses conllu data. It is a simple and reliable way to import conllu data into your application as Go structs.

The GoDoc can be found here with the specifics

Let's take a look at the example quick-start code from the Readme. First, download the package.

go get github.com/nuvi/go-conllu

Then in a new project:

package main

import (
	"fmt"
	"log"

	conllu "github.com/nuvi/go-conllu"
)

func main() {
	sentences, err := conllu.ParseFile("path/to/model.conllu")
	if err != nil {
		log.Fatal(err)
	}

	for _, sentence := range sentences {
		for _, token := range sentence.Tokens {
			fmt.Println(token)
		}
		fmt.Println()
	}
}