Creating a Dataset Sucks. Here's What I've Learned to Make it a Little Bit Easierby@calmdownkarm

Creating a Dataset Sucks. Here's What I've Learned to Make it a Little Bit Easier

tldt arrow
Read on Terminal Reader

Too Long; Didn't Read

Creating datasets is hard, even for relatively simple tasks like document classification/sentiment analysis. Even a relatively high agreement score can allow for 10% of your data to be wrong. Hiring a set of annotators is hard too, but once you figure out the hiring, here's some things that I've learned managing annotators. Getting a minimal set of data annotated, and trying to build models on it will help target the specific kinds of annotated data that you need, or even bad classifiers can help filter through a lot of data.

Companies Mentioned

Mention Thumbnail
Mention Thumbnail
featured image - Creating a Dataset Sucks. Here's What I've Learned to Make it a Little Bit Easier
Karmanya Aggarwal HackerNoon profile picture

@calmdownkarm

Karmanya Aggarwal
react to story with heart

RELATED STORIES

L O A D I N G
. . . comments & more!
Hackernoon hq - po box 2206, edwards, colorado 81632, usa