Building an Efficient AI Platform for Data Preprocessing and Model Trainingby@bin-fan
386 reads
386 reads

Building an Efficient AI Platform for Data Preprocessing and Model Training

by Bin Fan8mJuly 13th, 2022
Read on Terminal Reader
Read this story w/o Javascript
tldt arrow

Too Long; Didn't Read

Lei Li, AI Platform Lead, and Zifan Ni, Senior Software Engineer from Bilibili, share how they applied Alluxio to their AI platform to increase training efficiency, as well as best practices including technical architecture and specific tuning tips. They say they came across four major challenges before adopting the AI platform, including the container crash and the need to restart the container to download the data again. They also share their best practices and best practices for training AI jobs in the cloud and on-premises.

Companies Mentioned

Mention Thumbnail
Mention Thumbnail
featured image - Building an Efficient AI Platform for Data Preprocessing and Model Training
Bin Fan HackerNoon profile picture
Bin Fan

Bin Fan

@bin-fan

VP of Open Source and Founding Member @Alluxio

About @bin-fan
LEARN MORE ABOUT @BIN-FAN'S
EXPERTISE AND PLACE ON THE INTERNET.

Share Your Thoughts

About Author

Bin Fan HackerNoon profile picture
Bin Fan@bin-fan
VP of Open Source and Founding Member @Alluxio

TOPICS

THIS ARTICLE WAS FEATURED IN...

Permanent on Arweave
Read on Terminal Reader
Read this story in a terminal
 Terminal
Read this story w/o Javascript
Read this story w/o Javascript
 Lite
L O A D I N G
. . . comments & more!