How To Optimize Large S3 API Costs using Alluxioby@bin-fan

How To Optimize Large S3 API Costs using Alluxio

tldt arrow
Read on Terminal Reader
Read this story w/o Javascript

Too Long; Didn't Read

This article describes how engineers at Datasapiens brought down S3 API costs by implementing Alluxio as a data orchestration layer between S3 and Presto. The main conclusion from these use cases is that using AlluxIO has the following benefits: lower latency in data processing pipelines. The infrastructure costs were above expectations due to S3 cost increases, making up a large proportion of the overall cost (described in detail below) This architecture achieves faster performance and reduced compilation times. In addition, we saw a drop in infrastructure costs due to the increase in S3 costs.

Companies Mentioned

Mention Thumbnail
Mention Thumbnail
featured image - How To Optimize Large S3 API Costs using Alluxio
Bin Fan HackerNoon profile picture

@bin-fan

Bin Fan

VP of Open Source and Founding Member @Alluxio


Receive Stories from @bin-fan

react to story with heart

RELATED STORIES

L O A D I N G
. . . comments & more!