![Search icon](https://hackernoon.imgix.net/search-new.png?w=19&h=19)
A feed with diverse media content and infinite scroll is a typical task in modern web app development and system-design interviews. Let's try to implement the feed with heavy graphic elements, gradually applying various performance optimization techniques.
Here are the functional requirements I encountered when building such a feed for a data analytics platform:
Hundreds of charts on a single page.
The charts come in various types: diagrams, line graphs, heat maps, pins on Google maps, etc.
Each chart can consist of hundreds or even thousands of data points.
We want to display a feed with a Pinterest layout on one page. The page should perform well even on mobile devices.
Although the chart cards themselves may not be interactive, clicking on a card in the feed should open a full-screen version of that chart, which has tons of interactivity.
Let's try to build a simplified version of this feed, considering different techniques to optimize the performance of heavy pages with complex graphics. It may seem like a rather typical task. You can take your favorite framework or library for drawing charts, connect it to the API, and you're done, right?
We start with the application template and essential components: the feed page and card component.
The ChartCard
component will fetch chart data from the API and render a chart of the desired type. Let's assume that all cards have the same height. This will allow us to render the feed layout using a regular CSS grid instead of implementing a masonry layout.
const components = {
line: LineChartContainer,
bar: BarChartContainer,
//.. other chart types
};
export default function ChartCard({ id, type }: {id: string, type: ChartType }) {
const { data } = useQuery({
queryKey: ['chartData', id],
queryFn: () => getChartData(id),
});
return (
<div className="shadow-md p-4 bg-white rounded-lg max-h-96">
<ChartComponent data={data ?? []} />
</div>
);
}
function BarChartContainer({ data }: { data: ChartData[] }) {
return (
<ResponsiveContainer width="100%" height={300}>
<BarChart data={data} height={300}>
<CartesianGrid strokeDasharray="3 3" />
<XAxis dataKey="name" tick={false} />
<YAxis />
<Bar dataKey="pv" fill={colorsQCS[4]} />
<Bar dataKey="uv" fill={colorsQCS[1]} />
</BarChart>
</ResponsiveContainer>
);
}
// ... function LineChart({ data }: { data: ChartData[] }) {
Now, we need to build a Feed
page. We can start really simple and try to fetch and render all feed entries at once without pagination. We don't expect outstanding performance from this implementation, we just want to check that everything works
export default function FeedNaive() {
const { data } = useQuery({
queryKey: ['charts'],
queryFn: () => getCharts(),
});
return (
<div className="grid sm:grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-4">
{data?.map((c) => (
<ChartCard key={c.id} id={c.id} type={c.type} />
))}
</div>
);
}
We will also need the backend API. So we put together a tiny express application with several endpoints:
/api/charts
endpoint with list with the graph graphs. Let’s not forget to implement simple pagination here./api/charts/:id
enpdoint with data for rendering individual graph.
import express from 'express';
import { getCharts, getChartData } from './chart-service'
const app = express();
app.get('/api/charts', async (req, res) => {
const page = parseInt(req.query.page);
const from = !isNaN(page) ? page * 20 : 0;
const to = !isNaN(page) ? start + 20 : undefined;
res.json(getCharts(from, to));
});
app.get('/api/charts/:id', async (req, res) => {
res.json(getChartData(req.params.id));
});
app.listen(8080);
This is what we’ve got after the initial project setup. Visually it looks good, but the performance is not the best. This is especially true if there are more than a few dozen charts on the page and when some of the charts contain several hundred data points each.
The first thing that naturally comes to mind is to add paging to reduce the number of charts on the page. With @tanstack/react-query
we can easily achieve this by using useInfiniteQuery
hook. Next, it is better to turn off animations on the charts with isAnimationActive={false}
At this stage we should also ensure that that existing cards won’t re-render when we render updated list with a new cards.
import { Fragment } from 'react';
import { useInfiniteQuery } from '@tanstack/react-query';
import ChartCard from './ChartCard';
import { getCharts } from '../api';
export default function ChartsPaginated() {
const {
data, isFetchingNextPage, fetchNextPage, hasNextPage
} = useInfiniteQuery({
queryKey: ['charts-paginated'],
queryFn: ({ pageParam }) => getCharts(pageParam),
initialPageParam: 0,
getNextPageParam: (_, __, lastPage) => lastPage + 1,
});
return (
<>
<div className="grid sm:grid-cols-2 md:grid-cols-2 lg:grid-cols-3 gap-4">
{data?.pages.map((page, index) => (
<Fragment key={index}>
{page?.map((c) => (
<ChartCard key={c.id} id={c.id} type={c.type} name={c.name} />
))}
</Fragment>
))}
</div>
<div className="flex justify-center mt-4">
{hasNextPage && (
<button
className="bg-blue-500 hover:bg-blue-700 text-white py-2 px-4 rounded"
onClick={() => fetchNextPage()}
disabled={!isFetchingNextPage}
>
{isFetchingNextPage ? 'Loading more...' : 'Load More'}
</button>
)}
</div>
</>
);
}
This solution works well only until the user loads several pages. After that, the page performance degrades to almost the same level as without pagination.
Luckily, we still have a few tricks up our sleeves.
The next optimization we could implement is rather complex but typical for this problem: a virtual list with loading data on scroll. There are several good libraries we can use to implement virtual lists in React. The most popular by far are react-window and react-virtualized. We could also use Masonry component by Pinterest, it supports everything we need: virtual lists, loading data on scroll, rendering cards in grids. etc. But, perhaps the easiest option is to use TanStack's other great library @tanstack/react-virtual.
The idea is simple: we only render the charts that users see on the screen. This allows us to keep page performance at the same level, no matter how many charts we have loaded.
import { useRef, useEffect, useMemo } from 'react';
import { useInfiniteQuery } from '@tanstack/react-query';
import { useWindowVirtualizer } from '@tanstack/react-virtual';
import ChartCard from './ChartCard';
import { useWindowSize } from './useWindowSize';
import { getCharts } from '../api';
export default function ChartsVirtual() {
const listRef = useRef<HTMLDivElement | null>(null);
const { data, isFetchingNextPage, fetchNextPage, hasNextPage } = useInfiniteQuery({
queryKey: ['charts-paginated'],
queryFn: ({ pageParam }) => getCharts(pageParam),
initialPageParam: 0,
getNextPageParam: (_, __, lastPage) => lastPage + 1,
});
const rows = useMemo(() => data?.pages.flatMap((d) => d) ?? [], [data?.pages]);
const { width } = useWindowSize();
const cols = width < 640 ? 1 : width < 1024 ? 2 : 3;
const virtualizer = useWindowVirtualizer({
count: Math.ceil(rows.length / cols),
estimateSize: () => 350,
});
const virtualItems = virtualizer.getVirtualItems();
useEffect(() => {
const lastItem = [...virtualItems].pop();
if (
lastItem &&
(lastItem.index + 1) * cols >= rows.length - 1 &&
hasNextPage &&
!isFetchingNextPage
) {
fetchNextPage();
}
}, [
cols, hasNextPage, fetchNextPage, virtualItems,
rows.length, isFetchingNextPage,
]);
return (
<div ref={listRef}>
<div className="width-full relative" style={{ height: `${virtualizer.getTotalSize()}px` }}>
{virtualItems.map((vr) => (
<div
key={vr.key}
className="absolute top-0 left-0 w-full flex justify-start gap-4"
style={{ height: `${vr.size}px`, transform: `translateY(${vr.start}px)`}}
>
{Array.from({ length: cols }).map((_, i) => {
const c = rows[vr.index * cols + i];
return (
<div className="w-full" key={c?.id}>
{c && <ChartCard id={c.id} type={c.type} name={c.name} />}
</div>
);
})}
</div>
))}
</div>
</div>
);
}
Few things we should consider here:
useQuery({…, refetchOnMount: false })
in the card component to avoid repeated requests to the API.
Now, the feed page performance does not suffer even when there is a very long list of charts, with thousands of data points for each chart.
In theory, we could have ended our exercise here. But with our production data the performance sometimes slipped even with the virtual list and all other optimizations. So we've decided to try to solve this problem from the other end.
We realized that most charts were based on historical data, not real-time data. What we can do with that? The idea is super simple: Let’s pre-render previews for our charts on the backend and serve them to the client as SVG images.
With this approach, we can not only speed up the frontend but also dramatically simplify the codebase:
To illustrate the idea, let's add code to render SVG versions of graphs to the existing API backend.
import React from 'react';
import express from 'express';
import { renderToString } from 'react-dom/server';
import { LineChart, Line, XAxis, YAxis, CartesianGrid, BarChart, Bar, Area, AreaChart } from 'recharts';
function BarChartContainer({ data }) {
return (
<BarChart width={500} height={300} data={data}>
<CartesianGrid strokeDasharray="3 3" />
<XAxis dataKey="name" />
<YAxis />
<Bar dataKey="pv" fill="#42a5f6" isAnimationActive={false} />
<Bar dataKey="uv" fill="#ff9f40" isAnimationActive={false} />
</BarChart>
);
}
const components = {
line: LineChartContainer,
bar: BarChartContainer,
// etc..
};
export function ChartPreview({ data, type }) {
const Component = components[type];
return <Component data={data} />;
}
const app = express();
// other api methods
app.get('/api/charts/:id', async (req, res) => {
const data = getChartData(req.params.id);
const svg = renderToString(<ChartPreview type={req.query.type} data={data} />);
res.setHeader('Content-Type', 'image/svg+xml');
res.send(svg);
});
Since the backend code contains JSX, we must perform an additional build step that transforms the code into regular JS. For this purpose, we can use ESBuild.
"scripts": {
"api:build": "esbuild api/index.jsx --bundle --outfile=build/api.js --platform=node",
}
Now, all we are left to do is to fix the ChartCard
code on the frontend, replacing the chart rendering logic with an image that the backend API generates.
export default function ChartImgCard({ id, type }: { id: string, type: ChartType}) {
return (
<div className="shadow-md p-4 bg-white rounded-lg max-h-96">
<img src={`${process.env.BACKEND_URL}/api/charts-svg/${id}?type=${type}`} />
</div>
);
}
We can still use all the same performance optimization techniques on the front-end: virtual lists and lazy loading. But even without it, page performance has increased dramatically. The improvement is especially noticeable on weak devices.
This approach also has some disadvantages, which are worth mentioning:
We tackled a common challenge of implementing a feed with complex graphics, often encountered in work and system design interviews. Starting with a basic setup, we iterated through various optimization techniques to improve performance:
You can find the source code for this article in this GitHub repository.