Open Sourcing a Redux Component

This is the story all about how I built (with the incalculable help of taco) a React/Redux Grid Component, appropriately (and boringly named) React-Redux-Grid.

This is about how we started building a thing, how it changed along the way, the things we messed up, the things we learned, the things we’d do differently, and the things we’re going to do going forward.

Starting to Build a Thing

At the time, I was working for an organization that was revamping their UI stack from Ext JS to a more modern platform based on React. Without going into too much back story, our existing app was a monolith with dozens of views, hundreds of rich UI components, and a bunch of spaghetti holding together our concept of state management (more on this later). In short, we had a bunch to rebuild (exciting!), refactor (hard!), and reimagine. One of the big obstacles for us was that we needed a robust grid component since much of our application was built around displaying and interacting with tabular data. Although we did our research, we couldn’t find an extensible enough grid that suited our UI and data needs. I will also admit that we had a penchant for creating things early on, since our previous applications heavily involved extending EXT’s components rather than building our own, and we were feeling bitten by that mantra.

Very early, we knew we were going to commit to React. The community was behind it, it fit our functional tastes, and the API was clean and expressive. We got a little lucky in choosing Redux, though we toyed with Flux and Nuclear, finally choosing Redux because the single-store metaphor made sense to us.

I didn’t stop to think why our UI Components needed to be connected via Redux. Now, of course, I fully understand the convenience of having component state accessible via the `store`. It’s pretty cumbersome having to jump through hoops, retrieving React state from the component instance. But honestly I don’t think I thought about that at the time. I think I just wanted a Redux Component because it was an interesting problem.

Anyways, here’s the short list of high-level requirements we came up with prior to building:

* Handle large data sets* Open-ended event model, where DOM interactions can correspond to certain callbacks* Each row needs to be able to trigger dynamic menus and actions* Support a flat list or tree structure* Resizable/Orderable columns* Custom rendering models for cells* Editable rows, validation, persistence* Tested. Fully tested.* Fast. Real fast.* Extensible/Modular Style

Although we ran into a bunch of problems immediately, I’m only going to touch on the most interesting ones:

The `DOM` Problem

Now we all know that React renders things as a tree — which is great. Well it’s usually great. Take a look at this grid.

What do we notice? We have a header and some rows. Simple right? Well, yes, and no. The problem here is that we have a scrollable container with a bunch of rows, and a disparate header that should be in sync with all of these rows.

What’s the big deal you say? Well let’s look at the hypothetical DOM structure this presents:

<table><thead> …headers </thead><div style=”overflow-y:auto”><tbody><tr></tr>…more rows</tbody></div></table>

I’m sure you’ve guessed by now that this just isn’t going to work with React. If we tried this DOM, and believe me, we did, we’d get the following error:

Warning: validateDOMNesting(…): <tbody> cannot appear as a child of <div>

Which makes sense, since yeah, that’s pretty gross HTML. I’d also like to note that making a <tbody> scroll nicely seems nearly impossible. But since we needed to be able to resize columns, and we didn’t want to have to rely on JavaScript to keep our headers and our cells aligned (because of overall expense), we needed a new solution. We came upon the following solution because although React makes creating invalid HTML impossible, it makes rendering HTML super cheap and efficient:

<div><!--overall component container -><table><!--table for just our visible headers--><thead>…headers</thead></table><div style=”overflow-y:auto”><table><thead style=”visbility:hidden;”>…copy of above headers</thead> <!-- invisible headers --><tbody><tr></tr>…more rows</tbody></table></div></div>

We end up with a lot more DOM here, but we get something pretty powerful as well. Although we have two HTML copies of our Header component, they are actually the exact same React components (so no extra code required) — being driven by the same slice of Redux state (no extra state required). This was incredibly helpful. When one of the visible headers is resized, although it doesn’t update the corresponding cells automatically, it does update the Redux store with a new width prop. That width is also driving the invisible header’s width, which then triggers the browser to resize the corresponding <td>’s in the scrollable container.

And Viola, we have two disparate tables communicating via Redux state. We are dispatching an action when header actions occur, but we don’t need to keep track of cell width (which would be painful and expensive).

This was one of the first “ah-ha” moments for me, working with React and Redux. I kept running into problems I had solved before with things like jQuery or Angular that React made a little more difficult (like the invalid DOM problem). But when it came to core view-centric problems like having to efficiently update the DOM, or sensibly managing state, React simply made those afterthoughts. I really didn’t need to care at all, so I didn’t.

The ‘Making it Fast’ Problem

React is fast, but there’s always ways to optimize for speed. Although we went through a shouldComponentUpdate phase, where we nailed down each component so they only redrew when needed, I’m going to talk about how we optimized connect for our problem space.

Connect is a decorator exposed by Redux that essentially hooks a component up to the Redux store so that it can read out of the state atom. There is a bunch more to it (that’s incredibly important for performance optimization) but for our purposes, that’s probably enough. The only other thing I’ll say is that we incur a bit more overhead since all connected components are “listening” for state changes, and must run through mapStateToProps anytime the store gets updated.

We can see from our final DOM structure, that we’re going to have a bunch of components. The problem that grid poses more than other UI components is scale. For example, in our app we often had instances of thousands of records on a single page, each record with dozens of properties we wanted to display. That’s a lot of DOM, but it’s also a lot of event listeners, and JavaScript objects in memory, etc.

We started out by connecting everything — hey, everything needed to read from the store! That meant our header, the scroll-container, the footer, every row, and yes, every single cell. It worked — kind of. Sure it rendered, but interactions were too sluggish and redraws weren’t 60 fps crisp. We needed to do something different.

That’s all the details on the problem for now, since I’m going to discuss this in the “Things We Messed Up” section. But I will just say that our final solution (which maybe we should rethink) is to only have a single connected component — the grid container. That way we only run through mapStateToProps once, and we pass our props all the way down the chain, letting shouldComponentUpdate handle the rest. This resulted is some untenable code at times, but a lightning-quick reconciliation/render cycle.

The ‘What do we Export’ Problem

That brings us to a non-technical problem. At the time of inception, building a “Redux-Component” was a relatively new idea. There were React components that exposed a single export — the component itself. Sometimes they bundle CSS dynamically, and sometimes you needed to import a stylesheet manually.

But Redux is a little different because it brings the added dependencies of a store, reducers, and actions. Did these need to be exported? We didn’t know. We guessed they did, but to be honest, we weren’t sure until we started using the component ourselves.

We ended up exporting everything — including the demo store. We wanted the grid to run on its own so people could `git clone` and see how it worked themselves, kind of like a UI REPL. But we didn’t want to impose any pattern beyond the simplest usage. It’s via that thinking that we came up the following exports:

export const modules = {Actions,Grid,Reducers,applyGridConfig,Store};

It’s obvious to us now that the actions were a necessary export since they turned out to be the public API for the grid — how you communicate information to the component via dispatch. We also exposed a number of reducers, each responsible for their own slice of the grid state (i.e. data, columns, editor, pagination). In my mind, this made dealing with grid as heavy or as light as the importing developer wanted. But now I sort of feel like that’s one of the more annoying parts of the component. It makes for cleaner code inside grid, but for a more annoying API. Oh well, so it goes.

We also exported a couple utilities, and this is the area where I think we will spend our near term efforts — building some more utilities that make working with grid a little simpler.

We decided to include CSS dynamically at runtime, but to make it optional. By default the CSS will be appended to the document, but can ignored by setting a property via the applyGridConfig function. One of the simple joys for me, in this new brave React world, is that components can truly be self-sufficient and sandboxed — it always irked me that I needed to include CSS via <link rel> tags with jQuery components.

Things We Messed Up

This section could and should be the longest part of this story. But it’s not going to be, probably out of self-preservation.

Using ImmutableJS the Wrong Way

This brings us back to connect. Since we were worried about the connection cost, we simply reduced the number of decorated components we were authoring — to 1. This got us nearly the whole way, except that we saw big lags in performance when the data-set got large (1000+), and that didn’t sit well with us.

We starting diagnosing the problem using React Perf tools and saw that we were spending almost no time in wasted cycles. Here’s the readout from React-Perf:

What, we were only wasting 1.8ms on a grid with 10,000 rows? We weren’t needlessly recalculating props for components, and we weren’t needlessly redrawing either. So what was the problem?

We opened up the browser profiler (Chrome’s is first rate, if you’ve never used it), and we recorded a session where we performed a couple click events. We experienced the heavy lags we were consistently seeing with large data sets, so we went to profile logs to see what was going on.

Nearly 25% of the JavaScript runtime was being spent in garbage collection. And even more alarming, nearly 8% of the time was spent in ImmutableJS’ toJS function? For a couple single click events?

When I think about this now, it’s so obvious that this was due to our poor implementation of mapStateToProps which involved heavy deserialization of immutable data structures to plain JavaScript objects. However, back then, I remember this blowing my hair back. I remember thinking ‘what the hell kind of memory leak required 15% of our execution memory?’. And ‘what the hell is wrong with toJS?’. Here’s a simplified version of what was going on:

export const mapStateToProps = (state, props) => ({gridData: stateGetter(state, ‘dataSource’, props.stateKey),});

export const stateGetter = (state, key, gridStateKey) => state.getIn([key, gridStateKey]).toJS();

For each property that we cared about, whether it was columns, or data, we were converting our immutable data (which was being stored in our reducers) to plain JavaScript objects in mapStateToProps since the innards of grid were operating on native objects instead of immutable ones (some kind of feeble attempt at separation of concerns).

We knew, even then, that toJS was going to be expensive, but what we didn’t know was that mapStateToProps runs even before shouldComponentUpdate thanks to Redux. So anytime an action fired, we recursively toJS()ed our entire state node for the grid. This explained why we saw the problem grow linearly as the data sets got larger. It also explained the crazy amount of garbage collection, since the execution of mapStateToProps didn’t even do anything with the newly created native objects — specifically when an event didn’t require a redraw, those fresh new objects were simply thrown away.

The solution was simple after we figured out where and what the problem was. We simply removed the toJS calls and made the innards of grid work with immutable data. This is where we got the biggest performance boost. Since this change, we haven’t run into any real performance snags at all. Looking back now, I think this was my most egregious error, causing me to change the way I thought about open source libraries as a whole (more on that below).

Requiring a `store` Prop

This has since been remedied, but up until version 5.0.0, a Redux store was one of the required props for grid. It simply wouldn’t run unless the store was explicitly provided. I chalk this one up to not analyzing the landscape as grid matured. We ended up solving this very easily by simply pulling the store from context, the pattern many other popular Redux frameworks had already implemented.

Things we Learned Along the Way

Maybe this should be the longest section? No probably not, we messed up a lot along the way.

Take a look around

Although we were venturing out into unknown territory (especially myself), every time we didn’t know the solution to a problem, we were able to peruse other popular GitHub repos to see how something was accomplished, or what pattern was being employed. This is something I wish I would have done at every step of the way. A minute of research can save you weeks of refactoring.

Open Source is a lot of work

And it’s worth it. But open source projects don’t get awesome until you have a bunch of people all working together. I can’t tell you how exciting it is to have people you don’t know (and without provocation) collaborating on a project. Some of the most creative and sweeping changes come from having a new set of eyes on an old problem.

Submit a Pull Request. If you don’t want to, or you don’t know where to start, open an Issue or ask a question.

Testing pays dividends

Over the course of grid’s lifetime, its undergone 3 heavy refactors. Removing connect, and modifying components to use immutable objects required nearly full rewrites of much of the source. Because we were stalwarts in the beginning about adding tests, and checking coverage, these refactors were made easier, and felt less burdensome. It was a really great feeling completing a refactor and watching your tests pass. Even better, when a couple failed for good reason, corrections were made, and we could feel confident about what we had built.

Understand the tools you use

I’ve always thought about libraries as black boxes. If I needed a utility/component/thing, and it’s been built before, why recreate the wheel right? Sure, you learn the API and you use the tool to solve the problem at hand. But beyond that, I generally didn’t go look inside to see how that problem is solved. And that bit me here — I had a fundamental misunderstanding of how Immutable worked, and that caused a lot of pain and refactoring. If I’ve learned anything, it’s that before choosing your tools, take a moment to see how they work, if they’re truly right for the job, and make sure you implement them correctly.

And Now to the Future

Here’s a list of things we’d like to accomplish in the near to long future, in no particular order:

* Implement selectors* Addition of new UI plugins (row expansion, etc)* Add more extensibility around filtering data* More fleshed out examples for the demo site* Implement drag and drop for flat data* Better documentation

Please open a pull request and accomplish one of these goals! Or come up with a better idea! Or go fix some bugs, or refactor some of the gross away.

At the time of writing, grid is at v5.1.8. We’ve come a long way, but we’d like to go a lot further.

Hacker Noon is how hackers start their afternoons. We’re a part of the @AMIfamily. We are now accepting submissions and happy to discuss advertising &sponsorship opportunities.

To learn more, read our about page, like/message us on Facebook, or simply, tweet/DM @HackerNoon.

If you enjoyed this story, we recommend reading our latest tech stories and trending tech stories. Until next time, don’t take the realities of the world for granted!