Open Sourcing a Redux Component

This is the story all about how I built (with the incalculable help of ) a React/Redux Grid Component, appropriately (and boringly named) . taco React-Redux-Grid This is about how we started building a thing, how it changed along the way, the things we messed up, the things we learned, the things we’d do differently, and the things we’re going to do going forward. Starting to Build a Thing At the time, I was working for an organization that was revamping their UI stack from Ext JS to a more modern platform based on . Without going into too much back story, our existing app was a monolith with dozens of views, hundreds of rich UI components, and a bunch of spaghetti holding together our concept of (more on this later). In short, we had a bunch to rebuild (exciting!), refactor (hard!), and reimagine. One of the big obstacles for us was that we needed a robust grid component since much of our application was built around displaying and interacting with tabular data. Although we did our research, we couldn’t find an extensible enough grid that suited our UI and data needs. I will also admit that we had a penchant for creating things early on, since our previous applications heavily involved extending EXT’s components rather than building our own, and we were feeling bitten by that mantra. React state management Very early, we knew we were going to commit to React. The community was behind it, it fit our functional tastes, and the API was clean and expressive. We got a little lucky in choosing , though we toyed with and , finally choosing Redux because the single-store metaphor made sense to us. Redux Flux Nuclear I didn’t stop to think why our UI Components needed to be via Redux. Now, of course, I fully understand the convenience of having component state accessible via the `store`. It’s pretty cumbersome having to jump through hoops, retrieving React state from the component instance. But honestly I don’t think I thought about that at the time. I think I just wanted a Redux Component because it was an interesting problem. connected Anyways, here’s the short list of high-level requirements we came up with prior to building: * Handle large data sets* Open-ended event model, where DOM interactions can correspond to certain callbacks* Each row needs to be able to trigger dynamic menus and actions* Support a flat list or tree structure* Resizable/Orderable columns* Custom rendering models for cells* Editable rows, validation, persistence* Tested. Fully tested.* Fast. Real fast.* Extensible/Modular Style Although we ran into a bunch of problems immediately, I’m only going to touch on the most interesting ones: The `DOM` Problem Now we all know that React renders things as a tree — which is great. Well it’s usually great. Take a look at this grid. What do we notice? We have a header and some rows. Simple right? Well, yes, and no. The problem here is that we have a scrollable container with a bunch of rows, and a disparate header that should be in sync with all of these rows. What’s the big deal you say? Well let’s look at the hypothetical DOM structure this presents: …headers …more rows I’m sure you’ve guessed by now that this just isn’t going to work with React. If we tried this DOM, and believe me, we did, we’d get the following error: Warning: validateDOMNesting(…): cannot appear as a child of Which makes sense, since yeah, that’s pretty gross HTML. I’d also like to note that making a scroll nicely seems nearly impossible. But since we needed to be able to resize columns, and we didn’t want to have to rely on to keep our headers and our cells aligned (because of overall expense), we needed a new solution. We came upon the following solution because although React makes creating invalid impossible, it makes rendering HTML super cheap and efficient: JavaScript HTML …headers …copy of above headers …more rows We end up with a lot more DOM here, but we get something pretty powerful as well. Although we have two HTML copies of our Header component, they are actually the exact same React components (so no extra code required) — being driven by the same slice of Redux state (no extra state required). This was incredibly helpful. When one of the visible headers is resized, although it doesn’t update the corresponding cells automatically, it update the Redux store with a new width prop. That width is also driving the invisible header’s width, which then triggers the browser to resize the corresponding ’s in the scrollable container. does And , we have two disparate tables communicating via Redux state. We are an action when header actions occur, but we don’t need to keep track of cell width (which would be painful and expensive). Viola dispatching This was one of the first “ah-ha” moments for me, working with React and Redux. I kept running into problems I had solved before with things like jQuery or Angular that React made a little more difficult (like the invalid DOM problem). But when it came to core view-centric problems like having to efficiently update the DOM, or sensibly managing state, React simply made those afterthoughts. I really didn’t need to care at all, so I didn’t. The ‘Making it Fast’ Problem React is fast, but there’s always ways to optimize for speed. Although we went through a phase, where we nailed down each component so they only redrew when needed, I’m going to talk about how we optimized for our problem space. shouldComponentUpdate connect is a decorator exposed by Redux that essentially hooks a component up to the Redux store so that it can read out of the state atom. There is a bunch more to it (that’s incredibly important for performance optimization) but for our purposes, that’s probably enough. The only other thing I’ll say is that we incur a bit more overhead since all connected components are “listening” for state changes, and must run through anytime the store gets updated. Connect mapStateToProps We can see from our final DOM structure, that we’re going to have a bunch of components. The problem that grid poses more than other UI components is scale. For example, in our app we often had instances of thousands of records on a single page, each record with dozens of properties we wanted to display. That’s of DOM, but it’s also a lot of event listeners, and JavaScript objects in memory, etc. a lot We started out by connecting everything — hey, everything needed to read from the store! That meant our header, the scroll-container, the footer, every row, and yes, every single cell. It worked — kind of. Sure it rendered, but interactions were too sluggish and redraws weren’t 60 fps crisp. We needed to do something different. That’s all the details on the problem for now, since I’m going to discuss this in the “Things We Messed Up” section. But I will just say that our final solution (which maybe we should rethink) is to only have a single connected component — the grid container. That way we only run through once, and we pass our props all the way down the chain, letting handle the rest. This resulted is some untenable code at times, but a lightning-quick reconciliation/render cycle. mapStateToProps shouldComponentUpdate The ‘What do we Export’ Problem That brings us to a non-technical problem. At the time of inception, building a “Redux-Component” was a relatively new idea. There were React components that exposed a single export — the component itself. Sometimes they bundle CSS dynamically, and sometimes you needed to import a stylesheet manually. But Redux is a little different because it brings the added dependencies of a store, reducers, and actions. Did these need to be exported? We didn’t know. We guessed they did, but to be honest, we weren’t sure until we started using the component ourselves. We ended up exporting everything — including the demo store. We wanted the grid to run on its own so people could `git clone` and see how it worked themselves, kind of like a UI REPL. But we didn’t want to impose any pattern beyond the simplest usage. It’s via that thinking that we came up the following exports: export const modules = {Actions,Grid,Reducers,applyGridConfig,Store}; It’s obvious to us now that the actions were a necessary export since they turned out to be the public API for the grid — how you communicate information to the component via . We also exposed a number of reducers, each responsible for their own slice of the grid state (i.e. data, columns, editor, pagination). In my mind, this made dealing with grid as heavy or as light as the importing developer wanted. But now I sort of feel like that’s one of the more annoying parts of the component. It makes for cleaner code inside grid, but for a more annoying API. Oh well, so it goes. dispatch We also exported a couple utilities, and this is the area where I think we will spend our near term efforts — building some more utilities that make working with grid a little simpler. We decided to include CSS dynamically at runtime, but to make it optional. By default the CSS will be appended to the document, but can ignored by setting a property via the function. One of the simple joys for me, in this new brave React world, is that components can truly be self-sufficient and sandboxed — it always irked me that I needed to include CSS via tags with jQuery components. applyGridConfig Things We Messed Up This section could and should be the longest part of this story. But it’s not going to be, probably out of self-preservation. Using ImmutableJS the Wrong Way This brings us back to . Since we were worried about the connection cost, we simply reduced the number of decorated components we were authoring — to 1. This got us nearly the whole way, except that we saw big lags in performance when the data-set got large (1000+), and that didn’t sit well with us. connect We starting diagnosing the problem using tools and saw that we were spending almost no time in wasted cycles. Here’s the readout from React-Perf: React Perf What, we were only wasting 1.8ms on a grid with 10,000 rows? We weren’t needlessly recalculating props for components, and we weren’t needlessly redrawing either. So what was the problem? We opened up the browser profiler ( is first rate, if you’ve never used it), and we recorded a session where we performed a couple click events. We experienced the heavy lags we were consistently seeing with large data sets, so we went to profile logs to see what was going on. Chrome’s Nearly of the JavaScript runtime was being spent in garbage collection. And even more alarming, nearly of the time was spent in ’ function? For a couple single click events? 25% 8% ImmutableJS toJS When I think about this now, it’s so obvious that this was due to our poor implementation of which involved heavy deserialization of immutable data structures to plain JavaScript objects. However, back then, I remember this blowing my hair back. I remember thinking ‘what the hell kind of memory leak required 15% of our execution memory?’. And ‘what the hell is wrong with toJS?’. Here’s a simplified version of what was going on: mapStateToProps export const mapStateToProps = (state, props) => ({gridData: stateGetter(state, ‘dataSource’, props.stateKey),}); export const stateGetter = (state, key, gridStateKey) => state.getIn([key, gridStateKey]).toJS(); For each property that we cared about, whether it was columns, or data, we were converting our immutable data (which was being stored in our reducers) to plain JavaScript objects in since the innards of grid were operating on native objects instead of immutable ones (some kind of feeble attempt at separation of concerns). mapStateToProps We knew, even then, that was going to be expensive, but what we didn’t know was that runs even before thanks to Redux. So anytime an action fired, we recursively toJS()ed our entire state node for the grid. This explained why we saw the problem grow linearly as the data sets got larger. It also explained the crazy amount of garbage collection, since the execution of didn’t even do anything with the newly created native objects — specifically when an event didn’t require a redraw, those fresh new objects were simply thrown away. toJS mapStateToProps shouldComponentUpdate mapStateToProps The solution was simple after we figured out where and what the problem was. We simply removed the calls and made the innards of grid work with immutable data. This is where we got the biggest performance boost. Since this change, we haven’t run into any real performance snags at all. Looking back now, I think this was my most egregious error, causing me to change the way I thought about open source libraries as a whole (more on that below). toJS Requiring a `store` Prop This has since been remedied, but up until version 5.0.0, a Redux store was one of the required props for grid. It simply wouldn’t run unless the store was explicitly provided. I chalk this one up to not analyzing the landscape as grid matured. We ended up this very easily by simply pulling the store from , the pattern many other popular Redux frameworks had already implemented. solving context Things we Learned Along the Way Maybe this should be the longest section? No probably not, we messed up a lot along the way. Take a look around Although we were venturing out into unknown territory (especially myself), every time we didn’t know the solution to a problem, we were able to peruse other popular GitHub repos to see how something was accomplished, or what pattern was being employed. This is something I wish I would have done at every step of the way. A minute of research can save you weeks of refactoring. Open Source is a lot of work And it’s worth it. But open source projects don’t get awesome until you have a bunch of people all working together. I can’t tell you how exciting it is to have people you don’t know (and without provocation) collaborating on a project. Some of the most creative and sweeping changes come from having a new set of eyes on an old problem. If you don’t want to, or you don’t know where to start, or ask a question. Submit a Pull Request. open an Issue Testing pays dividends Over the course of grid’s lifetime, its undergone 3 heavy refactors. Removing connect, and modifying components to use immutable objects required nearly full rewrites of much of the source. Because we were stalwarts in the beginning about adding tests, and checking coverage, these refactors were made easier, and felt less burdensome. It was a really great feeling completing a refactor and watching your tests pass. Even better, when a couple failed for good reason, corrections were made, and we could feel confident about what we had built. Understand the tools you use I’ve always thought about libraries as black boxes. If I needed a utility/component/thing, and it’s been built before, why recreate the wheel right? Sure, you learn the API and you use the tool to solve the problem at hand. But beyond that, I generally didn’t go look inside to see how that problem is solved. And that bit me here — I had a fundamental misunderstanding of how Immutable worked, and that caused a lot of pain and refactoring. If I’ve learned anything, it’s that before choosing your tools, take a moment to see how they work, if they’re truly right for the job, and you implement them correctly. make sure And Now to the Future Here’s a list of things we’d like to accomplish in the near to long future, in no particular order: * Implement selectors* Addition of new UI plugins (row expansion, etc)* Add more extensibility around filtering data* More fleshed out examples for the demo site* Implement drag and drop for flat data* Better documentation Please open a pull request and accomplish one of these goals! Or come up with a better idea! Or go fix some bugs, or refactor some of the gross away. At the time of writing, grid is at v5.1.8. We’ve come a long way, but we’d like to go a lot further. is how hackers start their afternoons. We’re a part of the family. We are now and happy to opportunities. Hacker Noon @AMI accepting submissions discuss advertising &sponsorship To learn more, , , or simply, read our about page like/message us on Facebook tweet/DM @HackerNoon. If you enjoyed this story, we recommend reading our and . Until next time, don’t take the realities of the world for granted! latest tech stories trending tech stories