Becoming fully reactive: an in-depth explanation of MobX

Due to popular demand (and to have a cool story for my grand-children), these are the inner workings of . A lot of people are surprised how consistent and fast MobX is. But rest assured, there is no magic in play! MobX First, let’s define the core concepts of MobX: . Any value that can be mutated and might serve as source for computed values is state. MobX can make most types of values (primitives, arrays, classes, objects, etc.) and even (potentially cyclic) references observable out of the box. Observable state Any value that can be computed by using a function that purely operates on other observable values. Computed values can range from the concatenation of a few strings up to deriving complex object graphs and visualizations. Because computed values are observable themselves, even the rendering of a complete user interface can be derived from the observable state. Computed values might evaluate either lazily or in reaction to state changes. Computed values. A reaction is a bit similar to a computed value, but instead of producing a new value it produces a side effect. Reactions bridge reactive and imperative programming for things like printing to the console, making network requests, incrementally updating the component tree to patch the DOM, etc. Reactions. React Actions are the primary means to modify the state. Actions are not a reaction to state changes but take sources of change, like user events or incoming web-socket connections, to modify the observable state. Actions. and are both referred to as in the remainder of this blog-post_._ So far, this might all sound a bit academic so let’s make it concrete! In a spreadsheet all data cells that have values would form the . Formulas and charts are that can be derived from the data cells and other formulas. Drawing the output of a data cell or a formula on the screen is a . Changing a data cell or formula is an . Computed values reactions derivations observable state computed values reaction action Anyway, here are all four concepts in a small example that uses MobX and React: Listing 1: Observable state, computed values, reactive Reactjs component and some actions We could draw a dependency tree based on the above listing. Intuitively it will look as follows: Figure 1: Dependency tree of profileView component. FullName is in reactive mode, actively observing firstName and lastName. The of this applications is captured in the properties (blue). The green fullName can be derived from the state automatically by observing the firstName and the lastName. Similarly the rendering of the profileView can be derived from the nickName and the fullName. The profileView will react to state changes by producing a side effect: it updates the React component tree. state observable computed value When using MobX the dependency tree is minimally defined. For example, as soon as the person being rendered has a nickname, the rendering will no longer be affected by the output of the fullName value, nor the first- or lastName (see listing 1). All observer relations between those values can be cleaned up and MobX will automatically simplify the dependency tree accordingly: Figure 2: Dependency tree of the profileView component if the user has a nickname (see listing 1). In contrast to figure 1, fullName is now in lazy mode and does not observe firstName and lastName MobX will always try to minimize the number of computations that are needed to produce a consistent state. In the rest of this blog post, I will describe several strategies used to achieve this goal. But before diving into the magic of how computed values and reactions are kept in sync with the state, let’s first describe the principle behind MobX: Reacting to state changes is always better then acting on state changes. Any imperative action that an application takes in response to a state change usually creates or updates some values. In other words, most actions manage a . Triggering the user interface to update? Updating aggregated values? Notifying the back-end? These can all be thought of as cache invalidations in disguise. To ensure these caches will stay in sync, you need to subscribe to future state changes that will enable your actions to be triggered again. local cache But working with subscriptions (or cursors, lenses, selectors, connectors, etc) has a fundamental problem: as your app evolves, you will make mistakes in managing those subscriptions and either oversubscribe (continue subscribing to a value or store that is no longer used in a component) or undersubscribe (forgetting to listen for updates leading to subtle staleness bugs). In other words; when using manual subscriptions, your app will eventually be inconsistent. Figure 3: Inconsistent Twitter page after updating profile. The pinned tweet is displaying stale values for both the name and the profile picture of the author. The above image is a nice example of the Twitter UI being inconsistent. As explained in my , there can only be two causes for this: Either there is no subscription that tells tweets to re-render if the profile of the associated author has changed. Or the data was normalized and the author of a tweet doesn’t even relate to the profile of the currently logged-in user, despite the fact that both pieces of data try to describe the same properties of the same person. Reactive2015 talk Coarse grained subscriptions like Flux-style store subscriptions are very susceptible to oversubscribing. When using React, you can simply tell whether your components are oversubscribing by printing . MobX will The idea is simple yet counterintuitive: More subscriptions result in fewer recomputations. MobX manages many thousands of observers for you. You can effectively tradeoff memory for CPU cycles. wasted renderings reduce this number to zero. Note that oversubscribing also exists in very subtle forms. If you subscribe to data that is used, but not under conditions, you are still oversubscribing. For example, if the profileView component subscribes to the fullName of a person that has a nickName, it is oversubscribing (see listing 1). So an important principle behind the design MobX is: all A minimal, consistent set of subscriptions can only be achieved if subscriptions are determined at run-time. The second important idea behind MobX is that for any app that is more complex than TodoMVC, you will often need a , instead of a normalized tree, to store the state in a yet optimal way. Graphs enable referential consistency and avoid data duplication so that it can be guaranteed that derived values are never stale. data graph mentally manageable How MobX keeps all derivations efficiently in a consistent state The solution: don’t cache, derive instead. People ask: “isn’t that extremely expensive?” No, it is actually very efficient! The reason for that is, as explained above: MobX doesn’t run all derivations, but ensures that only that are involved in some are kept in sync with the observable state. Those derivations are called to be . To draw the parallel with spreadsheets again: only those formulas that are currently visible or that are used indirectly by a visible formula, need to re-compute when one of the observed data cells change. computed values reaction reactive Lazy versus reactive evaluation So what about computations that aren’t used directly or indirectly by a reaction? You can still inspect the value of a computed value like fullName at any time. The solution is simple: if a computed value is not reactive, it will be evaluated on demand (lazily), just like a normal getter function. Lazy derivations (which never observe anything) can simply be garbage collected if they run out of scope. Remember that computed values should always be of the observable app state? This is the reason why: For pure functions it doesn’t matter whether they are evaluated lazily or eagerly; the evaluation of the function always yields the same result given the same observable state. pure functions Running computations Reactions and computed values are both run by MobX in the same manner. When a recomputation is triggered the function is pushed onto the ; a function stack of currently running derivations. As long as a computation is running, every observable that is accessed will register itself as a dependency of the topmost function of the derivation stack. If the value of a computed value is needed, the value can simply be the last known value if the computed value is already in the reactive state. And otherwise it will push itself on the derivation-stack, switch to reactive mode and start computing as well. derivation stack Figure 4: During the execution of the profileView reaction some observable state and some computed values are being observed. Computed values might recompute, this results in the the dependency tree as shown in figure 1. When a computation completes, it will have obtained a list of observables that were accessed during execution. In the profileView for example, this list will either just contain the nickName property, or the nickName and fullName properties. This list is diffed against the previous list of observables. Any removed items will be unobserved (computed values might go back from reactive to lazy mode at this point) and any added observables will be observed until the next computation. When the value of for example firstname is changed in the future, it knows that fullName needs to be recomputed. Which in turn will cause profile view to recomputed. The next paragraph explains this process in more detail. Propagating state changes Figure 5: The effects of changing value ‘1’ on the dependency tree. The dashed border indicates the observers that will be marked as stale. Numbers indicate the order of computation. Derivations will react to state changes automatically. All reactions happen and more importantly . When an observable value is modified the following algorithm is performed: synchronously glitch-free The observable value sends a to all its observers to indicate that it has become stale. Any affected computed values will recursively pass on the notification to their observers. As a result, a part of the dependency tree will be marked as . In the example dependency tree of figure 5, the observers that will become stale when value ‘1’ is changed are marked with an orange, dashed border. These are all the derivations that be affected by the changing value. stale notification stale might After sending the and storing the new value, a will be sent. This message also indicates whether the value did actually change. stale notification ready notification As soon as a derivation has received a ready notification for every stale notification received in step 1, it knows that all the observed values are stable and it will start to recompute. Counting the number of ready / stale messages will ensure that, for example, computed value ‘4’ will only re-evaluate after computed value ‘3’ has become stable. If none of the ready messages indicate that a value was changed, the derivation will simply tell its own observers that it is ready again, but without changing its value. Otherwise the computation will recompute and send a ready message to its own observers. This results in the order of execution as displayed in figure 5. Note that (for example) the last reaction (marked with ‘-’) will never execute if computed value ‘4’ did re-evaluate but didn’t produce a new value. The previous two paragraph summarize how dependencies between observable values and derivations are tracked at run-time and how changes are propagated through the derivations. At this point you might also realize that a is basically a that is always in mode. It is important to realize that this algorithm can be implemented very efficiently without closures and just with a bunch of pointer arrays. Additionally, MobX applies a number of other optimizations which are beyond the scope of this blog post. reaction computed value reactive Synchronous execution People are often surprised that MobX runs everything synchronously (like RxJs and unlike knockout). This has two big advantages: First it becomes simply impossible to ever observe stale derivations. So a derived value can be used immediately after changing a value that influences it. Secondly it makes stack-traces and debugging easier as it avoids the useless stack-traces that are typical to Promise / async libraries. Listing 2: Example transaction. It ensures that nobody is able to observe an intermediate value like “Mich Weststrate”. (See also listing 1) However, synchronous execution also introduces the need for transactions. If several mutations are applied in immediate succession, it is preferable to re-evaluate all derivations after all changes has been applied. Wrapping an action in a block achieves this. Transactions simply postpone all until the transaction block has completed. Note that transactions still runs and updates everything synchronously. transaction ready notifications That summarizes the most essential implementation details of MobX. We haven’t covered everything yet, but it is good to know for example that you can compose computed values. By it is even possible to automatically transform one graph of data into another graph of data and keep this derivation up to date with the minimum number of patches. This makes it easy to implement complex patterns like map-reduce, state tracking using immutable shared data, or sideways data loading. But more on that in a next blog post. composing reactive computations TL;DR: The application state of complex applications can best be expressed using to achieve referential consistency and stay close to the mental model of a problem domain. graphs One should not imperatively act on state changes by using manually defined subscriptions or cursors. This will inevitably lead to bugs as a result of under- or oversubscribing. Use to determine the of observer → observable relationships. This leads to a computational model where it can be guaranteed that the minimum amount of derivations are run without ever observing a stale value. runtime analysis smallest possible set Any derivation that is not needed to achieve an active side effect can be optimized away completely. For more info on MobX, just check out: My talk at Reactive2015 The or official website repo A boilerplate project: , , or the React + Babel React + TypeScript TodoMVC Flux-Challenge Edit 2–3–2016: MobX was called Mobservable before version 2.0 Image by xt0ph3r