For a very long time, I was a big fan and advocate of GraphQL's and directives. Actually, I've implemented them in my own GraphQL server implementation in Go . @defer @stream almost 3 years ago At that time, I was using a stream of JSON-Patch operations to continuously update the client. Recently, I've added support for TypeScript Operations to WunderGraph, including support for Subscriptions through Async Generators. After implementing this feature and playing around with it, I've realized that and are complete overkill, and the whole idea of incrementally loading data can be done in a much simpler way. @defer @stream Before we can compare the two approaches, we need to understand how and work. @defer @stream How the @defer and @stream Directives Work I'm using the Example from the . DeferStream RFC query { person(id: "cGVvcGxlOjE=") { ...HomeWorldFragment @defer(label: "homeWorldDefer") name films @stream(initialCount: 2, label: "filmsStream") { title } } } fragment HomeWorldFragment on Person { homeworld { name } } We're loading the person with the ID . cGVvcGxlOjE= For this person, we're loading only the name and the first two films synchronously. The is loaded asynchronously, and all films beyond the first two are loaded asynchronously. HomeWorldFragment From the example RFC, we get back the following responses: { "data": { "person": { "name": "Luke Skywalker", "films": [ { "title": "A New Hope" }, { "title": "The Empire Strikes Back" } ] } }, "hasNext": true } The first response is as expected; the only notable thing is the field which indicates that there are more responses to come. hasNext { "label": "homeWorldDefer", "path": ["person"], "data": { "homeworld": { "name": "Tatooine" } }, "hasNext": true } The second response is the which is loaded asynchronously. HomeWorldFragment The field is used to identify the response. The field is used to identify the location in the response where the data should be inserted. label path The field indicates that there are still more responses to come. hasNext { "label": "filmsStream", "path": ["person", "films", 2], "data": { "title": "Return of the Jedi" }, "hasNext": false } The third response is the first film beyond the first two films. The field is used to identify the location in the response where the data should be inserted. The field indicates that there are no more responses to come. path hasNext According to the RFC, this approach has the following : benefits Make GraphQL a great choice for applications that demand responsiveness. Enable interoperability between different GraphQL clients and servers without restricting implementation. Enable a strong tooling ecosystem (including GraphiQL). Provide concrete guidance to implementers. Provide guidance to developers evaluating whether to adopt incremental delivery. At the same time, the RFC also lists the following : “Type Generation Supporting - @defer can add complexity to type-generating clients. Separate types will need to be generated for the different deferred fragments. Caveats These clients will need to use the label field to determine which fragments have been fulfilled to ensure the application is using the correct types. Object Consistency - The GraphQL spec does not currently support object identification or consistency. It is currently possible for the same object to be returned in multiple places in a query. If that object changes while the resolvers are running, the query could return inconsistent results. @defer/@stream does not increase the likelihood of this, as the server still attempts to resolve everything as fast as it can. The only difference is some results can be returned to the client sooner. This proposal does not attempt to address this issue. Can @defer/@stream increase risk of a denial of service attack? This is currently a risk in GraphQL servers that do not implement any kind of query limiting as arbitrarily complex queries can be sent. Adding @defer may add some overhead as the server will now send parts of the query earlier than it would have without @defer, but it does not allow for any additional resolving that was not previously possible.” Why the @defer and @stream Directives Are Overkill First of all, I want to say that I love the idea of incrementally loading data, and I have a lot of respect for the people who came up with this idea and participated in this or any other RFC that has something to do with this topic. So please, don't take this as a personal attack on anyone. I don't criticize your work; I just want to point out that there is a much simpler way to achieve the same goal. So what follows now is my personal opinion, which is based on years of experience working with and implementing GraphQL as well as building WunderGraph. First of all, this RFC has been in the making for many years without getting merged into the GraphQL spec. As many library and framework authors were quite excited about this feature, they implemented it in their own way. So there are now many implementations of and out there. @defer @stream This is a nightmare because how should clients know which implementation the server is using? Additionally, the RFC is quite complex and requires a lot of work to implement. All client and server implementations have to implement handling of and directives. @defer @stream It's a lot of work to implement this feature in a reasonable way. Let me give you an example from GraphQL Yoga. Again, not criticizing the work of the GraphQL Yoga team, just trying to point out the complexity of this feature. import { createYoga, createSchema, Repeater } from 'graphql-yoga' import { createServer } from 'node:http' import { useDeferStream } from '@graphql-yoga/plugin-defer-stream' const yoga = createYoga({ schema: createSchema({ typeDefs: /* GraphQL */ ` type Query { alphabet: [String!]! } `, resolvers: { Query: { alphabet: () => new Repeater<string>(async (push, stop) => { const values = ['a', 'b', 'c', 'd', 'e', 'f', 'g'] const publish = () => { const value = values.shift() console.log('publish', value) if (value) { push(value) } if (values.length === 0) { stop() } } let interval = setInterval(publish, 1000) publish() await stop.then(() => { console.log('cancel') clearInterval(interval) }) }) } } }), plugins: [useDeferStream()] }) const server = createServer(yoga) server.listen(4000, () => { console.info('Server is running on http://localhost:4000/graphql') }) This server allows us to stream the alphabet. We set up a repeater that publishes a letter every second. Let's assume we actually have a data source that provides us with one letter every second. If we go back to the RFC example, it allowed us to use the argument to specify how many items we want to receive in the first response. initialCount First, this argument is missing in the Yoga implementation, but they could probably add it. But even if they added it, we'd still have the same problem. How does the user of the API know how many items to request in the first response? If we don't have fine-grained control over the data flow, how can we implement this feature efficiently? Imagine you're building a public GraphQL API, and someone asks for ridiculous values. initialCount You now have to add extra logic to validate the argument and protect your server. initialCount What if a user applies the directive to fields that are not supposed to be deferred? @defer E.g., if you're loading a user profile from the database, we will always load the user id, name, and email address using a single database query. If we now apply the directive to the email address, what is supposed to happen? @defer Should we load the email address in a separate database query? Should we load the email address in the same database query but defer it? Or should we just ignore the directive and return the email address in the first response? @defer What happens if an attacker intentionally sends queries with endless numbers of directives? @defer Should we now rate limit the use of directives? @defer Technically possible, but it just adds more complexity to the implementation. Keep in mind that every single implementation of and has to implement all of this logic. Last but not least, and that's what I'm afraid of the most, we could just hide all of this complexity from the user and swallow the performance penalty in the framework. @defer @stream This would mean that resolvers would not be aware of the and directives. @defer @stream However, this would lead to very inefficient implementations. Load all the data from the database, then apply the and directives and then return the data to the client; what's the point? @defer @stream Let's put an end to this rant and talk about a much simpler solution. What problem are we trying to solve here? What Problem Are We Solving With the and Directives in GraphQL? @defer @stream We need a blob of data in the user interface, but loading it all at once would be too slow. This can have multiple reasons: The data is too big to load all at once (e.g., a list of 1000 items) We need to load the data from multiple sources, one of which is slow How Does WunderGraph Solve This Problem? WunderGraph compiles GraphQL Operations into JSON-RPC requests. E.g., a GraphQL Subscription is turned into a stream of newline delimited JSON objects. Up until recently, you've had to implement Subscriptions in your GraphQL server and plug this server as a data source into WunderGraph, allowing you to expose the Subscription as a JSON-RPC stream. As I've said above, we've recently added support to implement JSON-RPC Operations directly in WunderGraph using TypeScript, supporting Queries, Mutations, and Subscriptions. Let's take a look at how we could implement the example from the RFC using a WunderGraph Subscription / JSON-RPC stream. // src/operations/user.ts import { createOperation, z } from '../../generated/wundergraph.factory'; export default createOperation.subscription({ input: z.object({ id: z.string(), }), response: z.object({ name: z.string(), films: z.array(z.object({ title: z.string(), })), homePlanet: z.object({ name: z.string(), }).nullable(), }), handler: async function* (ctx) { try { const userPromise = db.user.findUnique({ where: { id: ctx.input.id, }, }); const filmsPromise = db.film.findMany({ where: { id: { in: user.filmIds, }, }, limit: 2, }); const homePlanetPromise = db.planet.findUnique({ where: { id: user.homePlanetId, }, }); // we've defined three promises, but we don't want to wait for all of them const [user, films] = await Promise.all([userPromise, filmsPromise]); // we only wait for the user and the first two films to be loaded // Once these two promises are resolved, we yield the first chunk of data to the client yield { data: { person: { name: user.name, homePlanet: null, // we don't have the home planet yet, so we set it to null films, }, }, }; // Now we wait for the home planet to be loaded const homePlanet = await homePlanetPromise; // Once the home planet is loaded, we combine the data from the first two promises with the home planet // and yield the second chunk of data to the client yield { data: { person: { name: user.name, homePlanet, films, }, }, }; let skip = 2; const allFilms = Array.from(films); while (true) { // We now load the remaining films in chunks of 1 const films = await db.film.findMany({ where: { id: { in: user.filmIds, }, }, skip, limit: 1, }); if (films.length === 0) { break; } allFilms.push(...films); // We append the additional films to the list of films we've already loaded // and yield the updated response to the client yield { data: { person: { name: user.name, homePlanet, films: allFilms, }, }, }; skip += 1; } } finally { console.log('Client disconnected'); } }, }); The function allows us to implement a JSON-RPC Stream using an async generator function. Compared to an async function, an async generator function can yield multiple values. createOperation.subscription So, instead of returning a single value, like an async function, we can yield multiple values. Most importantly, we have exact control over the data flow. We're not limited to implementing a resolver that returns a stream of values. As we can yield multiple times, we can easily achieve the same result as the and directives. @defer @stream On the client side, you can use the generated type-safe client to subscribe to the stream. When there are no more yield statements in the generator function (when it returns), the stream will be closed. Here's what the client code would look like: import {useSubscription, withWunderGraph} from '../../components/generated/nextjs'; const Functions = () => { const {data} = useSubscription({ operationName: 'user', input: { id: '1', }, }); return ( <div> <h1>User</h1> <div> <div>name: {data.person.name}</div> {data.person.homePlanet && <div>homePlanet: {data.person.homePlanet}</div>} <div>films: {data.person.films.map((film) => film.title).join(', ')}</div> </div> </div> ); }; export default withWunderGraph(Functions); The client code is quite simple, and it's type-safe. This is because WunderGraph infers the response type from the operation definition. We can define the shape in the operation definition by creating a schema. response zod The generated client will use this schema to infer the response type. However, defining the response shape is optional. If we don't define it, the client will infer the response type from the operation handler, so it's up to the developer to either define the shape themselves or infer it from the operation handler. How Wundergraph Reduces the Amount of Data That Needs to Be Transferred As you can see in the example above, we've successfully implemented the and directives with a simple TypeScript function. @defer @stream So, from a developers' perspective, the problem seems to be solved. However, there's still one problem left. It seems like with every statement, we're sending the entire response object to the client. This means that we're sending the same data over and over again, only to add a single field or append it to an array. yield GraphQL Subscriptions suffer from the same problem. If you're using Subscriptions in your GraphQL server, and only a single field changes between two updates, you're still sending the entire response object to the client. WunderGraph solves this problem elegantly by using a technique called . Instead of reinventing the wheel and building our own implementation of incrementally loading data, we're using the JSON Patch standard to describe the changes between two updates. JSON Patch JSON Patch (RFC 6902) is a standard for describing changes to a JSON document. It's a format for expressing a sequence of operations to apply to a JSON document. Here's an example: [ { "op": "replace", "path": "/homePlanet", "value": { "name": "Tatooine" } }, { "op": "add", "path": "/films/-", "value": { "title": "A New Hope" } }, { "op": "add", "path": "/films/-", "value": { "title": "The Empire Strikes Back" } }, { "op": "add", "path": "/films/-", "value": { "title": "Return of the Jedi" } } ] The JSON Patch above describes the following changes: Replace the field with a new object homePlanet Add three new films to the array films The WunderGraph Server calculates the list of JSON Patch operations for each update. If the JSON Patch is smaller than sending the entire response object, the WunderGraph Server will send the JSON Patch to the client. In some cases, especially when the initial response is small, the JSON Patch will be larger than the entire response object. In this case, the WunderGraph Server will send the entire response object to the client instead of the JSON Patch. All WunderGraph clients support handling JSON Patch operations out of the box, but it's opt-in, so a client can choose if they want to use JSON Patch or not. Here's a curl command that shows how to subscribe to a Subscription Operation without using JSON Patch: curl http://localhost:9991/operations/user If you want to use JSON Patch, use the following command: curl http://localhost:9991/operations/user?wg_json_patch Additionally, you can also combine this with SSE (Server-Sent Events) by using the following command: curl http://localhost:9991/operations/user?wg_json_patch&wg_sse Conclusion I think that in 90% of the cases, you don't need and at all. For those rare cases where you want to micro-optimize a page, WunderGraph gives you all the tools you need to efficiently load data incrementally. @defer @stream More importantly, we're building on top of existing standards. As I said earlier, I was a big fan of the and directives. @defer @stream However, I think that there's a much simpler solution to the problem without having to introduce new directives to the GraphQL spec and without having to implement a new protocol. Streaming JSON Patch is a simple solution that works beyond just GraphQL. That said, WunderGraph also applies the same technique to GraphQL Subscriptions and Live Queries. So, if you're using GraphQL Subscriptions in your application already, you can easily switch to WunderGraph and the generated client & server will transparently use JSON Patch to reduce the amount of data that needs to be transferred. The and directives put control over the data flow in the hands of the API consumer, who not necessarily has the required knowledge to apply them in the most efficient way. @defer @stream Micro-optimizations should be avoided in most cases, but if you really need them, control over the data flow should be in the hands of the API provider. Try It Out for Yourself If you'd like to try out this feature for yourself, check out the on GitHub and try out one of the examples. WunderGraph Mono Repo Also published here
