Almost every web application requires data serialization. This need arises in situations like:
In many cases, data loss or corruption can lead to serious consequences, making it essential to provide a convenient and safe serialization mechanism that helps detect as many errors as possible during the development stage. For these purposes, it's convenient to use JSON as the data transfer format and TypeScript for static code checking during development.
TypeScript serves as a superset of JavaScript, which should enable the seamless use of functions like JSON.stringify
and JSON.parse
, right? Turns out, despite all its benefits, TypeScript doesn't naturally understand what JSON is and which data types are safe for serialization and deserialization into JSON.
Let's illustrate this with an example.
Consider, for example, a function that saves some data to LocalStorage. As LocalStorage cannot store objects, we use JSON serialization here:
interface PostComment {
authorId: string;
text: string;
updatedAt: Date;
}
function saveComment(comment: PostComment) {
const serializedComment = JSON.stringify(comment);
localStorage.setItem('draft', serializedComment);
}
We will also need a function to retrieve the data from LocalStorage.
function restoreComment(): PostComment | undefined {
const text = localStorage.getItem('draft');
return text ? JSON.parse(text) : undefined;
}
What’s wrong with this code? The first problem is that when restoring the comment, we will get a string
type instead of Date
for the updatedAt
field.
This happens because JSON only has four primitive data types (null
, string
, number
, boolean
), as well as arrays and objects. It is not possible to save a Date
object in JSON, as well as other objects that are found in JavaScript: functions, Map, Set, etc.
When JSON.stringify
encounters a value that cannot be represented in JSON format, type casting occurs. In the case of a Date
object, we get a string because the Date
object implements the toJson() method, which returns a string instead of a Date
object.
const date = new Date('August 19, 1975 23:15:30 UTC');
const jsonDate = date.toJSON();
console.log(jsonDate);
// Expected output: "1975-08-19T23:15:30.000Z"
const isEqual = date.toJSON() === JSON.stringify(date);
console.log(isEqual);
// Expected output: true
The second problem is that the saveComment
function returns the PostComment
type, in which the date field is of type Date
. But we already know that instead of Date
, we will receive a string
type. TypeScript could help us find this error, but why doesn't it?
Turns out, in TypeScript's standard library, the JSON.parse
function is typed as (text: string) => any
. Due to the use of any
, type checking is essentially disabled. In our example, TypeScript simply took our word that the function would return a PostComment
containing a Date
object.
This TypeScript behavior is inconvenient and unsafe. Our application may crash if we try to treat a string like a Date
object. For example, it might break if we call comment.updatedAt.toLocaleDateString()
.
Indeed, in our small example, we could simply replace the Date
object with a numerical timestamp, which works well for JSON serialization. However, in real applications, data objects might be extensive, types can be defined in multiple locations, and identifying such an error during development may be a challenging task.
What if we could enhance TypeScript's understanding of JSON?
To start with, let's figure out how to make TypeScript understand which data types can be safely serialized into JSON. Suppose we want to create a function safeJsonStringify
, where TypeScript will check the input data format to ensure it's JSON serializable.
function safeJsonStringify(data: JSONValue) {
return JSON.stringify(data);
}
In this function, the most important part is the JSONValue
type, which represents all possible values that can be represented in the JSON format. The implementation is quite straightforward:
type JSONPrimitive = string | number | boolean | null | undefined;
type JSONValue = JSONPrimitive | JSONValue[] | {
[key: string]: JSONValue;
};
First, we define the JSONPrimitive
type, which describes all the primitive JSON data types. We also include the undefined
type based on the fact that when serialized, keys with the undefined
value will be omitted. During deserialization, these keys will simply not appear in the object, which in most cases is the same thing.
Next, we describe the JSONValue
type. This type uses TypeScript's ability to describe recursive types, which are types that refer to themselves. Here, JSONValue
can either be a JSONPrimitive
, an array of JSONValue
, or an object where all values are of the JSONValue
type. As a result, a variable of this type JSONValue
can contain arrays and objects with unlimited nesting. The values within these will also be checked for compatibility with the JSON format.
Now we can test our safeJsonStringify
function using the following examples:
// No errors
safeJsonStringify({
updatedAt: Date.now()
});
// Yields an error:
// Argument of type '{ updatedAt: Date; }' is not assignable to parameter of type 'JSONValue'.
// Types of property 'updatedAt' are incompatible.
// Type 'Date' is not assignable to type 'JSONValue'.
safeJsonStringify({
updatedAt: new Date();
});
Everything seems to function properly. The function allows us to pass the date as a number but yields an error if we pass the Date
object.
But let's consider a more realistic example, in which the data passed to the function is stored in a variable and has a described type.
interface PostComment {
authorId: string;
text: string;
updatedAt: number;
};
const comment: PostComment = {...};
// Yields an error:
// Argument of type 'PostComment' is not assignable to parameter of type 'JSONValue'.
// Type 'PostComment' is not assignable to type '{ [key: string]: JSONValue; }'.
// Index signature for type 'string' is missing in type 'PostComment'.
safeJsonStringify(comment);
Now, things are getting a bit tricky. TypeScript won't let us assign a variable of type PostComment
to a function parameter of type JSONValue
, because "Index signature for type 'string' is missing in type 'PostComment'".
So, what is an index signature and why is it missing? Remember how we described objects that can be serialized into the JSON format?
type JSONValue = {
[key: string]: JSONValue;
};
In this case, [key: string]
is the index signature. It means "this object can have any keys in the form of strings, the values of which have the JSONValue
type". So, it turns out we need to add an index signature to the PostComment
type, right?
interface PostComment {
authorId: string;
text: string;
updatedAt: number;
// Don't do this:
[key: string]: JSONValue;
};
Doing so would imply that the comment could contain any arbitrary fields, which is not typically the desired outcome when defining data types in an application.
The real solution to the problem with the index signature comes from Mapped Types, which allow for recursively iterating over fields, even for types that don't have an index signature defined. Combined with generics, this feature allows converting any data type T
into another type JSONCompatible<T>
, which is compatible with the JSON format.
type JSONCompatible<T> = unknown extends T ? never : {
[P in keyof T]:
T[P] extends JSONValue ? T[P] :
T[P] extends NotAssignableToJson ? never :
JSONCompatible<T[P]>;
};
type NotAssignableToJson =
| bigint
| symbol
| Function;
The JSONCompatible<T>
type is a mapped type that inspects whether a given type T
can be safely serialized into JSON. It does this by iterating over each property in type T
and doing the following:
T[P] extends JSONValue ? T[P] : ...
conditional type verifies if the property's type is compatible with the JSONValue
type, assuring it can be safely converted to JSON. When this is the case, the property's type remains unchanged.T[P] extends NotAssignableToJson ? never : ...
conditional type verifies if the property’s type isn't assignable to JSON. In this case, the property's type is converted to never
, effectively filtering the property out from the final type.
The unknown extends T ? never :...
check at the beginning is used to prevent the unknown
type from being converted to an empty object type {}
, which is essentially equivalent to the any
type.
Another interesting aspect is the NotAssignableToJson
type. It consists of two TypeScript primitives (bigint and symbol) and the Function
type, which describes any possible function. The Function
type is crucial in filtering out any values that aren't assignable to JSON. This is because any complex object in JavaScript is based on the Object type and has at least one function in its prototype chain (e.g., toString()
). The JSONCompatible
type iterates over all of those functions, so checking functions is sufficient to filter out anything that isn't serializable to JSON.
Now, let's use this type in the serialization function:
function safeJsonStringify<T>(data: JSONCompatible<T>) {
return JSON.stringify(data);
}
Now, the function uses a generic parameter T
and accepts the JSONCompatible<T>
argument. This means it takes an argument data
of type T
, which should be a JSON-compatible type. Now we can use the function with data types without an index signature.
The function now uses a generic parameter T
that extends from the JSONCompatible<T>
type. This means that it accepts an argument data
of type T
, which ought to be a JSON-compatible type. As a result, we can utilize the function with data types that lack an index signature.
interface PostComment {
authorId: string;
text: string;
updatedAt: number;
}
function saveComment(comment: PostComment) {
const serializedComment = safeJsonStringify(comment);
localStorage.setItem('draft', serializedComment);
}
This approach can be used whenever JSON serialization is necessary, such as transferring data over the network, embedding data in HTML, storing data in localStorage, transferring data between workers, etc. Additionally, the toJsonValue
helper can be used when a strictly typed object without an index signature needs to be assigned to a variable of JSONValue
type.
function toJsonValue<T>(value: JSONCompatible<T>): JSONValue {
return value;
}
const comment: PostComment = {...};
const data: JSONValue = {
comment: toJsonValue(comment)
};
In this example, using toJsonValue
lets us bypass the error related to the missing index signature in the PostComment
type.
When it comes to deserialization, the challenge is both simpler and more complex simultaneously because it involves both static analysis checks and runtime checks for the received data's format.
From the perspective of TypeScript's type system, the challenge is quite simple. Let's consider the following example:
function safeJsonParse(text: string) {
return JSON.parse(text) as unknown;
}
const data = JSON.parse(text);
// ^? unknown
In this instance, we're substituting the any
return type with the unknown
type. Why choose unknown
? Essentially, a JSON string could contain anything, not just the data that we expect to receive. For example, the data format might change between different application versions or another part of the app could write data to the same LocalStorage key. Therefore, unknown
is the safest and most precise choice.
However, working with the unknown
type is less convenient than merely specifying the desired data type. Apart from type-casting, there are multiple ways to convert the unknown
type into the required data type. One such method is utilizing the Superstruct library to validate data at runtime and throw detailed errors if the data is invalid.
import { create, object, number, string } from 'superstruct';
const PostComment = object({
authorId: string(),
text: string(),
updatedAt: number(),
});
// Note: we no longer need to manually specify the return type
function restoreDraft() {
const text = localStorage.getItem('draft');
return text ? create(JSON.parse(text), PostComment) : undefined;
}
Here, the create
function acts as a type guard, narrowing the type to the desired Comment
interface. Consequently, we no longer need to manually specify the return type.
Implementing a secure deserialization option is only half the story. It's equally crucial not to forget to use it when tackling the next task in the project. This becomes particularly challenging if a large team is working on the project, as ensuring all agreements and best practices are followed can be difficult.
Typescript-eslint can assist in this task. This tool helps identify all instances of unsafe any
usage. Specifically, all usages of JSON.parse
can be found and it can be ensured that the received data's format is checked. More about getting rid of the any
type in a codebase can be read in the article Making TypeScript Truly "Strongly Typed".
Here are the final utility functions and types designed to assist in safe JSON serialization and deserialization. You can test these in the prepared TS Playground.
type JSONPrimitive = string | number | boolean | null | undefined;
type JSONValue = JSONPrimitive | JSONValue[] | {
[key: string]: JSONValue;
};
type NotAssignableToJson =
| bigint
| symbol
| Function;
type JSONCompatible<T> = unknown extends T ? never : {
[P in keyof T]:
T[P] extends JSONValue ? T[P] :
T[P] extends NotAssignableToJson ? never :
JSONCompatible<T[P]>;
};
function toJsonValue<T>(value: JSONCompatible<T>): JSONValue {
return value;
}
function safeJsonStringify<T>(data: JSONCompatible<T>) {
return JSON.stringify(data);
}
function safeJsonParse(text: string): unknown {
return JSON.parse(text);
}
These can be used in any situation where JSON serialization is necessary.
I've been using this strategy in my projects for several years now, and it has demonstrated its effectiveness by promptly detecting potential errors during application development.
I hope this article has provided you with some fresh insights. Thank you for reading!