While preparing this article, I decided to reorganize their structure slightly, so in this article, we will write our service, which will work well but will not be decentralized. In the following article, we will transfer user data storage to the blockchain and enrich our service with smart contracts. Read part 1 . here Shared services This function validates JWT token that he allowed for current row and for credentials of row; export function validateJWKS( jwtKey: string, data: object | string, validate: Record<string, any>, ) { const keys = typeof data === "object" ? data : JSON.parse(data); let k = keys.keys ? keys.keys : keys; if (!Array.isArray(k)) { k = [k]; } for (const jwk of k) { const key = createPublicKey({ format: "jwk", key: jwk }); const spki = key.export({ format: "pem", type: "spki" }); try { const result = jwt.verify(jwtKey, spki, validate) as Record<string, any>; const valid = Object.entries(validate).every(([key, value]) => { return result[key] === value; }); if (!valid) { throw new Error("Invalid token"); } return true; } catch (e) { // tslint:disable-next-line:no-console console.log(e); } } return false; } Auth Service This service is the first one our user encounters (in fact, the person who codes the interaction with the service). Its main task is to obtain a JWT key from the user, check whether this key can provide access to services, and, if so, generate a new key with which the user will continue to use the services. Why is it so difficult and impossible to go to services immediately with the existing key? Good question =) The answer to it will be quite simple: the service is designed in such a way that some of its parts can be distributed among community members, and so that these participants cannot use the user’s data in any way to harm him, we are giving away our anonymized identifier. This microservice contains one table in the database: Auth: in this table should be stored available OAUTH providers with their credentials export class Auth extends Model implements AuthAttributes { public id!: string; public name!: string; public jwkUrl!: string; public verifier!: string; public checks!: AuthCheckAttribute[]; // timestamps! public readonly createdAt!: Date; public readonly updatedAt!: Date; public readonly deletedAt!: Date; } And two rest endpoints - on this endpoint, we can get certificates to validate our JWT POST /api/certs const keyStore = await jose.JWK.asKeyStore( (process.env.AUTH_KEYS as string).toString(), ); res.status(200).json(keyStore.toJSON()); We get stored secret data from env variable and get public key - in this endpoint we exchange users JWT into our POST /api/exchange At first we need to encode and check field sub const { token } = req.body; const encoded = jwt.decode(token); if (encoded === null || !encoded.hasOwnProperty("sub")) { res.status(400).json({ message: "Invalid token" }); return; } After it we check that the current token is allowed for the next iterations. const tokens = await Auth.findAll({}); let isValid = false; for (const authToken of tokens) { try { if (!cache.get(authToken.get("id"))) { const k1 = await fetch(authToken.get("jwkUrl")); const k2 = await k1.json(); if (k2) { cache.set(authToken.id, { validate: Object.fromEntries( authToken.get("checks").map((check) => [check.key, check.value]), ), ks: k2, }); } } const c = cache.get(authToken.id); if (!c) { continue; } isValid = validateJWKS(token, c.ks, c.validate) || isValid; } catch (e) { // tslint:disable-next-line:no-console console.error(e); } } if (!isValid) { res.status(400).json({ message: "Invalid token" }); return; } And if all checks are done, we generate a new token. Shares service This service is required to exchange JWT tokens for information where stored parts of his private key are. This microservice contains two tables table in the database: Node: here, stored information about nodes that can be saved user share export class Node extends Model implements NodeAttributes { public id!: string; public name!: string; public value!: string; // timestamps! public readonly createdAt!: Date; public readonly updatedAt!: Date; public readonly deletedAt!: Date; } Storage: here store information about current users parts. Identifier is a hashed string with salt, which prevents disclosure of which specific user owns the data export class Storage extends Model implements StorageAttributes { public id!: string; public value!: StorageValueAttribute[]; // timestamps! public readonly createdAt!: Date; public readonly updatedAt!: Date; public readonly deletedAt!: Date; } Also we need auth middleware: we check token validity and hash user identifier for next iterations: function authMiddleware( req: RequestWithUID, res: Response, next: NextFunction, ) { let encoded = jwt.decode(req.body.token); if (encoded === null || !encoded.hasOwnProperty("sub")) { return res.status(400).json({ message: "Invalid token" }); } req.uid = keccak256(encoded.sub as string, process.env.AUTH_SECRET as string); next(); } And two rest endpoints: - return saved user data POST /api/get const value = await Storage.findByPk(req.uid); if (value === null) { return res.status(404).json({ message: "Node not found" }); } res.status(200).json(value.get("value")); - generate shares and return them POST /api/generate let nodes = await Node.findAll({}); if (nodes.length < 5) { return res.status(400).json({ message: "Not enough nodes" }); } const values = nodes .map((node) => node.get("value")) .sort(() => Math.random() - 0.5) .slice(0, 5) .map((node) => ({ node: node, index: randomBytes(32).toString("hex"), })); await Storage.create({ id: req.uid, value: values, }); const value = await Storage.findByPk(req.uid); if (value === null) { return res.status(404).json({ message: "Node not found" }); } res.status(200).json(value.get("value")); That happens here? we get all saved nodes, shuffle them and get 5 values, also we generate 5 indices for next iterations, store them in the database and return to the user. Share service This service is required for store user’s part of a share. each entry is encrypted with a generated private key to protect user data. We will use the for encryption and decryption. As a private key, we will use a hashed user identifier specific to each service salt. Elliptic Curve Integrated Encryption Scheme (ECIES) We get the public key from the private key, which will be an identifier for our record, and encoded with the private key and ECIES. This microservice contains one table in the database: Storage: here stores encrypted user share. Identifier is a hashed string with salt, which prevents disclosure of which specific user owns the data, value encrypted with a calculated private key export class Storage extends Model implements StorageAttributes { public id!: string; public value!: string; // timestamps! public readonly createdAt!: Date; public readonly updatedAt!: Date; public readonly deletedAt!: Date; } And two rest endpoints: create stored data. After saving, we get this row from the database to be sure that data is saved. I know that when saved, the model should return the saved row, but in this case it is better to query from the database POST /api/set const sk = new PrivateKey(req.uid as Buffer); const decData = encrypt( sk.publicKey.toHex(), Buffer.from(req.body.data as string, "utf-8"), ); await Storage.create({ id: sk.publicKey.toHex(), value: decData.toString("base64"), }); const encrypted = await Storage.findByPk(sk.publicKey.toHex()); if (encrypted === null) { return res.status(404).json({ message: "Not found" }); } const encData = Buffer.from(encrypted.get("value"), "base64"); res.status(200).json({ key: sk.publicKey.toHex(), data: decrypt(sk.secret, encData).toString(), }); encrypt and retrieve saved users share POST /api/get const sk = new PrivateKey(req.uid as Buffer); const encrypted = await Storage.findByPk(sk.publicKey.toHex()); if (encrypted === null) { return res.status(404).json({ message: "Not found" }); } const data = Buffer.from(encrypted.get("value"), "base64"); res.status(200).json({ key: sk.publicKey.toHex(), data: decrypt(sk.secret, data).toString(), }); Meta service This service is similar to the previous one in many ways, with the difference that the data is encrypted not on the backend side but on the client and signed by the client with his private key. This service can store any user information. In our case, we will store indexes from user parts of the shares. Before any interaction, like create or retrieve, we should check the signature created from the namespace on which data will be saved and the current timestamp to prevent the next use. For now, we window in one minute. For additional security, you can additionally use and save it, for example, in radish, so that the entry is deleted in a minute. nonce This microservice contains one table in the database: Storage: here stores encrypted user shares. Identifier is a hashed string with salt, which prevents disclosure of which specific user owns the data, value encrypted with a calculated private key. export class Storage extends Model implements StorageAttributes { public id!: string; public value!: string; // timestamps! public readonly createdAt!: Date; public readonly updatedAt!: Date; public readonly deletedAt!: Date; } store user data POST /api/set const { pk, namespace, signature, ts, message } = req.body; const msgHash = Buffer.from(keccak256(`${namespace}:${ts}`), "hex"); const isValid = ec.verify(msgHash, signature, pk, "hex"); if (!isValid || ts < Math.floor(Date.now() / 1000) - 60) { return res.status(400).json({ message: "Invalid signature" }); } const key = keccak256(`${pk}:${namespace}`); await Storage.create({ id: key, value: message, }); const data = await Storage.findByPk(key); if (data === null) { return res.status(404).json({ message: "Not found" }); } res.status(200).json(data.get("value")); retrieve saved users share. POST /api/get const { pk, namespace, signature, ts } = req.body; const msgHash = Buffer.from(keccak256(`${namespace}:${ts}`), "hex"); const isValid = ec.verify(msgHash, signature, Buffer.from(pk, "hex"), "hex"); if (!isValid || ts < Math.floor(Date.now() / 1000) - 60) { return res.status(400).json({ message: "Invalid signature" }); } const key = keccak256(`${pk}:${namespace}`); const data = await Storage.findByPk(key); if (data === null) { return res.status(404).json({ message: "Not found" }); } res.status(200).json(data.get("value")); Conclusion In this article, I described the service in basic terms and how it should work. Unfortunately, it turned out a little crumpled; in the future, I will try to cover the topic in more detail. In the next part of the article, I plan to transfer parts of the storage functionality to smart contracts to make the service more decentralized. All code available on GitHub Links: Elliptic Curve Integrated Encryption Scheme (ECIES): Encrypting Using Elliptic Curves Elliptic Curve Digital Signature Algorithm
