While preparing this article, I decided to reorganize their structure slightly, so in this article, we will write our service, which will work well but will not be decentralized. In the following article, we will transfer user data storage to the blockchain and enrich our service with smart contracts.

Read part 1 here.

Shared services

This function validates JWT token that he allowed for current row and for credentials of row;

export function validateJWKS(
  jwtKey: string,
  data: object | string,
  validate: Record<string, any>,
) {
  const keys = typeof data === "object" ? data : JSON.parse(data);
  let k = keys.keys ? keys.keys : keys;

  if (!Array.isArray(k)) {
    k = [k];
  }

  for (const jwk of k) {
    const key = createPublicKey({ format: "jwk", key: jwk });
    const spki = key.export({ format: "pem", type: "spki" });

    try {
      const result = jwt.verify(jwtKey, spki, validate) as Record<string, any>;

      const valid = Object.entries(validate).every(([key, value]) => {
        return result[key] === value;
      });

      if (!valid) {
        throw new Error("Invalid token");
      }

      return true;
    } catch (e) {
      // tslint:disable-next-line:no-console
      console.log(e);
    }
  }

  return false;
}

Auth Service

This service is the first one our user encounters (in fact, the person who codes the interaction with the service). Its main task is to obtain a JWT key from the user, check whether this key can provide access to services, and, if so, generate a new key with which the user will continue to use the services.

Why is it so difficult and impossible to go to services immediately with the existing key? Good question =) The answer to it will be quite simple: the service is designed in such a way that some of its parts can be distributed among community members, and so that these participants cannot use the user’s data in any way to harm him, we are giving away our anonymized identifier.

This microservice contains one table in the database:

Auth: in this table should be stored available OAUTH providers with their credentials

export class Auth extends Model implements AuthAttributes
{
  public id!: string;
  public name!: string;
  public jwkUrl!: string;
  public verifier!: string;
  public checks!: AuthCheckAttribute[];

  // timestamps!
  public readonly createdAt!: Date;
  public readonly updatedAt!: Date;
  public readonly deletedAt!: Date;
}

And two rest endpoints

POST /api/certs - on this endpoint, we can get certificates to validate our JWT

const keyStore = await jose.JWK.asKeyStore(
    (process.env.AUTH_KEYS as string).toString(),
);
res.status(200).json(keyStore.toJSON());

We get stored secret data from env variable and get public key

POST /api/exchange - in this endpoint we exchange users JWT into our

At first we need to encode and check sub field

const { token } = req.body;

const encoded = jwt.decode(token);
if (encoded === null || !encoded.hasOwnProperty("sub")) {
  res.status(400).json({ message: "Invalid token" });
  return;
}

After it we check that the current token is allowed for the next iterations.

const tokens = await Auth.findAll({});

let isValid = false;
for (const authToken of tokens) {
  try {
    if (!cache.get(authToken.get("id"))) {
      const k1 = await fetch(authToken.get("jwkUrl"));
      const k2 = await k1.json();
      if (k2) {
        cache.set(authToken.id, {
          validate: Object.fromEntries(
            authToken.get("checks").map((check) => [check.key, check.value]),
          ),
          ks: k2,
        });
      }
    }

    const c = cache.get(authToken.id);
    if (!c) {
      continue;
    }

    isValid = validateJWKS(token, c.ks, c.validate) || isValid;
  } catch (e) {
    // tslint:disable-next-line:no-console
    console.error(e);
  }
}

if (!isValid) {
  res.status(400).json({ message: "Invalid token" });
  return;
}

And if all checks are done, we generate a new token.

Shares service

This service is required to exchange JWT tokens for information where stored parts of his private key are.

This microservice contains two tables table in the database:

Node: here, stored information about nodes that can be saved user share

export class Node extends Model implements NodeAttributes {
    public id!: string;
    public name!: string;
    public value!: string;

    // timestamps!
    public readonly createdAt!: Date;
    public readonly updatedAt!: Date;
    public readonly deletedAt!: Date;
}

Storage: here store information about current users parts. Identifier is a hashed string with salt, which prevents disclosure of which specific user owns the data

export class Storage extends Model implements StorageAttributes {
    public id!: string;
    public value!: StorageValueAttribute[];

    // timestamps!
    public readonly createdAt!: Date;
    public readonly updatedAt!: Date;
    public readonly deletedAt!: Date;
}

Also we need auth middleware: we check token validity and hash user identifier for next iterations:

function authMiddleware(
  req: RequestWithUID,
  res: Response,
  next: NextFunction,
) {
  let encoded = jwt.decode(req.body.token);

  if (encoded === null || !encoded.hasOwnProperty("sub")) {
    return res.status(400).json({ message: "Invalid token" });
  }

  req.uid = keccak256(encoded.sub as string, process.env.AUTH_SECRET as string);

  next();
}

And two rest endpoints:

POST /api/get - return saved user data

const value = await Storage.findByPk(req.uid);

if (value === null) {
  return res.status(404).json({ message: "Node not found" });
}

res.status(200).json(value.get("value"));

POST /api/generate - generate shares and return them

let nodes = await Node.findAll({});

if (nodes.length < 5) {
  return res.status(400).json({ message: "Not enough nodes" });
}

const values = nodes
  .map((node) => node.get("value"))
  .sort(() => Math.random() - 0.5)
  .slice(0, 5)
  .map((node) => ({
    node: node,
    index: randomBytes(32).toString("hex"),
  }));

await Storage.create({
  id: req.uid,
  value: values,
});

const value = await Storage.findByPk(req.uid);

if (value === null) {
  return res.status(404).json({ message: "Node not found" });
}

res.status(200).json(value.get("value"));

That happens here? we get all saved nodes, shuffle them and get 5 values, also we generate 5 indices for next iterations, store them in the database and return to the user.

Share service

This service is required for store user’s part of a share. each entry is encrypted with a generated private key to protect user data.

We will use the Elliptic Curve Integrated Encryption Scheme (ECIES) for encryption and decryption. As a private key, we will use a hashed user identifier specific to each service salt.

We get the public key from the private key, which will be an identifier for our record, and encoded with the private key and ECIES.

This microservice contains one table in the database:

Storage: here stores encrypted user share. Identifier is a hashed string with salt, which prevents disclosure of which specific user owns the data, value encrypted with a calculated private key

export class Storage extends Model implements StorageAttributes {
  public id!: string;
  public value!: string;

  // timestamps!
  public readonly createdAt!: Date;
  public readonly updatedAt!: Date;
  public readonly deletedAt!: Date;
}

And two rest endpoints:

POST /api/set create stored data. After saving, we get this row from the database to be sure that data is saved. I know that when saved, the model should return the saved row, but in this case it is better to query from the database

const sk = new PrivateKey(req.uid as Buffer);

const decData = encrypt(
  sk.publicKey.toHex(),
  Buffer.from(req.body.data as string, "utf-8"),
);

await Storage.create({
  id: sk.publicKey.toHex(),
  value: decData.toString("base64"),
});

const encrypted = await Storage.findByPk(sk.publicKey.toHex());

if (encrypted === null) {
  return res.status(404).json({ message: "Not found" });
}

const encData = Buffer.from(encrypted.get("value"), "base64");
res.status(200).json({
  key: sk.publicKey.toHex(),
  data: decrypt(sk.secret, encData).toString(),
});

POST /api/get encrypt and retrieve saved users share

const sk = new PrivateKey(req.uid as Buffer);

const encrypted = await Storage.findByPk(sk.publicKey.toHex());

if (encrypted === null) {
  return res.status(404).json({ message: "Not found" });
}

const data = Buffer.from(encrypted.get("value"), "base64");
res.status(200).json({
  key: sk.publicKey.toHex(),
  data: decrypt(sk.secret, data).toString(),
});

Meta service

This service is similar to the previous one in many ways, with the difference that the data is encrypted not on the backend side but on the client and signed by the client with his private key. This service can store any user information. In our case, we will store indexes from user parts of the shares.

Before any interaction, like create or retrieve, we should check the signature created from the namespace on which data will be saved and the current timestamp to prevent the next use. For now, we window in one minute.

For additional security, you can additionally use nonce and save it, for example, in radish, so that the entry is deleted in a minute.

This microservice contains one table in the database:

Storage: here stores encrypted user shares. Identifier is a hashed string with salt, which prevents disclosure of which specific user owns the data, value encrypted with a calculated private key.

export class Storage extends Model implements StorageAttributes {
  public id!: string;
  public value!: string;

  // timestamps!
  public readonly createdAt!: Date;
  public readonly updatedAt!: Date;
  public readonly deletedAt!: Date;
}

POST /api/set store user data

const { pk, namespace, signature, ts, message } = req.body;
const msgHash = Buffer.from(keccak256(`${namespace}:${ts}`), "hex");
const isValid = ec.verify(msgHash, signature, pk, "hex");

if (!isValid || ts < Math.floor(Date.now() / 1000) - 60) {
  return res.status(400).json({ message: "Invalid signature" });
}

const key = keccak256(`${pk}:${namespace}`);

await Storage.create({
  id: key,
  value: message,
});

const data = await Storage.findByPk(key);

if (data === null) {
  return res.status(404).json({ message: "Not found" });
}

res.status(200).json(data.get("value"));

POST /api/get retrieve saved users share.

const { pk, namespace, signature, ts } = req.body;
const msgHash = Buffer.from(keccak256(`${namespace}:${ts}`), "hex");

const isValid = ec.verify(msgHash, signature, Buffer.from(pk, "hex"), "hex");

if (!isValid || ts < Math.floor(Date.now() / 1000) - 60) {
  return res.status(400).json({ message: "Invalid signature" });
}

const key = keccak256(`${pk}:${namespace}`);
const data = await Storage.findByPk(key);

if (data === null) {
  return res.status(404).json({ message: "Not found" });
}

res.status(200).json(data.get("value"));

Conclusion

In this article, I described the service in basic terms and how it should work. Unfortunately, it turned out a little crumpled; in the future, I will try to cover the topic in more detail. In the next part of the article, I plan to transfer parts of the storage functionality to smart contracts to make the service more decentralized.

All code available on GitHub

Links:

Elliptic Curve Integrated Encryption Scheme (ECIES): Encrypting Using Elliptic Curves

Elliptic Curve Digital Signature Algorithm

Utilizing the Mechanics of Shamir's Secret Sharing Service — Part 2

Shared services

Auth Service

Shares service

Share service

Meta service

Conclusion

Links: