112 reads

How We Built Open Previews and Used Invisible Divs in GitHub Discussions as a Key-Value Store

by Stefan Avram August 1st, 2023

Too Long; Didn't Read

Open Previews allows you to add commenting functionality to your previews/staging environments. It's a great way to collect feedback from your non technical team members, customers or other stakeholders. We had to store additional information with the comments, like the position, selection, and other meta data. The solution we came up with is inspired by [Giscus, which uses Github discussions to store comments.

featured image - How We Built Open Previews and Used Invisible Divs in GitHub Discussions as a Key-Value Store

Everything is a key-value store if you try hard enough. 🤓

We're excited to announce that the Open Previews beta is now available. In this blog post, we will explain how we built Open Previews and how we use divs in GitHub discussions as a key-value store. The source code is available on Github.

What is Open Previews?

Open Previews allows you to add commenting functionality to your previews/staging environments or any other website that you want to collect feedback on, like documentation pages. It's a great way to collect feedback from your nontechnical team members, customers, or other stakeholders. Preview comments are not a novel idea, services like Vercel and Netlify have supported this for a while now, but we wanted to build something that is open source and can be self-hosted.

How we built Open Previews

Before building Open Previews, we had a few requirements:

No database
The backend had to be stateless
Utilize WunderGraph as much as possible
Embeddable using a single script tag

The solution we came up with is inspired by Giscus, which uses GitHub discussions to store comments. This is a great solution and allows us to build Open Previews without a database.

The Problem of Storing Metadata in Github Discussions

There was one problem, though, we had to store additional information with the comments, like the position, selection, and other metadata. How do we store this information in Github discussions? But not only that, but the metadata should also not be visible to the user to not distract them, and we need to be able to retrieve it from the discussion when we render the comments.

The Solution: Using divs in HTML comments as a key-value store

GitHub comments support basic HTML, so what if we can simply embed JSON inside a data attribute in an empty DIV? After a simple experiment, we found out that this works great.

Adding JSON directly in a data attribute didn't work, though, because of the quotes and some other escaping issues, so we encoded the JSON using encodeURIComponent. Now we had a functional key-value store using divs and could store all the information that we needed. 🥳

Here's what it looks like in the code:

export const constructComment = ({
  body,
  meta,
}: {
  meta: {
    timestamp: number
    x: number
    y: number
    path: string
    href: string
    resolved?: boolean
    selection?: string
  }
  body: string
}) => {
  const jsonMeta = JSON.stringify(meta)

  const encodedJsonMeta = encodeURIComponent(jsonMeta)

  return `${body}

  <div data-comment-meta="${encodedJsonMeta}" />`
}

Now when we retrieve the comments, we can simply get the encoded data from the div, decode it, parse the JSON and render the comments on the page. The possibilities are endless 😎

Why GitHub Discussions is a great key-value store

GitHub Discussions as a key-value store is amazing because it comes with a lot of powerful features out of the box:

Our application is stateless
GitHub Discussions are virtually free and scale infinitely
built-in authentication: We can use GitHub’s authentication to authenticate users
build-in authorization: You can only comment on a discussion if you have access to the repository
built-in notifications: Users get notified when someone replies to their comment
built-in versioning: You can see the history of a discussion
built-in spam protection: GitHub Discussions has spam protection built-in

Stateless authentication

The next challenge was building stateless authentication. Adding authentication with WunderGraph is easy using the built-in Github Oauth provider, but we had to store the access tokens somewhere safely. Since previews are typically hosted on a different domain than the WunderGraph server, we can't use secure cookies either.

You might think that we could store the access tokens in our new GitHub Discussions KV, but that would be a very bad idea. Instead, we came up with a solution that uses JSON Web Encryption to store the access tokens in the browser. The JWT is encrypted using a secret that is only known to the backend. This allows us to verify the JWT and extract the access token from the JWT payload. The access token is then used to authenticate the user with Github.

After logging in, the encrypted JWT is exchanged between the WunderGraph server and the preview. We do this by starting the authentication flow in a popup that allows us to pass the JWT to the preview using the postMessage API. Alternatively, a compact token could also be appended to the redirect URI. The JWT is stored in the browser using the localStorage API.

The only limitation of this approach is that the JWT is only valid for a limited time, and the user needs to log in again after the JWT has expired.