How We Made a Virtual Drawing Board

Yay me – I just released an AlloApp! 🎉

The first of many to come, this basic drawing board is a server-side-running, inherently collaborative “tool” to be used simultaneously by multiple people in a virtual Place. It’s being built side-by-side with the continuous development of our AlloUI Library, whose purpose is to enable anyone to build their own AlloApps! So if you don’t like my drawing board – just make a better one yourself! 😉

Regardless of what you think of my drawing board – it may not look great or have its full intended feature set yet – it is working, and seing the result in action is extremely satisfying.

So, without further ado, I’d like to take a minute to walk through and demystify the process of building an AlloApp. Beginning with a set of lofty “what if you could…” ideas, and ending with a useful product that eventually led to this magical moment of trans-atlantic communication:

Ideation

A journey of a thousand miles should probably begin by figuring out where you’re going. As such, let’s consider some properties of a good drawing board. In my opinion, it should be a lagom sized black or white board, allowing its users to do free-hand drawing on it with a chalk or pen with high color contrast. In addition, you should be able to erase the board, and access to multiple colored pens is a plus. OK, that’s pretty much it.

Next, let’s get crazy. Remember, this is an app made for a world unconstrained by our traditional notions of physics. So, no holds barred, how can we imagine a drawing board that’s better than IRL? Let’s see what we can think of. What about…

• Being able to change the pen thickness and colour on the fly (without ever running out of ink or accidentally picking up a permanent marker some prankster left behind)
• The ability to draw from a distance
• The ability to dynamically change the size of the board if we run out of space

Now that’s a bit more exciting! Let’s get to building!

Implementation

“But wait a minute”, you say. “Shouldn’t there be a design phase first?“. If you do, then surely you haven’t peeked at the finished product yet. Seriously though, UI- and UX design for the Alloverse is something we’re very excited about establishing and iterating on, but this specific project is more about the technology. If you’re interested in the future of spatial UX, come discuss it in our Discord! We’d love to hear from you!

Anyway – for now, let’s focus on the immediate concern of creating a drawable surface. How do? After a quick foray into a completely over-engineered vector-based solution, I sobered up and found Cairo – a popular and super potent open source 2D graphics library. With it, we can create a Surface, paint it black and place it in the world. Due to the inherent simplicity of a drawing board, that’s all we need for a minimum representation of one in 3D!

Now, enter: interactions. Thanks to Nevyn’s efforts, Alloverse has built-in support for nifty interactions called point and poke. A point is triggered when a user overlaps their pointer beam (originating from the their Avatar’s hand and beaming outward) with an app component, giving the latter access to:

1. An identifier of the hand that pointed,
2. and in world coordinates, the point of intersection between the component and the beam.

Once we have that, converting the intersection point to local app coordinates is easy doable.

Now that we know where on the board the user is pointing, we simply wait for them to trigger the poke interaction by pushing the trigger button. As this happens, we can “make the pen touch the board” by telling Cairo to draw a small circle. And guess what? To emulate freehand line drawing, we now simply need to keep drawing circles at the point of intersection for as long as the user is holding the trigger button!

The current implementation requires us to continously save and load the board Surface to disk as a png image as long as someone is drawing. Thus, we’re limiting it to 20 updates per second for performance reasons. 😅 I know what you’re thinking, but hey, it works! We’re well aware this is a proof of concept and we’re happy with hacks – the ability to stream textures is on the roadmap and we’ll write a Real Solution™ soon™ .

Drawing the rest of the owl

With the basic functionality completed, we can move on to adding some other basic features:

• Buttons that increase/decrease the thickness of the pen. Controlling it is as easy as modifying the variable that sets the radius of the drawing circle.
• A “clear” button that resets the board by simply drawing it black.
• Like any other AlloApp, we can allow users to freely move it around in the Place. This can be achieved from the get-go by simply attaching to it the generic moveHandle created by Nevyn and supported by his latest math adventure.
• Resizing the board is done by grabbing and dragging the top-right resizeHandle which is locked to be moveable only along the board’s x- and y axes. This component started out as specific to the drawing board, but will likely be broken out into a more generic component in the future (much like the moveHandle above). Upon detecting the handle being moved, we:

1. Save the contents of the current board (technically, its Surface)
2. Create a new Surface with a size matching the position of the resizeHandle
3. Load the previous Surface‘s contents onto the new one.

All right, that’s about it! Let’s take a look at the final result:

After trying it out “in the field” for the first time I soon noticed a bug caused by my “single player” mindset. In a place with multiple people, the board couldn’t differentiate between which users sent the points and pokes, resulting in a line being drawn along any pointer beams intersecting with the board as long as someone was pushing their trigger button. Thankfully, this was patched relatively easily by implementing a drawingUserControlTable that keeps track of who initiated the intent to draw with a poke, and only allow the running of the circle-drawing function on point for those users.

Regardless, this proved a valuable lesson reminding me that Alloverse is an inherently multi-user environment, and that all apps need to keep this in mind from the get-go. Thus, in the future, the AlloUI framework will be inspired by my control table solution and in the future provide built-in support to more easily keep track of interactions from multiple users.

Moving Forward

Though the Drawing Board is functionally complete, it’s still obviously lacking some parts of the puzzle:

• Improved UI/UX: The board should simply look more like what it is, so as to reflect how it should be used. This probably includes at least having borders, and the buttons probably shouldn’t be visible for all users at all times.
• Prettier buttons and controls. This one’s hingeing on our work to enable the importing and streaming of assets. This one’s roadmapped to be coming soon™ .
• Additional features – at the very least, there should probably be an “eraser mode” and a way to change ink color.
• Under-the-hood improvements, like using streams instead of continuously saving and loading to/from disk. We’ll also need access controls, i.e. limiting who is able to draw or erase. Just imagine running a class in a virtual classroom without this feature… 😅

Final thoughts

Hey, thanks for reading this far! I hope you found it interesting or useful. For more posts like this, follow us on Twitter @alloverse or join our Discord!

Till next time!

— Tobias