In this tutorial I’ll show you how to build an app that allows you to use your midi keyboard to make music in real-time with friends! Here’s what we’re building:
In onMIDISuccess(), a for loop listens for midi messages. Right now it should be causing an error in the console. Why? Because we haven’t defined what to do when it receives a midi message.
Let’s create the onMIDImessage function referenced in the loop:
function onMIDImessage(messageData) {var newItem = document.createElement('li');newItem.appendChild(document.createTextNode(messageData.data));newItem.className = 'user-midi';document.getElementById('midi-data').prepend(newItem);}
This function creates a new <li> element. It appends a text node with our midi data. It adds a css class of user-midi (this will be important later). Finally, it adds that new list item to the unordered list with the id “midi-data”.
Midi keyup and keydown events
Pretty cool, eh?
But what do these numbers mean? Also: where’s the sound?
MIDI Protocol
MIDI Protocol is a rabbit hole all its own, but for our purposes you can understand the numbers with a simple chart:
On / Off → 144 / 128
Pitch → 0–127
Velocity → 0–127
When working with this data, we’ll treat the first number like an on/off switch. 144 = on, 128 = off. The second number is the range of pitches. The final velocity input could also be understood as volume in our usage here.
If you’d like a more in-depth look at midi, here’s a good place to start.
Sound
The MIDI data is not the sound, but a set of directions: ON/OFF, PITCH, VELOCITY. We’ll need to make our own synth that can turn this information into musical tones.
First, let’s convert those value sets from an array into a ‘note’ object that we can pass to our sound player. In onMIDImessage function, add:
var d = messageData.data; // Example: [144, 60, 100]var note = {on: d[0],pitch: d[1],velocity: d[2]}play(note);
Above, the variable ‘d’ is assigned the incoming data: an array of three numbers. Those three values are accessed by their index and assigned as values for the object properties. The note object is then passed to the play function. Let’s write that function:
function play(note){switch(note.on) {case 144:noteOn(frequency(note.pitch), note.velocity);break;case 128:noteOff(frequency(note.pitch), note.velocity);break;}
function frequency(note) {
return Math.pow(2, ((note - 69) / 12)) \* 440;
}
function noteOn(frequency, velocity) {
var osc = oscillators\[frequency\] = context.createOscillator();
osc.type = 'sawtooth';
osc.frequency.value = frequency;
osc.connect(context.destination);
osc.start(context.currentTime);
}
function noteOff(frequency, velocity) {
oscillators\[frequency\].stop(context.currentTime);
oscillators\[frequency\].disconnect();
}
}
This function checks to see if the note is on (144) or off (128) and triggers the appropriate command. Both noteOn and noteOff reference two global variables that we established at the top of index.js to handle the starting and stopping of sound.
The frequency function is used to convert the midi note number into a hertz frequency. If you’re curious, you can read more about it here.
Your app should now play like a synth. Huzzah!
Step 3: Adding Socket.io for Collaboration
Now it’s time for the fun stuff: syncing multiple users in a session! Socket.io will help us do that by enabling ‘real-time bidirectional event-based communication.’ That means we can send and receive midi messages as they’re triggered on any browser as they occur. Add Socket.io to the project:
npm install socket.io --save
You’ll also need to add this inside the head of index.html:
<!-- public/index.html -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/2.0.3/socket.io.js"></script>
Note: I’m using version 2.0.3 — be sure your versions of Socket.io match your package.json and html script. You might want this link to the CDN.
For Socket.io to connect users we need to assure that:
- Users can both send and receive notes
- The server listens for user input and broadcasts it to other users
User send/receive
First, let’s make sure the user emits a signal every time they play a note. This can be accomplished by opening index.js and adding a single line to the function onMIDImessage. Be sure to add this after the note object has been defined.
// public/index.js (inside onMIDImessage)
socket.emit('midi', note);
We also want to play any external notes that come in from other users. Inside index.js, add the following:
// public/index.js
var socket = io();socket.on('externalMidi', gotExternalMidiMessage);
function gotExternalMidiMessage(data) {var newItem = document.createElement('li');newItem.appendChild(document.createTextNode('Note: ' + data.pitch + ' Velocity: ' + data.velocity));newItem.className = "external-midi";document.getElementById('midi-data').prepend(newItem);
playNote(data);}
When we receive an ‘externalMidi’ message from the server, it triggers gotExternalMidiMessage. This function should look familiar — in fact, we could refactor this later, but for now we’ll repeat code for clarity. It displays the external note in the view in a manner that’s almost identical to how we treat midi input from our own keyboard. However, we’ve given the <li> a class name ‘external-midi’. This will be important in a moment when we add styles to differentiate between our midi input and that of outside users.
Finally, the note is passed to the player to trigger a sound.
Server
Now let’s make a bridge between users. We want to handle any incoming signals and pass them to any other sessions.
In server.js, require Socket.io and add the function newConnection. This will be triggered when the server gains a new connection.
// server.js
var socket = require('socket.io');var io = socket(server);
io.sockets.on('connection', newConnection);
function newConnection(socket) {console.log('new connection from:' + socket.id);
socket.on('midi', midiMsg);function midiMsg(data) {socket.broadcast.emit('externalMidi', data);}}
The newConnection console.log will appear in the terminal every time a new connection is made. We also have socket.on listening for messages from any user and then triggering midiMsg, which broadcasts that data to every user except the original user.
With that, we’re all set!
Step 3: Styling
If you’re just interested in seeing your notes differentiated from other players, you can take a shortcut here and simply add these classes to your style.css:
// public/css/sytle.css
.user-midi {color: green;}
.external-midi {color: blue;}
That’s it! Now you will see your own input in green and any external signal in blue. Feel free to skip to the next step and deploy your app to Heroku!
If you’d like to create a keyboard interface, let’s keep rolling:
Build a keyboard
This keyboard we’ll make builds off of @baileyparker’s CodePen project. We’ll make some design and functionality changes to fit our usage. To start, open up this pen in a separate window:
HTML
The div with the class ‘piano’ houses all of our keys, which are labeled with a variety of selectors. Follow the html structure in the pen and paste the piano div and all its keys into your document (/public/index.html).
CSS
This is a big update for our styles. We’re importing a google font, assigning a background image to the body, and finally giving colors to differentiate between .user-midi (pink) and .external-midi (blue) signals. The ul and li elements have been styled so that they’ll slant back at a pleasing angle.
The keyboard styling takes up the remainder of the CSS. Worth noting here are the ‘active’ classes like ‘.piano-key-natural:active’ or ‘.piano-key-natural-external:active’. These are triggered by the Javascript when a note is played. If it matches the data number, the CSS will activate that key to be pink for your notes and blue for any external input.
When you copy the CSS from the pen into your project’s style sheet (/public/style.css), be sure to follow the notes included inside. Most importantly, you’ll need to update the path to the background.
JS
You’ll do the same thing for the Javascript: cut and paste it into /public/index.js below the code we’ve written. Again, it is important to read and follow the few comments within the code.
This code will manage midi controller connections. But you’ll want to focus in on two functions: onMidiMessage & updateKeyboard. The former handles local input and applies the appropriate class to light up the keys. The latter does the same thing (but with a different class) for external messages.
To get external notes to light up the keyboard, we need to return to the function gotExternalMidiMessage. After we call play(), add the following code:
var msg = { }msg.data = [];msg.data.push(data.on);msg.data.push(data.pitch);msg.data.push(data.velocity);
updateKeyboard(data);
The keyboard needs a specific kind of object data structure to update, so we create msg and fill it with data from our note. That msg object is passed to updateKeyboard which lights up the keys in blue for that external signal.
You should now see your keys light up pink for your own input! When we connect to Heroku add more users, you’ll see those external midi messages light up in blue!
Step 4: Deploy to Heroku
Instead of walking through each step here, I’ll direct you towards the comprehensive documentation at Heroku. They’ve done a great job reviewing options for deployment:
Deploying with Git | Heroku Dev Center_Git is a powerful decentralized revision control system, and is the means for deploying apps to Heroku._devcenter.heroku.com
Summary
If all went well, you should now have an app that passes signals from one user to the next and plays notes from every user. The keyboard should light up with different colors for your input and that of external sources.
Completed App!
I’m excited about this collaborative app. It has been a real thrill hooking it up and inviting friends from other states to log on and play music! Looking back, I also see ways that the data could be passed more efficiently and areas where the design could be enhanced — it’s not friendly to smaller screens, for instance.
I’d love to hear how you’re using the app, and I hope it has given you a better understanding of how to combine Express.js, Socket.io, and the Web MIDI API!