I love JavaScript and I love the productivity and flexibility that comes from a good command line interface. If you have read any of my previous articles here:
Mike Parsons - Hacker Noon_Read writing from Mike Parsons in Hacker Noon. Software Research Geek. Every day, Mike Parsons and thousands of other…_hackernoon.com
you know that I am a big fan of getting a lot done in a few lines of code. I spend most of my time inside the browser console and while I love the power and expressiveness of the development tools, they are often overkill for what I want. I also like having a consistent JavaScript Console interface that is available from anywhere, not just inside the browser. For example, there are times when I want to use JavaScript outside the browser to automate Window’s Office Applications or script Node.js or use the browser in conjunction with native applications like SQL Server to perform some data tasks that are just not available inside the browser sandbox. One common use case I have frequently is to scrape some data from a web page and insert it into a native application like Microsoft Excel for further data analysis.
While each of the environments listed in the title have there own console environments and REPL’s, I have found the command line interfaces limited in terms of integration with the host Operating System. Most operate inside a sandbox that makes perfect sense for the casual user but really restricts the capabilities I want as a power user or system’s engineer.
Another use for this console is during presentations or live coding sessions when you want to demonstrate ideas quickly. I use it frequently to prototype ideas and to validate features available in the browser or the underlying OS.
My Universal JavaScript Console is based on the terrific work done by Remy Sharp
remy/jsconsole_jsconsole - Web based console - for presentations and workshops_github.com
I took Remy’s excellent console and extended it with a few features that allow me to host the console in multiple environments.
Let’s get started with the browser interface, found here:
Console_Universal JavaScript Console_s3.amazonaws.com
When you first navigate to the console, you are presented with this interface:
You can start entering any valid JavaScript commands and immediately see the output. Try entering a few commands:
Date()locationnavigator.userAgent1+1
You can enter any valid JavaScript, including setting variables and/or creating new functions.
Many times you want to leverage an existing JavaScript library such as jQuery, lodash or the amazing ALASQL that allows you to use SQL in the browser. You can simply load the library via it’s URL, for example:
I have added a few shortcuts for libraries that I frequently use:
jqueryprototypedojomootoolsunderscorelodashmomentyuialasql
In addition to loading JavaScript Libraries, you can also load HTML content from another Web Site. This is useful in scraping web pages, for example:
These previous examples demonstrate using the Console in standalone mode.
This is useful but my number one use for the Console is by leveraging it in the context of an existing Web Page or Site. In this mode, you basically inject the Console inside an existing Web Page and it acquires the context of the page so you can script the contents of the page just as you would if you opened the browser development tools. However, unlike the browser console, which is sandboxed via the web site settings, the Universal Console provides a great deal more power in terms of bringing in existing libraries into an existing web page and/or via extended network capabilities, like WebSockets or cross domain XMHttpRequest calls. This is accomplished via the use of a Bookmarklet, which you can access via the :help command. Simple drag the Bookmarklet from the help page to your browser Bookmarks and it becomes available from any web site.
To illustrate this functionality, we are going to navigate to https://news.ycombinator.com/newest and Inject the Console into this page so we can script it’s contents. For this example, we are going to use the Console and one of my favourite Libraries (ALASQL) to get a list of all the new’s links on the HackerNews web site and export those to Excel. We are going to do this with 2 lines of code:
:load alasql
alasql("select href,innerText INTO XLSX('news.xlsx',{headers:true}) from ? where className='storylink'",[[...page.document.all]])
That’s pretty cool!
When you inject the console into the page, it loads an external page (make sure you allow pop ups!) and creates a new variable page within the console. This variable is a reference to the JavaScript window object of the host page, so you can access any properties or methods of that page (i.e. page.document, page.location, etc, etc).
Unlike the sandbox settings of the host page, however, the console has access to any web page, JavaScript library or Network endpoint (via WebSockets or XMLHttpRequest).
Hopefully, this gives you some idea of the power of the Universal Console.
Electron Hosting
In the title of this article, I mentioned that the Console is also available in environments other than just the browser. First, lets look at how we host it in Electron. The first thing you need to do is download Electron via
Electron_Build cross platform desktop apps with JavaScript, HTML, and CSS._electron.atom.io
Once you have Electron installed and running, you need to create a new Electron Application, which in this case is a simple as creating a folder(call it console or whatever you like) on your machine and adding these 3 files to the folder:
package.json
{"name": "console","version":"1.0.0","main": "index.js"}
index.js
const {app, BrowserWindow} = require('electron')const path = require('path')const url = require('url')
let win
function createWindow () {win = new BrowserWindow({width: 800, height: 600})win.loadURL(url.format({pathname: path.join(__dirname, 'console.html'),protocol: 'file:',slashes: true}))
win.on('closed', () => {win = null})}
app.on('ready', createWindow)
app.on('window-all-closed', () => {if (process.platform !== 'darwin') {app.quit()}})
app.on('activate', () => {if (win === null) {createWindow()}})
console.html
<html><meta http-equiv="X-UA-Compatible" content="IE=Edge"><head><title>Console</title></head><body></body><script src="https://s3.amazonaws.com/mparsons/jsconsole/js/inject.js"></script></html>
Once you have created these files, simply drag the folder on the electron executable to run it:
You should see something like the following:
And then you can start typing any commands that reference the Electron, Chrome Browser or Node.js objects:
NW.js Hosting
Hosting in NW.js is just as easy. Download the application from here:
As in the Electron example, create a local folder and add these two files:
package.json
{"name": "console","version":"1.0.0","main": "console.html"}
console.html
<html><meta http-equiv="X-UA-Compatible" content="IE=Edge"><head><title>Console</title></head><body></body><script src="https://s3.amazonaws.com/mparsons/jsconsole/js/inject.js"></script></html>
Again, just drag the folder onto the nw executable to run it and start typing commands to access the NW.JS environment:
Note that both NW.JS and Electron host Node.js so you can use the Universal Console to script against the Node.js context in both these applications:
Finally, we come to my most frequent usage of the Universal Console …
Hosting the Console in a Window’s Application.
You might say that both NW.JS and Electron are Window’s Applications and you would be correct. However, there are a lost of scenarios where Electron, NW.js as well as Node.js fall short in terms of native Window’s functionality, in particular, COM Automation and specifically Microsoft Office integration. Most businesses still run Microsoft Excel, Word, Outlook, etc and I wanted a way to script these applications using my favourite language, JavaScript. While the Window’s Developer Ecosystem has a ton of ways to build and extend Office Applications, they usually require a ton of dependencies and skills in using C# or Visual Studio, which in my case is overkill if I just want to do something quick and easy. So, we are going to host the Universal Console in a Window’s Application using a technology that already exists on every Window’s machine — the Window’s Scripting Host, and in particular, the HTML Application, or HTA:
HTML Application - Wikipedia_On the other hand, an HTA runs as a fully trusted application and therefore has more privileges than a normal HTML file…_en.wikipedia.org
This is as simple as creating a text file called console.hta on your local machine:
console.hta
<html><meta http-equiv="X-UA-Compatible" content="IE=Edge"><head><title>Console</title><HTA:APPLICATION ID="oHTA"APPLICATIONNAME="Console"BORDERSTYLE="normal"CAPTION="yes"CONTEXTMENU="no"ICON="favicon.ico"INNERBORDER="no"MAXIMIZEBUTTON="yes"MINIMIZEBUTTON="yes"NAVIGABLE="yes"SELECTION="yes"SHOWINTASKBAR="yes"SCROLL="no"SCROLLFLAT="yes"SINGLEINSTANCE="yes"SYSMENU="yes"VERSION="1.0"WINDOWSTATE="normal"/></head><body></body><script src="https://s3.amazonaws.com/mparsons/jsconsole/js/inject.js"></script></html>
That’s it! Save this file on your local machine and double click to run it:
It works the same as in other environments with the important distinction that it now has access to the entire Window’s Ecosystem via COM integration. That means, you can do stuff like talk to Excel, Word, script Internet Explorer to scrape web sites or use third party applications that have a COM (or OLE Automation Style Interface), like Adobe, AutoDesk, Microsoft Dynamics, etc. Indeed, you will find that most Window’s Applications can be accessed and programmed this way. In this simple example, I am going to load Excel and read and write some simple values using the Excel Object Model. As a security measure, Window’s ask for permission to allow this:
This show’s how easy it is to script Excel:
Finally, in this next example, I am going to load the contents from an external Web Site and paste the values into the spreadsheet:
So the following 6 lines of code are all that’s needed to perform this task:
//Load our data from the web:load https://news.ycombinator.com/newest
//Assign it to a variable for easy reference [Optional]var doc=docs["https://news.ycombinator.com/newest"]
//load our web page scaping library:load alasql
//Run our query to scrape the datavar data=alasql("select href,innerText INTO TAB({headers:true}) from ? where className='storylink'",[[].slice.call(doc.all)])
//copy it to the clipboardclipboardData.setData ("Text", data);
//Paste it into Excelwb.ActiveSheet.Range("A1").PasteSpecial()
Hopefully, this demonstrates the power of our Universal Console. You can access an early version here and I hope to document and share the code on Github soon.
Universal Console_By Mike Parsons_s3.amazonaws.com
If you like this article, please give it a recommendation!
Thank you!