Visualizing Bitcoin prices moving averages using Dash

Everyone seems ecstatic about Bitcoin right now, and for a while I’ve been interested in writing about — the cool frontend tool for building analytical applications. The reason I like Dash is it allows me to remain in my comfort zone: Python. And it boasts of being lightweight, easily customizable, you know, all those good things. Dash by plotly are a pretty useful indicator in stock trading. It’s still a as to whether Bitcoin is like a stock. As such, I’m using the term loosely here. help identify trends, provide trade signals, and smooth out price action by filtering out the “noise” from random price fluctuations. There are various variants of, and ways to calculate moving averages. Simple Moving Average is simply the average of the last values with being the period to average over. Weighted Moving Average and Exponential Moving Average are less simple in that they introduce weights and also take into consideration the fact that more recent values may provide a better indication of the trend and thus apply some form of decay to the weights. Moving averages (MAs) grey area MAs (SMA) N N (WMA) (EMA) Here, I’ll give a brief tutorial on how to visualize Bitcoin prices, the and the using Dash. SMA EMA Our first challenge is getting real-time Bitcoin prices. Thankfully, provides an that does just that. Coindesk API $ curl https://api.coindesk.com/v1/bpi/currentprice/USD.json {"time":{"updated":"Dec 31, 2017 19:26:00 UTC","updatedISO":"2017-12-31T19:26:00+00:00","updateduk":"Dec 31, 2017 at 19:26 GMT"},"disclaimer":"This data was produced from the CoinDesk Bitcoin Price Index (USD). Non-USD currency data converted using hourly conversion rate from openexchangerates.org","bpi":{"USD":{"code":"USD","rate":"13,935.3500","description":"United States Dollar","rate_float":13935.35}}} Nice. The prices are updated every minute so we need a service that calls this minute by minute. What we would like is a process that schedules the call every minute which is then executed in the background. API REST According to Heroku: Scheduling a job and executing a job are two related but independent tasks. Separating a job’s execution from its scheduling ensures the responsibilities of each component are clearly defined and results in a more structured and manageable system. As you would expect, Python is up to the task. The Advanced Python Scheduler (APScheduler) is a Python library that lets you schedule your Python code to be executed later, either just once or periodically. We also need a queuing system. We’ll use simply because it makes it easy to add background tasks to Python code on Heroku — my favorite . RQ (Redis Queue) PAAS $ pip install apscheduler rq redis We also need to install . There is a comprehensive guide . All I had to do from my Mac was and from my Ubuntu Redis here brew install redis sudo apt install redis-server We can then go on to create a process that listens to, and processes queued jobs. import osimport redisfrom rq import Worker, Queue, Connection listen = ['high', 'default', 'low']redis_url = os.getenv('REDISTOGO_URL', 'redis://localhost:6379') conn = redis.from_url(redis_url) if __name__ == '__main__':with Connection(conn):worker = Worker(map(Queue, listen))worker.work() This is simply a worker process that reads jobs from the given queues in an endless loop, waiting for new work to arrive when all jobs are done. Name the file worker.py Now we can go ahead to author a file named that would serve as our scheduler. We’ll use the class which is quite simple, and obtain a connection to our queue. scheduler.py BlockingScheduler from apscheduler.schedulers.blocking import BlockingSchedulerfrom rq import Queue scheduler = BlockingScheduler() from worker import connq = Queue(connection=conn) Having created and obtained a connection to the queue in our worker process, we can now schedule our job i.e making a REST call to Coindesk’s API. We’ll go ahead to author a file named You may want to install the following packages first utils.py $ pip install iso8601 requests And then import requests, calendar, timefrom iso8601 import parse_date def retrieve_current_price(endpoint): response = requests.get(url=endpoint) data = response.json() updatedTime = data\['time'\]\['updatedISO'\] currentPrice = data\['bpi'\]\["USD"\]\['rate\_float'\] parsed = parse\_date(updatedTime) timetuple = parsed.timetuple() parsedTime = calendar.timegm(timetuple) return Nothing groundbreaking here. We are simply making an request to the endpoint and extracting the needed information from the JSON response formatted as shown earlier. What we would like to do next is save these values to a datastore. We’ll use — a straightforward document oriented DB. http GET TinyDB $ pip install tinydb We’ll then go on to add the following snippets of code into Notice that the prices are stored in a file named utils.py bpi.db from tinydb import TinyDB, Query db = TinyDB('bpi.db')BPI = Query() db.insert({'updatedTime': int(parsedTime), 'QueryTime': int(time.time()), 'currentPrice': currentPrice}) So that looks like this utils.py import requests, calendar, timefrom iso8601 import parse_datefrom tinydb import TinyDB, Query def retrieve_current_price(endpoint): response = requests.get(url=endpoint) data = response.json() updatedTime = data\['time'\]\['updatedISO'\] currentPrice = data\['bpi'\]\["USD"\]\['rate\_float'\] parsed = parse\_date(updatedTime) timetuple = parsed.timetuple() parsedTime = calendar.timegm(timetuple) db = TinyDB('bpi.db') BPI = Query() db.insert({'updatedTime': int(parsedTime), 'QueryTime': int(time.time()), 'currentPrice': currentPrice}) return Now that we’ve defined our job, we can go ahead to schedule it in our file. Let’s dig up the file again and add the following snippet of code just underneath everything else in the file. scheduler.py from utils import retrieve_current_price endpoint = "https://api.coindesk.com/v1/bpi/currentprice/USD.json" #To attach the needed metadata we use a decorator#Coinbase publishes prices every minute (60 seconds) @scheduler.scheduled_job('interval', minutes=1)def retrieve_current_price_job(): q.enqueue(retrieve_current_price, endpoint) scheduler.start() Sweet. Notice that we first import the method that executes our REST call. This method is then added to the redis queue with and another such job is added every minute. q.enqueue(retrieve_current_price, endpoint) Running and should begin to populate the file every minute with the Bitcoin price indexes. scheduler.py worker.py bpi.db We now need to move over to visualizations using Dash. As a teaser, here is a look at the expected final output from this tutorial. Its dynamic, so this just gives a peek. Dash is written on top of , , and . Flask Plotly.js React.js Snip of the Dash app As you can see, it does use a bit of styling here and there and most of the styling code is borrowed from . That said, we won’t go over the styling. Go ahead and download the stylesheet. https://github.com/plotly/dash-wind-streaming Stylesheet for the Dash app A bit annoying to install but we also need to download the file that contains packages for the application . You can install them using requirements.txt here $ pip install -r requirements.txt Once that is done, we can go ahead to author a file named that would contain the code for our Dash application. app.py As usual, import a few modules. We’ll describe the functionality of some later on. import osimport datetime as dtimport time import dashimport dash_core_components as dccimport dash_html_components as htmlfrom dash.dependencies import Input, Output, State, Event import plotly.plotly as pyfrom plotly.graph_objs import * import numpy as npfrom scipy.stats import rayleigh from tinydb import TinyDB, Query from flask import Flask, send_from_directory We’ll go ahead and create our object which takes a instance as an argument. Dash Flask server = Flask('BitcoinMAvgs') app = dash.Dash('BitcoinMAvgs-App', server=server, url_base_pathname='/dash/', csrf_protect=False) Voila! we have a Dash application. Pretty useless I know but all we need to do now is define our and . To get a comprehensive dive into Dash head . layouts callbacks here The Dash describes what your app would look like and is composed of a set of declarative Dash components. On the other hand, make Dash apps interactive using Python functions that are automatically called whenever an input component’s property changes. There are components and components. layout callbacks core html we’ll go ahead to start defining our layout. First we define the root div app.layout = html.Div([...], className='main-div') We can then add sub-components to this div If you look at the final layout you’ll notice the banner. We’ll go ahead to add that. Replace the ellipsis in the snippet above with the code below html.Div([html.H2("Bitcoin prices with moving averages")], className='banner'), This basically creates a , sets the reference to its styling and adds a text. div header The next thing we need to do is create the container for our graph, its title, and the corresponding components. Go ahead and add the snippet beneath the banner component. html.Div([ html.Div([html.H3("Price(USD) " + str(dt.datetime.now().date()))], className='Title'), html.Div([dcc.Graph(id='bpi'),],className='twelve columns bpi'), dcc.Interval(id='bpi-update', interval=12000) ],className='row bpi-row'), A few explanations. As we’ve seen already the and are simply components as we know them. But now we have 2 core components. and . The component shares the same syntax as written in the open-source library. The element allows you to update components on a predefined interval. In our case, every or . Also note that these components have IDs. We’ll use these IDs when writing our callbacks. divs header html dcc.Graph dcc.Interval Graph plotly.py Interval 2 minutes 12000 milliseconds Beneath the that contains our , we’ll add 3 more components —all wrapped in their own each having a corresponding label. div graph Dropdowns divs html.Div([ html.P("Period"), dcc.Dropdown(id='period',options= [{'label': i, 'value': i}for i in ['5', '10', '15']], value='10')] style={'width': '31%', 'display': 'inline-block'}), html.Div([ html.P("Box plot Period"), dcc.Dropdown(id='box_plot_period',options=[{'label': i, 'value': i}for i in ['5', '10', '15']], value='10')], style={'width': '31%', 'margin-left': '3%', 'display': 'inline-block'}), html.Div([ html.P("Time frame"), dcc.Dropdown(id='timeframe',options=[{'label': i, 'value': i}for i in ['1', '2', '3', '4']], value='2')], style={'width': '31%', 'float': 'right', 'display': 'inline-block'}) These will dynamically control the for our averages, the grouping for our and the timeframe in hours for which to show the averages respectively. dropdowns period box-plots Having defined our layouts, we can then go ahead to write what happens when we interact with the layouts — callbacks @app.callback( Output(component_id='bpi', component_property='figure'),[ Input(component_id='period', component_property='value'),Input(component_id='box_plot_period', component_property='value'),Input(component_id='timeframe', component_property='value')], [], [Event('bpi-update', 'interval')]) What we have here is simply the for our interface which we will define shortly that describes declaratively its and . The are the properties of a particular component. Here, our inputs are the properties of the components that have the IDs , , and i.e our and our output is the property of the component with the ID which is our . decorator input output inputs value period box_plot_period timeframe dropdowns figure bpi graph Now, we can code up the method to load the data from our datastore and return the figure based on our arguments. Beneath our , we’ll add the code below graph decorator def get_bpi(N, bN, T):T = int(T)curr_time = time.time()begin_time = curr_time - T * 60 * 60 db = TinyDB('bpi.db') BPI = Query() data = db.search(BPI.updatedTime > begin\_time) First we need to determine what slice of the data to process. And this is controlled by the argument; . We simply query the datastore for the last hours of data. Now that we have the raw data, we want to extract the prices and also build a friendly format of the — we’ll show the and for each data point in our graph. timeframe T T timestamp hour minute for bpi in data: prices.append(bpi\["currentPrice"\]) a = dt.datetime.fromtimestamp(int(bpi\["updatedTime"\])) dtime.append(str(a.hour) + ":" + str(a.minute)) plen = len(prices) So we would like to build for our prices but remember that we have parameterized the grouping of these so that we can group our prices into various chunks — , or as you wish. box-plots box-plots 10 minute 15 minute boxtraces = []names = [] for i in xrange(0, plen, bN): y = prices\[i:i + bN\] ind = i + bN - 1 if (i + bN) > len(dtime): ind = len(dtime) - 1 name = dtime\[ind\] names.append(name) trace = Box(y=y, name=name, showlegend=False, x=\[i for j in range(len(y))\]) boxtraces.append(trace) Some explanations here. We loop through our list skipping over in steps — the parameter and for each of these data slices, we associate a label — which is the of the last datapoint in that slice. We then build a graph object using for each slice and append all those objects to a list. prices bN boxplot period timestamp Boxplot trace = Box(…) Now, we can calculate and plot our first moving average. The . The is just as its called — simple. We take an over the prices as we move across the data in chunks — which is our parameter. SMA SMA average N period SMA = []if plen > N: SMA = \[None for i in range(N - 1)\] for i in xrange(0, plen - N - 1, 1): y = prices\[i: i + N\] sma = reduce(lambda x, y\_\_: x + y\_\_, y) / len(y) SMA.append(sma) The Exponetial moving average — is slightly different and has the following formular EMA new EMA = [Closing price - previous EMA] x (2 / N+1) + previous EMA where N = period With that, we can go ahead and code up its implementation EMA = []if plen > N: EMA = \[None for i in range(N - 1)\] y = prices\[0: i + N\] avg = reduce(lambda x, y\_: x + y\_, y) / len(y) EMA.append(avg) for i in xrange(N, plen, 1): new\_ema = ((prices\[i\] - EMA\[-1\]) \* 2.0 / 11.0) + EMA\[-1\] EMA.append(new\_ema) Now to some things. plotly Just as we did for our traces, we also create a using the our and data. box-plot line-plot SMA EMA trace = Scatter( y=SMA, x=[i for i in xrange(0, plen, 1)],line=Line(color='#42C4F7'), mode='lines', name='SMA' ) trace2 = Scatter( y=EMA, x=[i for i in xrange(0, plen, 1)],line=Line(color='#32DD32'), mode='lines',name=str(N) + '-period-EMA' ) Nothing spectacular here, we’re simply creating a object and passing and properties. Scatter data styling With that done, we can create our plotly layout layout = Layout( xaxis=dict(tickmode="array",ticktext=names,tickvals=[i for i in xrange(0, plen, bN)],showticklabels=True), height=450,margin=Margin(t=45,l=50,r=50) ) traces = []traces.append(trace)traces.append(trace2) boxtraces.extend(traces) return Figure(data=boxtraces, layout=layout) The here is simply the list of our human friendly and our boxplots are grouped by so we want to use that too to set our . After which we can create a object instantiated with the and the . names variable timestamps bN tickvals Figure traces layout And that’s it! What’s left is to reference our the way. Just go ahead and copy the code below, and place it beneath all the code in stylesheet Flask app.py external_css = ["https://cdnjs.cloudflare.com/ajax/libs/skeleton/2.0.4/skeleton.min.css","https://fonts.googleapis.com/css?family=Raleway:400,400i,700,700i","https://fonts.googleapis.com/css?family=Product+Sans:400,400i,700,700i"] for css in external_css: app.css.append\_css({"external\_url": css}) css_directory = os.getcwd()stylesheets = ['BitcoinMAvgs.css'] static_css_route = '/static/' @app.server.route('{} '.format(static_css_route)) def serve_stylesheet(stylesheet): if stylesheet not in stylesheets: raise Exception('"{}" is excluded from the allowed static files'.format(stylesheet)) return send\_from\_directory(css\_directory, stylesheet) for stylesheet in stylesheets:app.css.append_css({"external_url":"/static/{}".format(stylesheet)}) This snippet of code assumes the local is located in the same directory as stylesheet BitcoinMAvgs.css app.py Finally! we can run our application. if __name__ == '__main__':app.run_server() Start your server if its not already running Redis $ redis-server and execute these in separate shells $ python worker.py $ python scheduler.py $ python app.py Your application should be accessible via [http://localhost:8050/dash/](http://localhost:8050/dash/) There we have it. Dynamic moving averages of prices without writing a single line of . The complete code can be found on Bitcoin Javascript GitHub https://github.com/Viktour19/BitcoinMAvgs Drop a comment if there are any thoughts on how to make this better. Cheers!