Even before the rise in virtual and hybrid festivals, Shift72 was a well-regarded leader in the industry — as their software was used as a tool for virtual press and industry components of events.

Finding the Least Expensive Option for Film Festivals to Go Hybrid with Virtual Screenings

Third-year in a row, the NYIFF will be virtual, from May 7-14 (screening via Shift72; some films will be geo-blocked outside of the US). However, there will be an in-person event to screen the closing-night film, the documentary The Beatles and India ...

NYIFF 2022: Five debut Indian films to watch at the New York festival

New Zealand Rugby Commercial (NZRC) has today announced the appointments of Gina Brogi and Rob Fyfe to its Board of Directors. Brogi, a past President of Global Content Distribution for 20th Century Fox,

Gina Brogi and Rob Fyfe appointed as Board Directors of NZ Rugby Commercial

This week in Winners’ Insight, Dani Simpson talks with Shift72′s head of growth marketing Damian Bartolomucci and digital marketing manager Nathan Bregmen. Shift72 won the Marketing Award.

On-demand video entertainment platform wins WCOC marketing award

Low – Fi assembles musicians and singers from all corners of the world for live shows in Copenhagen, creating unique and intimate experiences for music lovers. At this critical time, Low – Fi has arranged a series of online concerts on their Facebook ...

Copenhagen Lockdown Events: Art, classes, dancing, singing, concerts, quizzes and some fun too

Programmes will be accessible through Open City’s unique streaming platform, in partnership with Festival Scope and Shift72. Open City Docs was founded at University College London by the academic Michael Stewart in 2010, as a means to celebrate the art ...

Open City Documentary Festival in London announces its 2020 edition

SHIFT72

shift72

What Are the Top Startups in Oceania?

This is the first post in our series, “Getting APIs on the Blockchain”. There is a famous David Foster Wallace commencement speech titled “This is Water” where he offers a little parable of fish discussing water and briefly explicates the title with: “The most obvious important realities are often the ones that are hardest to see and talk about.” In the first blog post of this series, we will be talking about APIs.¹ APIs are everywhere — they permeate our digital world. The concept of an API is so inescapable in computer software that it’s ironically difficult to appropriately define it. To further this difficulty, the term (API) has superseded its original meaning on quite a few occasions. Application Programming Interface What is an API? (Or: a very brief history of APIs) One could argue that the concept of an API was independently discovered several times, its early history being intrinsically linked to the instantly ubiquitous concept of a . That is, it’s difficult to discuss APIs separately from the (very fundamental) concepts of and in computer science. subroutine library modular programming abstraction This idea of a subroutine library was first formalized in 1948 by Herman Goldstine and John von Neumann in their widely-circulated, multi-volume treatise : Planning and Coding of Problems for an Electronic Computing Instrument² The importance of being able to do this [permit ] is very great. It is likely to have a decisive influence on the ease and the efficiency with which a computing automat of the type that we contemplate will be operable. This possibility should, more than anything else, remove a bottleneck at the preparing, setting up, and coding of problems … we envisage that properly organized automatic, high speed establishment will include an extensive collection of such subroutines… I.e. a “library” of records … in memory. the use of a coded sequence of a problem as a single entity, when that problem occurs as part of a more complicated one Essentially, Goldstine and von Neumann argued that most programs would have to utilize existing subroutine libraries in order for computers to grow to the capacity envisaged at the time (and would otherwise, due to errors and other issues, “be quite dangerous”). Subroutines were proposed as an essential method for speeding up development and decreasing errors by way of decreasing the amount of new code to be written. The treatise, unfortunately, didn’t offer much in the way of implementation specifics, instead opting for more of a theoretical and high-level description of subroutine construction and use. In a well-known presentation delivered by Joshua Bloch entitled “A Brief, Opinionated History of the API,” Bloch bluntly puts it: “they were peddling vapourware”.³ Goldstine and von Neumann’s preparatory routine … was a surprisingly pedestrian piece of work. It would have required extensive operator intervention and it is difficult to imagine that it would have ever been a satisfactory procedure in practice. — from Theory to Practice: The Invention of Programming, 1947–51 That is why, in that same presentation, Bloch opines that Maurice Wilkes, David Wheeler, and Stanley Gill are the true creators/discoverers of the API. Or — more accurately — the proto-API, given that the term was never actually used by them. The researchers built the EDSAC (an early British computer) and later authored Preparation of Programs for Electronic Digital Computers in 1951, in which subroutine libraries were effectively introduced.⁴ A few months after writing the initial orders and upon completing his first “real program”, Wilkes would write: By June 1949 … I was trying to get working my first non-trivial program … the realization came over me with full force that a good part of the remainder of my life was going to be spent in finding errors in my own programs. people had begun to realize that it was not so easy to get programs right — from Memoirs of a Computer Pioneer, MIT Press, 1985, p. 145 Wilkes’ fear of errors in his own programs largely inspired his development of subroutines as an organizing principle. The first time the term Application Programming Interface (API) was actually used was in 1968, in a paper⁵ about designing remote computer graphics libraries. In the abstract, the authors state: “it has been adequately demonstrated that computer graphics systems need not require the dedication of a large scale computer for their operation. Computer graphics has followed the trends of computing in general, where … has become a key phrase”. That is, the authors note that (by the late 1960s) it became clear to computer scientists that computer graphics systems . (The authors address how to properly interface with such systems in their paper.) remote access encapsulated and modular made more sense When searching for “Application Programming Interface” in Google Scholar for the years 1970–1995, we find the term used in numerous contexts (far beyond a simple wrapper for a subroutine): APIs for interacting with external systems (e.g. storage devices) APIs for accessing network functions nd, an extensive literature on APIs for databases By the early 1990s, APIs quickly evolved to very generally denote the communication layer between software abstractions (local or remote). Indeed, one definition from 1990 states an API is “a set of services available to a programmer for performing certain tasks”.⁶ Another from 1993 states an API is “a term for any language and format used by one computer program to help it communicate with another program”.⁷ The notion of an API expanded its meaning yet again with the dawn of web APIs. As opposed to what we just previously discussed—APIs that can be roughly categorized as and needing to be hardware/software compatible with whichever system they are used by — APIs are interfaces that connect publicly exposed endpoints of a network, typically via a predefined message-passing system and most commonly accessed using the HTTP protocol. library APIs web In his thesis (in which Representational State Transfer⁸ is introduced), Roy Fielding distinguishes between APIs and : library-based network-based APIs The design of the Web had to shift from the development of a reference protocol library to the development of a network-based API, extending the desired semantics of the Web across multiple platforms and implementations. Basically, . It didn’t matter what kind of machine the service operated on or which language(s) it was implemented in, it now only mattered that it’s services were accessible via the web (most commonly by means of an HTTP-based web server). Today, a web API is the most common connotation of “API”. network-based APIs offered a further abstraction from library-based APIs, because the implementation of the interfaced service was further abstracted and black-boxed from the interfacing user The earliest uses of web APIs were for sales and commerce management.⁹ SalesForce launched the first web API in 2000 under the umbrella of arguably the first SaaS product. Later in the same year, eBay launched the eBay API in order to “openly provide the tools that developers need to create applications based on eBay technology”¹⁰ by way of standardizing how applications would interface with eBay (since developers were already hacking away at eBay’s website and using the data in their own applications). In 2003, del.icio.us — an early social media site — offered users a simple API for sharing and tagging their internet bookmarks. Users had read/write access to bookmarks on del.icio.us via REST-like urls (e.g. http://del.icio.us/tag/[tag name]). In 2004, Flickr launched their API which allowed users/developers to, among other things: upload images, manage their “favourites” lists, query for users, and search photos. The launch of the API was motivated by the fact that Flickr couldn’t keep up with demands for their services and “established the API as a self-service way to deal with business development” which ultimately resulted in “Flickr quickly becom[ing] the image platform of choice for the early blogging and social media movement by allowing users to easily embed their Flickr photos into their blogs and social network streams”.¹² The launch of the Flickr API was the catalyst for “BizDev2.0”. … a small startup … wanted an introduction, but I said we (that is, Flickr) probably didn’t have the space on our schedules to engage with them in any meaningful way, but that they should feel free to apply for a Commercial API key and build something off the API. “BizDev 2.0” I call it. — Caterina Fake, co-founder of Flickr In 2006, Facebook, Twitter, and Google followed suit, launching their own APIs in hopes of encouraging BizDev2.0 on their respective platforms. And then came cloud computing … Amazon S3 is storage for the Internet. It’s designed to make web-scale computing easier for developers … It gives any developer access to the same highly scalable, reliable, fast, inexpensive data storage infrastructure that Amazon uses to run its own global network of web sites. The service aims to maximize benefits of scale and to pass those benefits on to developers. — from the AWS press release (March 14, 2006) The capability of web APIs did a quantum leap with the introduction of Amazon Web Services (AWS). AWS and its three initial offerings (S3, SQS, and EC2)¹² launched in mid 2006. This was the dawn of cloud computing. Amazon Web Services succeeded in abstracting away much of the day-to-day work of software engineers, making essential and complex services (like computing, storage, networking, and deployment) available via simple web API calls (albeit for a fee). With AWS, a developer could set up and deploy a server (for example) in minutes, something that would have previously taken days or weeks. Cloud computing quickly revolutionized the way developers build applications on the web. Where are we now? The role of software engineering has changed quite a bit, and coding isn’t even a huge part of it anymore … You know how in the old days people used to write in assembly code? And then we abstracted upon that, so more people started writing C++? And then people started abstracting upon that, and writing Java. And now we have Python, and we’ve abstracted upon that too… Most top apps [these days] support so much functionality, that it’s pretty much beyond the knowledge of a single person to handle all of those anymore. Now programmers are gluing pieces together through API endpoints and frameworks. — Ex-Facebook/Ex-Google tech lead and famous YouTuber, TechLead¹³ Nowadays we take for granted that software applications can and do communicate with each other with relative ease via each other’s APIs. Developers can now build complex applications at a rate never before seen. Integrating existing services to their applications through APIs has allowed developers to build increasingly complex and capable applications, which has led to the rise of giant Web services and mobile applications. Because of this, APIs are often referred to as the glue that holds the digital world together. Providing in-demand APIs has become quite the lucrative business, by way of businesses monetizing their data and services. As a result**, the concept of an API has transcended its meaning yet again.** The term no longer refers to the technical implementation of an interface, but to a full-fledged product that often includes the service it wraps.¹⁵ Global enterprises that provide APIs generate 25% of their organizational revenue from APIs on average.¹⁶ Companies such as Salesforce, Expedia, and eBay have reported to generate the majority of their revenue through APIs¹⁷, and even entrenched industries such as banking are expected to be disrupted by this movement.¹⁸ We are now at the cusp of fully API-centric business models.¹⁹ As APIs continue to encapsulate increasingly higher and higher abstractions of software services, the meaning of “API” will continue to evolve. What’s to come with the rest of this blog series? You might be wondering at this point when are we to address topics suggested by the second half of our series title (“… on the Blockchain”). Well, there is a very critical problem we introduce in our next blog post called “The API Connectivity Problem” which will take the remainder of this series to fully explicate and map out a solution to. Briefly, existing APIs are not natively compatible with decentralized applications — a blockchain network is essentially walled-off from communicating with these systems and protocols that permeate the rest of our digital world. How do we bring these interfaces onto the blockchain? How do we get blockchain applications to safely interact with the richness offered by existing APIs? Stay tuned for the rest of our series! Footnotes
[1]: Of course, Wallace’s speech was tender and moving whereas we’ll just be discussing APIs (for now), so please excuse the abrupt change in tone…
[2]: Goldstine, Herman Heine, John Von Neumann, and John Von Neumann. “Planning and coding of problems for an electronic computing instrument.” (1947).
[3]: [4]: Wheeler, Maurice V., and Stanley Gill. The preparation of programs for an electronic digital computer: with special reference to the EDSAC and the use of a library of subroutines. Addison-Wesley, 1951.
[5]: Cotton, Ira W., and Frank S. Greatorex Jr. “Data structures and techniques for remote computer graphics.” Proceedings of the December 9–11, 1968, fall joint computer conference, part I. 1968.
[6]: Malamud, Carl. Analyzing Novell Networks. John Wiley & Sons, Inc., 1991.
[7]: Bardige, Stephen I. “A Glossary of Computer Terms and Definitions.” Judges J. 32 (1993): 72.
[8]: REST is a very common software architecture style that defines a set of constraints to be used for creating web services.
[9]: [10]: [11]: [11]: [12]: S3, SQS, and EC2 are Amazon’s storage, queue, and compute services offered remotely via API (also known as “cloud computing services”).
[13]: [14]: MuleSoft, “Connectivity benchmark report,” 2020.
[15]: Collins, G., and D. Sisk. “API economy: From systems to business services.” TechTrends (2015).
[16]: MuleSoft, “Connectivity benchmark report,” 2019.
[17]: Iyer, Bala, and Mohan Subramaniam. “The strategic value of apis.” Harvard Business Review 1.7 (2015): 2015.
[18]: R. Narain, A. Merrill, and E. Lesser, “Evolution of the API economy: Adopting new business models to drive future innovation,” IBM Institute for Business Value, 2016.
[19]: Capgemini, Efma. “World FinTech Report 2019.” (2019). https://www.infoq.com/presentations/history-api/ https://history.apievangelist.com/ https://www.ecommercetimes.com/story/5403.html https://history.apievangelist.com/#Social https://history.apievangelist.com/#Flickr https://youtu.be/yzhnRt6ZDKM Previously published . here

What is the Genuine Value of APIs

Photo by on Pietro Jeng Unsplash A deliberate mixture of metals and other elements makes stronger and/or more versatile metals… When your company provides video streaming to global entertainment brands like and it’s crucial for software developers to operate with unified source code and workflows, and to do so in the most platform-agnostic way possible. technology Festival De Cannes SXSW This is the challenge faced by the R&D team at , as we develop video streaming apps for a growing list of global entertainment brands like the ones mentioned above. We started exploring as one of the obvious solutions but were initially unsure how effective it would be, since hybrid apps, which rely on a heavy layer, rarely present with good UX. On the other hand, we soon found that the based framework can achieve native-level performance, for those powered components have not shown noticeable performance degradation in comparison to those programmed in . SHIFT72 React Native WebView React JavaScript Objective-C We have been using for almost a year now and things have been going well. We released our first powered app on and are very close to releasing our new mobile apps. It seems like a fitting time to review our development journey and provide some insight for those considering as we approach the one year mark. Whether the way is rugged or easy? Here is the report. React Native React Native Apple TV React Native The fundamental difference between React and React Native Whether using or , developers rely on for UI and for logic. This is done through _virtual DOM_s that represents nodes and a reconciliation algorithm that dictates the UI update in an on-demand manner. React React Native JSX JavaScript JSX More native than web Although and share this commonality, there is a fundamental difference: doesn’t use to render elements; instead, it relies on and to draw actual pixels on the user’s screen. This differs from where final modifications are passed onto real ,_ think about an tag defined in file. In order to display the designated image on , the tag is actually backed by a native class , which renders a with the properties contained in passed through. It is this encapsulation of underlying UI frameworks, that makes more consistent across various platforms. If were to expose platform specific APIs directly, we would end up with ad-hoc ( ) code for different platforms, which fundamentally contradicts our primary intention — to create a concise and converged code base. React Native React React Native HTML Cocoa Touch Android UI React DOM_s . To illustrate this difference in HTML <Image/> JSX iOS RCTImageView UIImageView props React Native React Native JavaScript With this difference understood, we progressed forward onto some real work. Implementing the Web API Layer Luckily for us is the official language of This means the battle-hardened source code of our existing web system can largely be re-used and the development of new API requests and model layers can easily be achieved by copy-and-pasting the sub-directory. This opens up more time for valuable and meaningful work. JavaScript React Native. /api We also found that a wider range of projects on GitHub could be used. However, since doesn’t use , anything that manipulates on object can not work. For these situations we can only keep the pure logic of the destined project, such as a protocol parser for example. Because of this, it’s very helpful if the project draws a clear line between the logic and UI layer. JavaScript React Native HTML document Implementing the UI Layer This is where we encountered our first real obstacle: implementing the UI layer for the app. At the moment, (0.46.x) appears to not include a full fledged focus engine, nor parallax animation. Due to this, we had to tinker until we found a way to fully support remote controls. Luckily for us, again, ’s intuitive logic flow and modularized source code made adding this feature a lot easier than expected. We’ll leave the modification of for another article, but to showcase the end result, see below. Apple TV React Native TVOS React Native React Native Next, we added more effects, per the requirement for mobile platforms, and rarely noticed lagging animation or sluggish transition in production; however, we did see slow animation in development build, which was likely . caused by the debugger Following the same design pattern of we encapsulated repeatedly used UI elements into s, as this ensures those elements only need to be developed once, and then can be used again anywhere. React, Component Implementing multi-tenancy mechanisms The tension we are facing differs from a lot of other R&D teams who normally work on builds across the development stages (development, staging, product, etc.), in which case can be used to define parameters for these different builds. react-native-config At we are required to deploy executable for various clients, and in some cases the executable needs to enlist meta data from multiple clients. Our clients also generally demand different tints, layouts and even UX flows, which can hardly be reflected by static configuration entries. So we don’t rely on , but rather on a combination of native multi-tenancy mechanisms (i.e., and ) and module dependency. This has worked well for us so far. SHIFT72 react-native-config Xcode targets build variant JavaScript To do so, we indicate the for different targets or variants. Then these entry files decide which components it relies on. Each entry also contains a object that mandates minor presentation tweaks on general purpose components, like tint and layout, if applicable. xxx.index.js index context Component Trees Conclusion With now using for app development, we can go shopping for the best parts from the world of and the world of This is great whenever we are looking for a solution. An interesting discovery we also made along the way is that developers with strong backgrounds in web can help the app team by fixing components, optimizing code and revising some obscured implementations, such as nested . Conversely, the know-how also helps our app developers establishing their web development capabilities. This highlights a key point: has not only made our apps cross-platform, but also our team. After all, if , then maybe somehow the same principle works on apps and the people who make them. SHIFT72 React Native React Native. JavaScript Promise React Native a deliberate mixture of metals and other elements can make stronger and/or more versatile metals References _TLDR => Having set up a number of React Native projects, learnt the quirks and pitfalls, I’ve started to settle on a…_medium.com How I set up my React Native Projects _Update. It’s been almost a year since I wrote this story. You can read my latest finding on this topic in the story…_hackernoon.com Image Caching in React Native _React Native is an exciting framework that enables the developer to easily build native apps using a common web…_moduscreate.com React Native ListView with Section Headers - Modus Create _Why should we add multiple targets to our apps?_medium.com Adding multiple target pipelines for React Native Apps (and Fastlane CircleCI deployment) pt. 1 _This problem has been bugging me for a while now and I am stuck. I want to animate the color property of my text when a…_stackoverflow.com Animate text color change in React-Native _Internally, React uses several clever techniques to minimize the number of costly DOM operations required to update the…_reactjs.org Optimizing Performance - React _React provides a declarative API so that you don't have to worry about exactly what changes on every update. This makes…_reactjs.org Reconciliation - React _React provides developers with many methods or “hooks” that are called during the life-cycle of an component, which…_medium.com Understanding React — Component life-cycle _Component state is a way of holding, processing and using information that is internal to a given Component and allows…_medium.com Understanding ReactJS — setState Special thanks to @ for the surgeries. Mathew Joubert

Build VOD Apps Using React Native

A classification approach with natural language processing in R Audience This short blog post is intended for someone who has basic understanding of the natural language processing algorithm in addition to familiarity with machine basics, particularly in classification techniques, and is keen to learn the actual integration of the two in R. We will go over detailed data preparation as well as model optimization. It is also recommended to review for a brief introduction on word2vec, learning this post word2vec. Objective The objective is to learn how the Dow Jones Industrial Average would be affected by daily popular news on . Reddit Dataset The datasets can be found . The simplified dataset is consisted of daily Dow Jones data and top daily headlines from Reddit WorldNews Channel for the past 8 years. The target variable is Label (0 is down and 1 is up). here Screenshot of the stock news data set Approach We will use the algorithm to first create word vectors of 100 dimensions on all the headlines. We will then transform the headlines into sentence vectors by averaging the word vectors that every sentence is consisted of. Finally we will use a variety of classification algorithms to predict whether Dow Jones Industrial Average would go up or down. word2vec Data Preparation 1…Remove undesired characters , including punctuations and upper cases 2…Remove English stop words such as “i,” “my,” “myself” Screenshot of the data set with unwanted features removed 3…Input cleaned headlines into model to obtain word vectors word2vec Screenshot of the word vectors generated from cleaned headlines 4…Apply and average word vectors to all headlines 5…Return 2-dimension vector representations on headlines using t-SNE 6…Create dataset with target variable and 2-dimension vectors Screenshot of the finished data set Model Evaluation Here we will use the package in R to evaluate the prediction accuracy on a variety of algorithms, including CART (rpart), logistic regression (glm), linear discriminate analysis (LDA), k-Nearest Neighbors (kNN), Support Vector Machine.with Radial Basis Function (svmRadial), and eXtreme Gradient Boosting (xgbLinear). caret It is observed that all algorithms above have unsatisfying performance, with close to 50% accuracy and poor Kappa values. However, SVM with the Radial Basis Function kernel tends to perform better than other algorithms, with a 95% confidence interval on accuracy between 51% and 55%. Performance on various classification algorithms As a result, we will explore a few optimization options. Performance Optimization 1…Stacking Multiple Models We will first try to stack all the models to see if performance will improve. Unfortunately, not much improvement is observed. 2…Averaging All Headlines Instead of treating top headlines as individual variables, we will combine all headlines to create one set of 2-dimension vector. Performance on the algorithms with averaged headlines A slight increase of 0.1% in mean accuracy is observed in SVM algorithm. 3…Shifting Target Variable We will try shifting the target variable back 1–7 days, as daily news may have lagged effects on . Alternatively, we can also shift the target variable forward a couple of days, however, this approach will contradict with the objective of stock marketing performance using daily top news. stock performance predicting Performance on the algorithms with 3 day lag on target variable We can observe that CART (rpart) now has the best performance in terms of mean accuracy at 53.66%. There is also another slight increase of 0.1% in mean accuracy is observed in the SVM algorithm. 4…Tuning Model Parameters Here we will demonstrate how to tune parameters of the CART and SVM algorithms to see if we can increase the accuracy on our predictions. CART In rpart, the tuning parameter is , or . We will examine the accuracy on different values of . The optimal cp value is found to be 0.15, as shown below. cp complexity parameter cp SVM In SVM with Radial Basis Function, the tuning parameters are and ( ). We will examine the accuracy on different values of and . The optimal values are shown below, with and . sigma C cost sigma C sigma = 0.05 C = 3 R Code library(devtools)library(tm)library(ggplot2)library(text2vec)library(rword2vec)library(wordVectors)library(tidyverse)library(Rtsne)library(caret)library(mlbench)library(caretEnsemble) setwd("~/Desktop/stock")#Simple Visualization on DJIA datastock <- read.csv("DJIA_table.csv")stock$Date <- as.Date(stock$Date) ggplot(data=stock,aes(x=Date,y=Open))+geom_line()+labs(title="Dow Jones Industrial Average") #Read the simplified data setdata <- read.csv("stock.csv") #Remove unwanted columns for natural language processingtext <- data text$Date <- NULLtext$Label <- NULL #Remove unwanted characters/punctuations/upper-casesfor (i in 1:25){text[,i] <- gsub('b"',"",text[,i])text[,i] <- gsub("b'","",text[,i])text[,i] <- gsub("[[:punct:]]", "", text[,i])text[,i] <- tolower(text[,i])} write.table(text,"text.txt") #Rename column namesnew_text <- subset(text,select=paste("Top",1,sep=""))new_text <- setNames(new_text,"x") for (i in 2:25){new_text2 <- subset(text,select=paste("Top",i,sep=""))new_text2 <- setNames(new_text2,"x")new_text <- rbind(new_text,new_text2)} write.table(new_text,"text.txt") final_text <- new_text #Remove stopwordsstopWords <- stopwords("en")'%nin%' <- Negate('%in%') train <- lapply(final_text$x, function(x) {t <- unlist(strsplit(x, " "))t[t %nin% stopWords]}) train.df = as.data.frame(do.call(rbind, train))train.df$x <- do.call(paste, c(train.df[,1:45], sep=" ")) finaltext <- subset(train.df,select="x") write.table(finaltext,"text_data.txt") #Input words to word2vecmodel = train_word2vec("text_data.txt", output="vec.bin",threads = 4, vectors = 100, window = 12, min_count = 5, iter=10, force=TRUE) #Plot the word vectorsplot(model)#Convert binary to text formatbin_to_txt("vec.bin","model1text.txt") #Remove first rows manually in the .txt file #Convert .txt to .cvsm1 <- read.table("model1text.txt",header = F,quote = "", row.names = NULL, stringsAsFactors = FALSE) colnames(m1)[1] <- "word" #Apply word vectors to headlines and return 2-dimension representations with t-SNElabel <- subset(data,select="Label")labeltext <- cbind(label,text)labeltext$id <- 1:nrow(labeltext)labeltext$id <- as.factor(labeltext$id) for (i in 2:24){s <- strsplit(labeltext[,i], split= " ")new <- data.frame(V1 = rep(labeltext$id, sapply(s, length)), V2 = unlist(s))colnames(new) <- c("id","word")new <- merge(new,m1,by="word")new2 <- aggregate(V2 ~ id, new, mean)for (j in 4:102){new3 <- aggregate(new[,j] ~ id, new, mean)new2 <- merge (new2,new3,by="id")}colnames(new2)[2:101] <- paste("V", 1:100, sep="")tsne <- Rtsne(new2, dims = 2, perplexity=100, verbose=TRUE, max_iter = 500)t = as.data.frame(tsne$Y)colnames(t) <- c(paste("top",i,"x",sep=""),paste("top",i,"y",sep=""))labeltext <- cbind(labeltext,t)} write.csv(labeltext,"labeltext.csv") #Add back Label informationmydata <- labeltextmydata[,2:27] <- NULL mydata$Label[mydata$Label 0]<- "No" 1]<- "Yes"mydata$Label[mydata$Label #Classfication Modelscontrol <- trainControl(method="repeatedcv", number=10, repeats=3, savePredictions=TRUE, classProbs=TRUE)algorithmList <- c('rpart','glm','lda','knn','svmRadial','xgbLinear')set.seed(1)models <- caretList(Label~., data=mydata, trControl=control, methodList=algorithmList)results <- resamples(models)summary(results)dotplot(results) #-----Performance Optimization #1: Stacking Multiple Models-----#Model CorrelationsmodelCor(results)splom(results)#Model Stacking with GLMstackControl <- trainControl(method="repeatedcv", number=10, repeats=3, savePredictions=TRUE, classProbs=TRUE)set.seed(2)stack.glm <- caretStack(models, method="glm", metric="Accuracy", trControl=stackControl)print(stack.glm) #-----Performance Optimization #2: Averaging All Headlines-----mydata2 <- mydatamydata2$headline <- rowMeans(mydata2[,2:45], na.rm = TRUE)mydata2[,2:45] <- NULL control <- trainControl(method="repeatedcv", number=10, repeats=3, savePredictions=TRUE, classProbs=TRUE)algorithmList <- c('rpart','glm','lda','knn','svmRadial','xgbLinear')set.seed(1)models <- caretList(Label~., data=mydata2, trControl=control, methodList=algorithmList)results <- resamples(models)summary(results)dotplot(results) #-----Performance Optimization #3: Shifting Target Variable-----shift <-subset(mydata,select="Label")shift1 <- as.data.frame(shift[-1,])shift2 <- as.data.frame(shift[-c(1,2),])shift3 <- as.data.frame(shift[-c(1,2,3),])shift4 <- as.data.frame(shift[-c(1,2,3,4),])shift5 <- as.data.frame(shift[-c(1,2,3,4,5),])shift6 <- as.data.frame(shift[-c(1,2,3,4,5,6),])shift7 <- as.data.frame(shift[-c(1,2,3,4,5,6,7),]) mydata1 <- mydata[-1989,]mydata2 <- mydata[-c(1988,1989),]mydata3 <- mydata[-c(1987,1988,1989),]mydata4 <- mydata[-c(1986,1987,1988,1989),]mydata5 <- mydata[-c(1985,1986,1987,1988,1989),]mydata6 <- mydata[-c(1984,1985,1986,1987,1988,1989),]mydata7 <- mydata[-c(1983,1984,1985,1986,1987,1988,1989),] mydata1$Label <- NULLmydata2$Label <- NULLmydata3$Label <- NULLmydata4$Label <- NULLmydata5$Label <- NULLmydata6$Label <- NULLmydata7$Label <- NULL mydata1 <- cbind(shift1,mydata1)colnames(mydata1)[1] <- "label1day"mydata2 <- cbind(shift2,mydata2)colnames(mydata2)[1] <- "label2day"mydata3 <- cbind(shift3,mydata3)colnames(mydata3)[1] <- "label3day"mydata4 <- cbind(shift4,mydata4)colnames(mydata4)[1] <- "label4day"mydata5 <- cbind(shift5,mydata5)colnames(mydata5)[1] <- "label5day"mydata6 <- cbind(shift6,mydata6)colnames(mydata6)[1] <- "label6day"mydata7 <- cbind(shift7,mydata7)colnames(mydata7)[1] <- "label7day" control <- trainControl(method="repeatedcv", number=10, repeats=3, savePredictions=TRUE, classProbs=TRUE)algorithmList <- c('rpart','glm','lda','knn','svmRadial','xgbLinear')set.seed(1)models <- caretList(label3day~., data=mydata3, trControl=control, methodList=algorithmList)results <- resamples(models)summary(results)dotplot(results) #-----Performance Optimization #4: Tuning Model Parameters-----set.seed(3)control <- trainControl(method="repeatedcv", number=10, repeats=3, savePredictions=TRUE, classProbs=TRUE) #Tuning cp in CARTrpartGrid <-  expand.grid(cp = c(0.01,0.05,0.1,0.15)) rpartFit <- train(label3day ~ .,data = mydata3,method = "rpart",tuneLength = 10,tuneGrid=rpartGrid) rpartFitggplot(rpartFit) #Tuning sigma and C in SVMsvmGrid <-  expand.grid(sigma = c(0.01,0.05,0.1,0.15),C = c(1,3,5,7,9)) svmFit <- train(label3day ~ ., data = mydata3,method = "svmRadial",trControl = control,verbose = FALSE,tuneGrid = svmGrid) svmFitggplot(svmFit) Questions, comments, or concerns?jchen6912@gmail.com

Stories, News & Mentions about SHIFT72

Categories

Trending Topics

Stories, News & Mentions about SHIFT72

Shift72’s Stories on HackerNoon

Shift72's News & Mentions

Categories

Trending Topics