State of AI: How did we get here, and where are we going next?

This is the text of a talk I gave in March 2017.

The world has woken up to AI.

Here’s an illustration of this exponential trend: the number of press releases published by companies mentioning “Artificial Intelligence” skyrocketed over the last few years.

I conducted a research myself and asked my Twitter network: when AI is being used so frequently in so many different ways, does it have any valid utility in differentiating one thing to another?

They were split.

About 58% said it doesn’t really have that much use as a description anymore. It brought to mind this kind of idea that comes back to John McCarthy who said: “As soon as it works, no one calls it AI any more.”.

It’s been a six-decade journey to get here. Sixty-one years ago the first conference in AI got together at Dartmouth University, bringing together the likes of McCarthy and Herb Simon, Norbert Wiener & Martin Minsky.

At the time they thought they’d solve what was the problem of AI then, which we would now call artificial general intelligence: building a machine that could replicate human intelligence within a few years — within a decade is generally what they would say.

More, McCarthy, Minsky, Selfridge and Solomonoff

How could they be that optimistic?

There are a few reasons for this, and it’s important to put them in their historical context:

The American scientists who came together were part of what was known as the Greatest generation. They had lived through the Depression, they had seen America win the WWII. They’d become the major exporter of the world, and they had marshaled their resources to the Manhattan Project to build the Atom Bomb.

In 1956 they hadn’t yet lost the Space Race to the Soviet Union.

But, there were a few problems:

1. They didn’t really know how the human brain worked. It’s pretty hard to build something when you only have a vague understanding of the mechanics of the thing that you’re trying to emulate.
2. The way in which their math or their logic worked was very much grounded in symbolic logic or decision trees which, as we’ve seen a with recent breakthrough in statistical or probabilistic models, has not necessarily been the ideal way of delivering stuff that looks vaguely intelligent.
3. They didn’t have integrated circuits at the time.

This group of great minds faced some steep challenges. They didn’t really know what they were building, they didn’t really have the kind of maths to build it and they really didn’t have the processing power on which to run the math.

The Cycle of Innovation — Where Are We?

So much for sixty years ago. But today AI seems to not just be present but on the cusp of exploding into something significant. The question: is how do we go from that moment to a moment where we seem to be building small things and then suddenly have a moment where there seems to be rush, a ubqiuity, of development? Carlota Perez developed a powerful framework: she thinks about technology waves in cycles.

In what she describes over the course of five revolutions, technical revolution is a process where society reconfigures itself over a forty to fifty year period through installation, readjustment, and deployment.

This is where there is some kind of breakthrough technology, and combining breakthrough technology with really rare skills creates a set of opportunities.

The technology isn’t that useful, so it needs to be deployed into markets using technology push. That’s what many of us would have remembered having gadgets that didn’t really work.

During this phase financial speculators, financial capital recognize that there is a new market about to be created and because they can reconfigure themselves very quickly financial speculative capital starts to pour into this market early. It pours in ahead of the technology being widely deployed.

Re-adjustment or the “turning point” is the interim period of instability, and, ultimately, economic collapse.

The final phase is a reconfiguring stage.

This is a time of creative construction; it’s a time where the skills that are needed to make use of this breakthrough technology are much more widely disbursed.

It’s a time when consumers actively seek out solutions that depend on that technology. It’s also a time when production capital rather than purely speculative capital is applied for the build out of this new infrastructure.

This model can help us think about where we are in the cycle, and what the world is going to look like.

Find a Ford: The Model T was introduced in 1908; it was incredibly popular within five or six years. This photos from 1926 in New York City, 18 years after the introduction of Model T.

This is New York in 1926. Eighteen years after the introduction of Model T Ford. At that point, technological revolution was still in its deployment phase.

In 1908 Henry Ford had introduced the idea of Fordism and the production line. It was a brand new concept. Sales of the Ford car accelerated pretty quickly, pretty exponentially for the time.

There’s the readjustment phase: Wall Street crashed in 1929, followed by the depression and then World War II. After that deployment phase came an incredibly optimistic time in American history: they built out highways, got into air travel, holidays became a new normal, suburbs flourished, as well as the suburban shopping malls. Lots of economic, technology and business innovations were deployed across an installation platform.

With Carlota’s theory in mind, we need to ask: where are we with ICT? (AI is really part of that wave?)

We’ve seen the installation phase beginning in 1971 with Intel 4004 processor. We saw demand for rare skills. It was a small number of people involved in the tech industry, and there were companies that got a tremendous amount of market power: Intel, Microsoft, Oracle. We alongside this saw the growth of the venture capital industry starting in the late ’60s.

We’ve had our moments of readjustment. We had the dot-com bubble. Look at WebVan.com and Kozmo.com. They had classic problem of e-commerce in 2000. Not enough users on the Internet, technology infrastructure was immature, and sotware was shonky. In short, far from the prime-time we see today.

I can speak from personal experience, too. I invested in an online real estate portal in France. Seems like a great idea today. Most people today use online search for their homes. Not in 1999. No real estate agents had computers in France and none of their customers buying houses had computers. We had this enormous customer acquisition cost where we literally had to buy them the technology: PCs, modems, second phone lines.

Moore’s Law, the Sea of Data and Business Reengineered

Does Perez’s model fit where we are today? If it does, what does it mean for us in the field of AI? We do seem to have got to a stage where there are a lot of key infrastructural things in place for this. I’ll look into three.

Limits of Moore’s Law and Beyond

When I was 13 years old I had lusted after Cray-2. It weighs two and a half tons and it costs $35 million and can do 1.9 gig of FLOPS of calculation power. The PC I had at home was doing about 10,000. Interestingly the 4004 processor back from 1971, the year before I was born could add two eight bit numbers 11,000 times a second. Try running your neural networks on that.

With the continued application of Moore’s Law we get to the point where the iPhone 4 is as powerful as a Cray-2, costs $400 and it sits in the drawers in our kitchen. Apple Watch is 100 grams, $300 and is twice the raw processing force power as a Cray-2.

What I find interesting is that the industry hasn’t stood still even as we hit the physics limits of Moore’s Law. The blue curve shows us that we started to find architectural enhancements and more specialized architectures and video range of GPUs and you see their growth curve in terms of processing.

Availability of data

Data is the rocket fuel that powers the math that we do. In 2013, the world had a stock of about four zettabytes of data where 90% of it had been produced in the last two years.

Essentially in 2025, which is not so long away, the amount of data we will produce in a week will be the same as the entirety that humanity produced up until 2013 as a species. That’s quite a sea change.

Source: IDC

Prior to us all having smartphones we didn’t really create that much data; and now with smartphones, without really thinking about it, we are creating a lot of data: we’re creating Tweets and photos and videos and interactions and messages. Also, it’s not just the human data, it’s the machine data, too.

The kind of data they would generate is going to be really, really intriguing.

If we look at the growth of the CMOS sensors we see flat growth until just about the point in 2011 where iPhone sales are taking off. Since then that’s two billion shipments of CMOS sensors. We’re forecasting six billion or a 3x improvement in an eight-year period.

Source: IC Insights

We’ve also seen data sources come from other places. These are happening on top and separate to what’s happening in a consumer home with the IoT. Just looking at Earth observation data there’s a tremendous growth in the demand for sale of Earth observation data to the military and to private. There’s also growth in the number of satellites that are being sent up to observe the Earth.

Companies such as Orbital Insights and SpaceKnow are taking photos from Earth observation satellites, running them through machine learning and machine vision to make predictions about how well human companies are doing.

If the JC Penney shopping mall has fewer cars in it on a given Tuesday in February than previous Tuesdays in February that might tell you something about how well their sales are doing. Of course you can apply that to oil tankers pulling up to the docks in Mumbai or cargo being unloaded at Singapore.

The world has become software driven

Three things are happening.

1. Architecturally, software doesn’t look like a monolith of spaghetti code that’s really poorly documented.

It looks like this perfect architectural diagram with really neat functions and APIs and well-defined contracts in between them.

2. Businesses have reengineered themselves. BPR, business process reengineering was a thing that all consultancies were selling to medium and large firms from the late ’80s through to the dot-com bubble.

Business process engineering was essentially APIfying and contractualising internal operations of the company by making them separable, allowing us to find core competencies, enabling outsourcing.

3. The AI Lock In Loop effect is present. Here’s how it works:

Once you put a piece of AI into a product some kind of prediction or some kind of optimization, safety or routing algorithm for your logistics company, your service and your product gets better.

As your product gets better, you generate more data, and at the same time if you’ve got a decent business model a better product should mean more profits. More profits means reinvesting into improving AI. There’s no way back.

What we have seen is that once you implement AI successfully into an industry and into a sector it’s impossible to compete without it.

Think about web search — web search has had to have the machine learning, artificial intelligence in it, and it would be almost impossible to imagine someone launching a web search engine that didn’t use some kind of machine learning or data science. You wouldn’t launch a web search engine that just did Tf-idf as its ranking.

The other thing that we’ve seen is that AI beating humans is very, very normal. Almost every week there’s read another example of a narrow domain piece of AI which is outperforming humans.

Nowadays speech recognition is performing with 5.9% error rates, far better than humans and has gone even further than that.

Back in 2013 or 2012 none of these things would have been done better using machine learning — not a single one, and that’s only a few years ago.

There’s one issue we seem to have in our industry, that we ultimately saw previously in the oil industry.

Huge companies such as Standard Oil and Ford dominated key parts of the oil industry.

In our case, Google and Facebook have essentially got infinite computing power and know-how compared to any of us. The question is what long-term structural advantage does it give them? Are we getting the crumbs that fall off the capitalist table onto this chair for what we are and do we really have what it takes to compete in some of these domains?

These companies are really hungry to build their talent bases globally and that creates a tension if you’re an entrepreneur because you can get fuck you money from a Google or an Apple early into the lifetime of your business_._

I still think a full staff matters_._ Uber is not a machine learning company but is worth 100 times more than the biggest AI exit so far and they have made a pretty significant commitment to automation_._ They’ve acquired Otto, they took the CMU robotics team and they used ML in lots of places in their business, not just in routing but even in the consumer app that some of us might still be using.

It’s an amazing period of innovation. And responsibility.

Today the products that you can build control our interface and access to all the services that we need.

You have to adopt the mantle of civic leader in some level.

You determine how our attention gets spent, you determine how we understand our news. You kind of determine how our civil society is functioning. I think it’s really important to stand up to those responsibilities as product entrepreneurs particularly when you are essentially going to be building what stands between me and the real world.

You can read more about the most pressing issues in technology in my weekly newsletter. SIGN UP FOR EXPONENTIAL VIEW HERE.