(Not) rolling in the deep: deep learning, deep thinking, deep working and shallow machines

Adele, when Rolling in the Deep

I have recently finished the first course of the Deep Learning Specialization by Andrew Ng on Coursera and learned a bit more about neural networks, neurons, backward and forward propagation. This required me to devote some time to do it. Recalling a bit of calculus here and there. Matrices and derivatives said “hi” once again. I was glad to devote some time to play with these concepts from the past and use them as a bridge to learn something new.

Not much earlier I had also finished reading the book “Deep Work” by Cal Newport which brought to my mind the value and need for focus and deep concentration in a “distracted world”. Just like the author, I was never fond of open office spaces, where Bose noise cancelling devices became pretty much an essential supply to go through the day. One year ago I have discovered Cold Turkey — an app that allows you to block your own access to social networks and other distractions for a set period of time. The book just legitimized the thought that there is something missing, at least for people that enjoy periods of deep contemplation, something that got stolen from ourselves by ourselves, and that in many cases can be priceless.

Some of the tactics to carve out time for deep concentration helped me to finish this course quite efficiently, while I have also completed other courses that go along particularly well with the “deep” philosophy. “Learning How to Learn” and the sequel “Mindshifter” by Barbara Oakley, that focus on helping people to be more effective in learning and developing a learning lifestyle have been in the leading positions of MOOC rankings. Together with Cal’s book these are a nice pack on depth in working and learning.

At the same time I have continued my read of “The Human Condition” by Hannah Arendt — yes, I have read many other books since I have started this one, but this is taking me longer to finish, and watched the documentary “Hannah Arendt: The Life of the Spirit”.

This mix of seemingly unrelated contents gave me literally, “food for thought”.

The depth in deep

Well, Andrew Ng, in one of the first lectures mentions that neural networks have been around for a long time, but since they have been relabeled as Deep Learning they seem to resonate more with the public, to capture the popular imagination. After all, “it’s deep”.

When I think about deep thinking or learning for that matter, I think about people like Hannah Arendt that are able to develop long conversations with themselves and clinically dissect one topic. In one 1964 interview with Günter Gaus— a very interesting one- she mentions how she writes “in order to understand”, and if you read one of her books it seems she is doing just that — going through her reasoning, and zooming in the seemingly most fragile and difficult aspects, to examine, disentangle and conquer her own understanding of them. She doesn’t seem to be writing to explain something to someone else (sometimes it is easy to lose track of the reasoning), but she seems to be convincing herself of some of the arguments.

But what does this have to do with deep learning or neural nets? Neural nets power relies in their capacity to identify complex patterns that are hidden to the naked eye. Just like Hannah they go trough the data, and converse with themselves to minimize error, to make their judgment closer to the truth. But unlike Hannah, and as mentioned several times by Garry Kasparov in his book “Deep thinking” — machines lack emotions. Which makes them relentless in performing this internal conversation.

One of the most interesting insights of the MOOC “Learning How to Learn” was that the brain has two modes of cognition: the focused and the diffuse mode, and that people are thinking even when they seem to be doing nothing. It is good to make a break now and then from the deep work type of focused thinking to allow for unexpected connections to happen when you are not focusing our attention on anything in particular. I wonder how machines can do that or will be able to do that in the future.

Kasparov mentions two types of search in computer chess: the type A/brute force one or the strategic and selective, that, when a assessing and planing a move, starts by focusing on a narrower but more probable range of options than the other type, that goes through them all. And how the force brute A seems to be dominating the programmers choice in our days, when it comes to designing computer chess games: “speed beats knowledge”. The two approaches have been competing for the spotlight in terms of chess games development but for the time being (and apparently unfortunately for Garry) it seems that being faster at evaluating all the possibilities beats slowly being focused in some more wisely selected ones.

Machines that play but can’t play

In any case the machines seem all focus, not needing to find time for some recreative breaks on order to get the absolute win. Kasparov mentions that for one of his colleagues “finding the truth in the chess board” meant this absolute win or lose regardless of who played better. “At the end of the day there are only three possible situations in chess: win, lose or draw”.

Machines don’t need recreation, don’t get distracted and increasingly don’t take the long hours that humans take to be involved in deep work. So more than understanding if machines can be made squarely intelligent or not, or actually think or not, my question would be, can machines be deep?

When I was reviewing the topic of matrix calculations, I have computed manually the multiplication of matrices that would take a computer a fraction of a second to process. For a neural network of several layers it could take really long to perform the calculations manually — and in terms of the nomenclature, the higher the number of layers the deeper the network is. So according to this, yes, machines can be deep, and can be faster to get deep than humans.

But how come that when I think of machines they seem doomed to be shallow? I can imagine a robot that plays at the level of a Russian chess player but a robot that writes with the depth of a Russian novelist it is harder to conceive, or a robot philosopher. Not only because they don’t ask questions as Kasparov also mentioned, but because they are not able to suffer and I always associate depth with a certain capacity for suffering or capacity feeling for that matter.

The transparency in deep

Another instance in which machines are not so good at yet is transparency. While Hannah shows her reasoning to reach a conclusion, machines are much less transparent. In neural networks there is even what are called the hidden layers (all layers, besides the input and output layers), were the bulk of the computations take place but that resemble a black box with difficult, if not forbidden, access. We have inputs and outputs, as for what happens in between, machines cannot clearly articulate to humans.

Perhaps I associate depth with depth at sea, where no matter how deep you dive you are always able to see the elements around you. It happens like that for human thinkers, that are able to convey their thought via language that makes it more or less transparent (depending on their capacity to communicate ideas to others and also themselves). As for machines it seems that depth is more like that of a pond or lake, where the you can see the bumps of Loch Ness but not its full frame.

Mystery ‘bumps’ photographed by Clive Limpkin at Drumnadrochit. REXSCANPIX. Metro article 2015

The machine condition

As a consequence, I don’t think I can admire machines, or that one day I can have a machine as hero. Not because they are not intelligent — honestly that can at times feel a bit overrated - but because they do not feel. I confess I was somewhat surprised by the seeming trauma that loosing to a machine caused to Kasparov — really? You lost to a machine? Keep calm and carry on.

But I would like to see a machine have some kind of discussion and conversation with Hannah, go with the motions of her reasoning and give something back in return or write the letters that Hannah wrote to Karl Jaspers — pretty much a father for her- or Heidegger — the love of her life, and what would those letters mean.

In the movie “Her”, where a lonely man falls in love with his virtual assistant Samantha (interpreted by Scarlett Johansson), he is not falling in love with the machine intelligence — I don’t recall any chess-like conversation but rather banal albeit intimate conversations at the end of the day, that could be uttered by anyone close enough, not a grandmaster.

Theodore, when Rolling in The Deep in the Movie “Her”

So, I repeat the question. Can machines be deep? Can machines have real deep conversations with other humans and perhaps other machines? And I don’t mean scripted, algorithmic, chess-like conversations, and (somewhat unfortunately) not even displaying the depth of reasoning of thinking and feeling of some writers and philosophers, but conversations that are able to strike a chord in the human heart.

The moment the answer is yes — and Amazon’s Alexa seems to be pushing that moment to far (but we can, as always) be surprised - it is the moment that “depth” really matches the “depth” that popular imagination is used to, and is not somewhat misleading publicity.

Furthermore, perhaps when we start framing the questions about machines in terms of depth and not (scripted) intelligence, we will stop thinking about machines as threats, as chess players that either allow us to win, lose or draw, but as co-authors of a richer and deeper life.

After all, as Cal Newport mentions, making some edits to the original from Socrates that “the unexamined life is not worth living”, I also believe that a “life without depth is not worth living”. And machines are still not equipped to assist us with that.

So call it everything, but don’t call it deep, just yet. Neural networks are getting quite good are identifying cats though… although some still are harder to recognize, like my friend’s cats below, that I used i one of the course exercises. I assure you they are as real as it gets 😸

“y=0.0, your algorithm predicts a non-cat picture”

Sometimes it is still a bit like this:

One of the Adele/Cat online memes. Didn’t know this was a thing until I searched for something to illustrate. I am not the author.