Algorithmic Complexes, Alpha Male Brains, and Winnie the Pooh (by Silly Rabbit)

Massively complex complexes of algorithms

Let me come straight out with it and state, for the record, that I believe the best current truth we have is that we humans, along with all other living beings, are simply massively complex complexes of algorithms. What do I mean by that? Well, let’s take a passage from the terrific Homo Deus by Yuval Noah Harari, which describes this concept at length and in detail:

In recent decades life scientists have demonstrated that emotions are not some mysterious spiritual phenomenon that is useful just for writing poetry and composing symphonies. Rather, emotions are biochemical algorithms that are vital for the survival and reproduction of all mammals. What does this mean? Well, let’s begin by explaining what an algorithm is, because the 21st Century will be dominated by algorithms. ‘Algorithm’ is arguably the single most important concept in our world. If we want to understand our life and our future, we should make every effort to understand what an algorithm is and how algorithms are connected with emotions. An algorithm is a methodical set of steps that can be used to make calculations, resolve problems and reach decisions. An algorithm isn’t a particular calculation but the method followed when making the calculation.

Consider, for example, the following survival problem: a baboon needs to take into account a lot of data. How far am I from the bananas? How far away is the lion? How fast can I run? How fast can the lion run? Is the lion awake or asleep? Does the lion seem to be hungry or satiated? How many bananas are there? Are they big or small? Green or ripe? In addition to these external data, the baboon must also consider information about conditions within his own body. If he is starving, it makes sense to risk everything for those bananas, no matter the odds. In contrast, if he has just eaten, and the bananas are mere greed, why take any risks at all? In order to weigh and balance all these variables and probabilities, the baboon requires far more complicated algorithms than the ones controlling automatic vending machines. The prize for making correct calculations is correspondingly greater. The prize is the very survival of the baboon. A timid baboon — one whose algorithms overestimate dangers — will starve to death, and the genes that shaped these cowardly algorithms will perish with him. A rash baboon —one whose algorithms underestimate dangers — will fall prey to the lion, and his reckless genes will also fail to make it to the next generation. These algorithms undergo constant quality control by natural selection. Only animals that calculate probabilities correctly leave offspring behind. Yet this is all very abstract. How exactly does a baboon calculate probabilities? He certainly doesn’t draw a pencil from behind his ear, a notebook from a back pocket, and start computing running speeds and energy levels with a calculator. Rather, the baboon’s entire body is the calculator. What we call sensations and emotions are in fact algorithms. The baboon feels hunger, he feels fear and trembling at the sight of the lion, and he feels his mouth watering at the sight of the bananas. Within a split second, he experiences a storm of sensations, emotions and desires, which is nothing but the process of calculation. The result will appear as a feeling: the baboon will suddenly feel his spirit rising, his hairs standing on end, his muscles tensing, his chest expanding, and he will inhale a big breath, and ‘Forward! I can do it! To the bananas!’ Alternatively, he may be overcome by fear, his shoulders will droop, his stomach will turn, his legs will give way, and ‘Mama! A lion! Help!’ Sometimes the probabilities match so evenly that it is hard to decide. This too will manifest itself as a feeling. The baboon will feel confused and indecisive. ‘Yes . . . No . . . Yes . . . No . . . Damn! I don’t know what to do!’

Why does this matter? I think understanding and accepting this point is absolutely critical to being able to construct certain classes of novel and interesting algorithms. “But what about consciousness?” you may ask, “Does this not distinguish humans and raise us above all other animals, or at least machines?”

There is likely no better explanation, or succinct quote, to deal with the question of consciousness than Douglas Hofstadter’s in I Am a Strange Loop:

“In the end, we are self-perceiving, self-inventing, locked-in mirages that are little miracles of self-reference.”

Let’s accept Hofstadter’s explanation (which is — to paraphrase and oversimplify terribly — that, at a certain point of algorithmic complexity, consciousness emerges due to self-referencing feedback loops) and now hand the mic back to Harari to finish his practical thought:

“This raises a novel question: which of the two is really important, intelligence or consciousness? As long as they went hand in hand, debating their relative value was just an amusing pastime for philosophers, but now humans are in danger of losing their economic value because intelligence is decoupling from consciousness.”

Or, to put it another way: if what I need is an intelligent algorithm to read, parse and tag language in certain reports based on whether humans with a certain background would perceive the report as more ‘growth-y’ vs ‘value-y’ in its tone and tenor, why do I need to discriminate whether the algorithm performing this action has consciousness or not, or which parts of the algorithms have consciousness (assuming that the action can be equally parallelized either way)?

AI vs. human performance

Electronic Frontier Foundation have done magnificent work pulling together problems and metrics/datasets from the AI research literature in order to see how things are progressing in specific subfields or AI/machine learning as a whole. Very interesting charts on AI versus human performance in image recognition, chess, book comprehension, and speech recognition (keep scrolling down; it’s a very long page with lots of charts).

Alpha male brain switch

Researchers led by Prof Hailan Hu, a neuroscientist at Zhejiang University in Hangzhou, China have demonstrated activating the dorsal medial prefrontal cortex (dmPFC) brain circuit in mice to flip the neural switch for becoming an alpha male. This turned the timid mice bold after their ‘alpha’ circuit was stimulated.  Results also show that the ‘winner effect’ lingers on and that the mechanism may be similar in humans. Profound and fascinating work.

Explaining vs. understanding

And finally, generally I find @nntaleb’s tweets pretty obnoxious and low value (unlike his books, which I find pretty obnoxious and tremendously high value), but this tweet really captured me: “Society is increasingly run by those who are better at explaining than understanding.” I pondered last week on how allocators and Funds of Funds are going to allocate to ‘AI’ (or ‘ALIS’). This quote succinctly sums up and generalizes that concern.

And finally, finally, this has nothing to do with Big Compute, AI, or investment strategies, but it is just irresistible: Winnie the Pooh blacklisted by China’s online censors: “Social media ban for fictional bear follows comparisons with Xi Jinping.” Original FT article here (possibly pay-walled) and lower resolution derivative article (not pay-walled) by inUth here. As Pooh says “Sometimes I sits and thinks, and sometimes I just sits…”

PDF Download (Paid Subscription Required): http://www.epsilontheory.com/download/15951/

Long Short-Term Memory, Algorithms for Social Justice, and External Cognition (by Silly Rabbit)

DARPA funds graph analytics processor

Last week I posted a bunch of links pointing towards quantum computing. However, there are also other compute initiatives which also offer significant potential for “redefining intractable” for problems such as graph comparison, for example, DARPA’s HIVE which aims to create a 1000x improvement in processing speed (and at much lower power) on this problem. Write-up on EE Times of the DARPA HIVE program here.

Exploring long short-term memory networks

Nice explainer on LSTMs by Edwin Chen: “The first time I learned about LSTMs, my eyes glazed over. Not in a good, jelly donut kind of way. It turns out LSTMs are a fairly simple extension to neural networks, and they’re behind a lot of the amazing achievements deep learning has made in the past few years.” (Long, detailed and interesting blog post, but even if you just read the first few page scrolls still quite worthwhile for the intuition of the value and function of LSTMs.)

FairML: Auditing black box predictive models

Machine learning models are used for important decisions like determining who has access to bail. The aim is to increase efficiency and spot patterns in data that humans would otherwise miss. But how do we know if a machine learning model is fair? And what does fairness in machine learning mean? Paper exploring these questions using FairML, a new Python library that audits black-box predictive models.

Fast iteration wins prizes

Great Quora answer on “Why has Keras been so successful lately at Kaggle competitions?” (By the author of Keras, an open source neural net library designed to enable fast experimentation). Key quote: ”You don’t lose to people who are smarter than you, you lose to people who have iterated through more experiments than you did, refining their models a little bit each time. If you ranked teams on Kaggle by how many experiments they ran, I’m sure you would see a very strong correlation with the final competition leaderboard.” 

Language from police body camera footage shows racial disparities in officer respect

This paper presents a systematic analysis of officer body-worn camera footage, using computational linguistic techniques to automatically measure the respect level that officers display to community members.

External cognition

Large-scale brainlike systems are possible with existing technology — if we’re willing to spend the money — proposes Jennifer Hassler in A Road Map for the Artificial Brain.

Pretty well re-tweeted and shared already, but interesting nonetheless: External cognition: The Thoughts of a Spiderweb.

And related somewhat related (or at least a really nice AR UX for controlling synthesizers), a demonstration of “prosthetic knowledge” — check out the two minute video with sound at the bottom of the page – awesome stuff!

PDF Download (Paid Subscription Required): http://www.epsilontheory.com/download/16010/

Quantum Supremacy, Correlating Unemployment, and Buddhists with Attitude (by Silly Rabbit)

Quantum supremacy

As Ben and I have discussed before on an Epsilon Theory podcast, my view is that quantum computing is going to be truly, truly transformational by “redefining intractable”, as 1Qbit say, over the coming years. My conviction around quantum continues to grow and — to put a pretty big stake in the ground — I believe, at this point, the only open questions are: Which approach will dominate, and how long exactly until we get quantum machines which work on a broad set of real-world questions? I’ve long been a big fan of the applied, real-world progress D-wave have made, and Rigetti too. However, the “majors” like IBM are also making substantial progress towards true “quantum supremacy” with R&D intensive approaches, while other pieces of the ecosystem, such as the ability to “certify quantum states“, continue to fall into place. In the meantime, here is a wonderful cartoon explainer on quantum computing by Scott Aaronson and Zach Weinersmith.

What web searches correlate to unemployment

Well, in order to get the answer to that question you will have to follow this link (and be prepared to blush). The findings were generated by Seth Stephens-Davidowitz using Google Correlate. “Frequently, the value of Big Data is not its size; it’s that it can offer you new kinds of information to study — information that had never previously been collected”, says Stephens-Davidowitz.

Using verbal and nonverbal behaviors to measure completeness, confidence and accuracy

I recently came across Mitra Capital in Boston who have an interesting strategy of “using verbal indicators to judge the completeness and reliability of messages, to form predictions about company performance (via) analysis of management commentary from quarterly earnings calls and investor conferences based on a proprietary and proven framework with roots in the Central Intelligence Agency” with the underlying tech/methodology based on BIA. They’re running a relatively small fund ($53m AUM in Q1 2017) and have returned an average of 8.5% for the past four years (including a +43% year, and a -12.5% year). Neat NLP approach, although these returns imply more of a “feature than a product” (i.e., a valuable sub-system addition to a larger system, rather than a stand-alone system.) But, hey, I said the same thing about Instagram.

Buddhists with attitude / Backtesting: Methodology with a fragility problem

Probably (hopefully!) anyone reading Epsilon Theory has already read Antifragile by Nassim Nicholas Taleb. Many things which could and have been said about this book, but the most important one to highlight for my narrow, domain application is the massively important distinction (although rarely talked about facet) of machine learning/big compute approaches vs. regression-driven back test approaches. Key distinction is a simple one: Does your system gain from exposure to randomness and stress (within bounds) and improve the longer it exists and the more events it is exposed to OR does it perform less well with stress, and decay with time. Antifragile machine learning systems are profoundly different to the fragile fitting of models.

And finally, since I have already invoked Taleb, and if for no other reason that the line “If someone wonders who are the Stoics I’d say Buddhists with an attitude problem”, here is Taleb’s Commencement address to American University of Beirut last year.

PDF Download (Paid Subscription Required): http://www.epsilontheory.com/download/16037/

Complex Systems, Multiscale Information and Strange Loops (by Silly Rabbit)

Complex systems

Neat and accessible primer on complex systems, multiscale information theory and universality by Yaneer Bar-Yan, and a related paper on the conceptual applications of the same topic: From Big Data To Important Information (suggest start reading from section VII if you read the primer, and from sub-section D on page 13 if you just want the markets application).

Machine learning software creates machine learning software

Lots of buzz about Google’s AutoML announcement at the Google Annual developer conference I/O 2017 last week. AutoML is machine learning software which takes over some of the work of creating machine learning software and, in some cases, came up with designs that rivals or beats the best work of human machine learning experts. MIT Technology Review article on AutoML.

One-shot imitation

Also lots of buzz around one-shot imitation using two neural nets, as demonstrated by OpenAI. Personally, one-shot imitation is the one AI-type concept which gives me the fear. But if Elon’s supporting it then it must be OK… right? One-shot imitation paper here but, more to the point, watch this video and tell me you are not at least a little bit afraid.

The power of the platform

And to the practical applications of technology, I really like the language of this recent press release by Two Sigma CEO, Nobel Gulati, and particularly the paragraph:

Moving forward, durable advantages will to accrue to those building a substantial platform based on massive amounts of data, along with the technology and institutional expertise to use it. Building such a platform requires significant and ongoing investment in R&D, and a fundamentally different culture and mindset to apply a scientific approach to the data-rich world of today.

Personally, I believe that the 2020s will be more defined by big compute than big data but this is, nonetheless, a powerful statement and language, and there’s a key implicit point buried in here on the cultural balance of ‘researchers’ (math and physics natural genii) and ‘production engineers’ (coders who, by nurture, have seen and solved many practical problems). Specifically, how the majority of quant funds have to-date been culturally focused too heavily on the math genius research folks to the detriment of hiring and rewarding the more workmanlike practical folks who can build and maintain a substantial platform which, I agree, is the new durable advantage.

去吧

I was reminded last week by China’s censorship of Google’s latest AlphaGo win against Ke Jie just how substantial a stance it was when Google shut down its Mainland search engine in 2010 and why these kind of bold moves (bets) are essential to developing a truly winning technology company (and also why I don’t live in China anymore!). As Rusty Guinn has written about: A man must have code.

Strange loops

Finally, to bring us back up to the level of self and consciousness, I finally got ‘round to reading Douglas R. Hofstadter’s 2007 book I am a Strange Loop. A long, winding and compelling book summarized by the quote “In the end, we are self-perceiving, self-inventing, locked-in mirages that are little miracles of self-reference.” If you dip in and only read one section, read the section on simmballs in Chapter 3, which loops us back to where we started this column on multiscale information.

PDF Download (Paid Subscription Required): http://www.epsilontheory.com/download/16045/

She Screams, He Kidnaps (by Silly Rabbit)

Proximity of verbs to gender

Sometimes biases embedded within language are subtle, counter-intuitive things which you have to tease out with many layered neural nets. Other times, they are just bluntly and painfully predictable: Data scientist David Robinson tracked the proximity of verbs to gender across 100,000 stories. She screams, cries and rejects. He kidnaps, rescues and beats.

Wiki-memory

Previously I shared some research on how recollections of successive events physically entangle each other when brain cells store them. As a fascinating and different approach to studying memory, in this paper a group of European researchers used Wikipedia page views of aircraft crashes to study memory.

Fool me once, fool me twice

Sooner or later, someone is probably going to put a visually compelling 2D ‘map’ of data reduced from hundreds or thousands of dimensions via t-SNE in front of you and make some bold assertions about it. This beautiful and interactive paper provides a handy guide on what to watch out for.

A veritable zoo of machine learning techniques

A couple of months old, but still useful: Two Sigma researchers Vinod Valsalam and Firdaus Janoos write up the notable advances in machine learning presented at NIPS (Neural Information Processing Systems Foundation) 2016. Headline: The dominating theme at NIPS 2016 was deep learning, sometimes combined with other machine learning methods such as reinforcement learning and Bayesian techniques.

The NIPS conference has, improbably, found itself at the center of the universe as it is the most important event for people sharing cutting edge machine learning work. It’s in LA this year in December and promises to be very interesting, although quite technical: https://nips.cc/

Silicon Valley: a reality check

And, finally, this one is a little inside baseball but, if you can push through that, there is a very useful and accurate parsing of the types of technology companies being started and funded in the Valley and the simultaneous parallel dimensions that exist here. (You can skip the Valley defense bit and jump to the smart parsing bit by hitting ‘CRTL + F’ , typing ‘Y Combinator’ and start reading from there).

PDF Download (Paid Subscription Required): http://www.epsilontheory.com/download/16051/