Decoding the code of life

We have known, since 1953, that the DNA molecule encodes the genetic information that transmits characteristics from ancestors to descendants, in all types of lifeforms on Earth. Genes, in the DNA sequences, specify the primary structure of proteins, the sequence of amino acids that are the components of the proteins, the cellular machines that do the jobs required to keep a cell alive. The secondary structure of proteins specifies some of the ways a protein folds locally, in structures like alpha helices and beta sheets. Methods that can determine reliably the secondary structure of proteins have existed for some time. However, determining the way a protein folds globally in space (its tertiary structure, the shape it assumes) has remained, mostly, an open problem, outside the reach of most algorithms, in the general case.

The Critical Assessment of protein Structure Prediction (CASP) competition, started in 1994, took place every two years since then and made it possible for hundreds of competing teams to test their algorithms and approaches in this difficult problem. Thousands of approaches have been tried, to some success, but the precision of the predictions was still rather low, especially for proteins that were not similar to other known proteins.

A number of different challenges have taken place over the years in CASP, ranging from ab-initio prediction to the prediction of structure using homology information and the field has seen steady improvements, over time. However, the entrance of DeepMind into the competition upped the stakes and revolutionized the field. As DeepMind itself reports in a blog post, the program AlphaFold 2, a successor of AlphaFold, entered the 2020 edition of CASP and managed to obtain a score of 92.4%, measured in the Global Distance Test (GDT) scale, which ranges from 0 to 100. This value should be compared with the value 58.9% obtained by AlphaFold (the previous version of this year’s winner) in 2018, and the 40% score obtained by the winner of the 2016 competition.

Structure of insulin

Even though details of the algorithm have still not been published, the information provided in the DeepMind post provides enough information to realize that this result is a very significant one. Although the whole approach is complex and the system integrates information from a number of sources, it relies on an attention-based neural network, which is trained end-to-end to learn which amino acids are close to each other, and at which distance.

Given the importance of the problem on areas like biology, medical science and pharmaceutics, it is to be expected that this computational approach to the problem of protein structure determination will have a significant impact in the future. Once more, rather general machine learning techniques, which have been developed over the last decades, have shown great potential in real world problems.

Novacene: the future of humanity is digital?

As it says on the cover of the book, James Lovelock may well be “the great scientific visionary of our age“. He is probably best known for the Gaia Hypothesis, but he made several other major contributions. While working for NASA, he was the first to propose looking for chemical biomarkers in the atmosphere of other planets as a sign of extraterrestrial life, a method that has been extensively used and led to a number of interesting results, some of them very recent. He has argued for climate engineering methods, to fight global warming, and a strong supporter of nuclear energy, by far the safest and less polluting form of energy currently available.

Lovelock has been an outspoken environmentalist, a strong voice against global warming, and the creator of the Gaia Hypothesis, the idea that all organisms on Earth are part of a synergistic and self-regulating system that seeks to maintain the conditions for life on Earth. The ideas he puts forward in this book are, therefore, surprising. To him, we are leaving the Anthropocene (a geological epoch, characterized by the profound effect of men on the Earth environment, still not recognized as a separate epoch by mainstream science) and entering the Novacene, an epoch where digital intelligence will become the most important form of life on Earth and near space.

Although it may seem like a position inconsistent with his previous arguments about the nature of life on Earth, I find the argument for the Novacene era convincing and coherent. Again, Lovelock appears as a visionary, extrapolating to its ultimate conclusion the trend of technological development that started with the industrial revolution.

As he says, “The intelligence that launches the age that follows the Anthropocene will not be human; it will be something wholly different from anything we can now conceive.”

To me, his argument that artificial intelligence, digital intelligence, will be our future, our offspring, is convincing. It will be as different from us as we are from the first animals that appeared hundreds of millions ago, which were also very different from the cells that started life on Earth. Four billion years after the first lifeforms appeared on Earth, life will finally create a new physical support, that does not depend on DNA, water, or an Earth-like environment and is adequate for space.

Could Venus possibly harbor life?

Two recently published papers, including one in Nature Astronomy (about the discovery itself) and this one in Astrobiology (describing a possible life cycle), report the existence of phosphine in the upper atmosphere of Venus, a gas that cannot be easily generated by non-biological processes in the conditions believed to exist in that planet. Phosphine may, indeed, turn out to be a biosignature, an indicator of the possible existence of micro-organisms in a planet that was considered, up to now, barren. Search for life in our solar system has been concentrated in other bodies, more likely to host micro-organisms, like Mars of the icy moons of outer planets.

The findings have been reported in many media outlets, including the NY Times and The Economist, raising interesting questions about the prevalence of life in the universe and the possible existence of life in one of our nearest neighbor planets. If the biological origin of phosphine were to be confirmed, it would qualify as the discovery of the century, maybe the most important discovery in the history of science! We are, however, far from that point. A number of things may make this finding another false alarm. Still, it is quite exciting that what has been considered a possible sign of life has been found so close to us and even a negative result would increase our knowledge about the chemical processes that generate this compound until now believed to be a reliable biomarker.

This turns out to be a first step, not a final result. Quoting from the Nature Astronomy paper:

Even if confirmed, we emphasize that the detection of PH3 is not robust evidence for life, only for anomalous and unexplained chemistry. There are substantial conceptual problems for the idea of life in Venus’s clouds—the environment is extremely dehydrating as well as hyperacidic. However, we have ruled out many chemical routes to PH3, with the most likely ones falling short by four to eight orders of magnitude (Extended Data Fig. 10). To further discriminate between unknown photochemical and/or geological processes as the source of Venusian PH3, or to determine whether there is life in the clouds of Venus, substantial modelling and experimentation will be important. Ultimately, a solution could come from revisiting Venus for in situ measurements or aerosol return.

The Book of Why

Correlation is not causation is a mantra that you may have heard many times, calling attention to the fact that no matter how strong the relations one may find between variables, they are not conclusive evidence for the existence of a cause and effect relationship. In fact, most modern AI and Machine Learning techniques look for relations between variables to infer useful classifiers, regressors, and decision mechanisms. Statistical studies, with either big or small data, have also generally abstained from explicitly inferring causality between phenomena, except when randomized control trials are used, virtually the unique case where causality can be inferred with little or no risk of confounding.

In The Book of Why, Judea Pearl, in collaboration with Dana Mackenzie, ups the ante and argues not only that one should not stay away from reasoning about causes and effects, but also that the decades-old practice of avoiding causal reasoning has been one of the reasons for our limited success in many fields, including Artificial Intelligence.

Pearl’s main point is that causal reasoning is not only essential for higher-level intelligence but is also the natural way we, humans, think about the world. Pearl, a world-renowned researcher for his work in probabilistic reasoning, has made many contributions to AI and statistics, including the well known Bayesian networks, an approach that exposes regularities in joint probability distributions. Still, he thinks that all those contributions pale in comparison with the revolution he speared on the effective use of causal reasoning in statistics.

Pearl argues that statistical-based AI systems are restricted to finding associations between variables, stuck in what he calls rung 1 of the Ladder of Causation: Association. Seeing associations leads to a very superficial understanding of the world since it restricts the actor to the observation of variables and the analysis of relations between them. In rung 2 of the Ladder, Intervention, actors can intervene and change the world, which leads to an understanding of cause and effect. In rung 3, Counterfactuals, actors can imagine different worlds, namely what would have happened if the actor did this instead of that.

This may seem a bit abstract, but that is where the book becomes a very pleasant surprise. Although it is a book written for the general public, the authors go deeply into the questions, getting to the point where they explain the do-calculus, a methodology Pearl and his students developed to calculate, under a set of dependence/independence assumptions, what would happen if a specific variable is changed in a possibly complex network of interconnected variables. In fact, graphic representations of these networks, causal diagrams, are at the root of the methods presented and are extensively used in the book to illustrate many challenges, problems, and paradoxes.

In fact, the chapter on paradoxes is particularly entertaining, covering the Monty Hall, Berkson, and Simpson’s paradoxes, all of them quite puzzling. My favorite instance of Simpson’s paradox is the Berkeley admissions puzzle, the subject of a famous 1975 Science article. The paradox comes from the fact that, at the time, Berkeley admitted 44% of male candidates to graduate studies, but only 35% of female applicants. However, each particular department (departments decide the admissions in Berkeley, as in many other places) made decisions that were more favorable to women than men. As it turns out, this strange state of affairs has a perfectly reasonable explanation, but you will have to read the book to find out.

The book contains many fascinating stories and includes a surprising amount of personal accounts, making for a very entertaining and instructive reading.

Note: the ladder of causation figure is from the book itself.

A conversation with GPT-3 on COVID-19

GPT-3 is the most advanced language model ever created, a product of an effort by OpenAI to create a publicly available system that can be used to advance research and applications in natural language. The model itself published less than three months ago, is an autoregressive language model with 175 billion parameters and was trained with a dataset that includes almost a trillion words.

Impressive as that may be, it is difficult to get some intuition of what such a complex model, trained on billions of human-generated texts, can actually do. Can it be used effectively in translation tasks or in answering questions?

To get some idea of what a sufficiently high-level statistical model of human language can do, I challenge you to have a look at this conversation with GPT-3, published by Kirk Ouimet a few days ago. It relates a dialogue between him and GPT-3 on the topic of COVID-19. The most impressive thing about this conversation with an AI is not that it gets many of the responses right (others not so much). What impressed me is that the model was trained with a dataset created before the existence of COVID-19, which provided GPT-3 no specific knowledge about this pandemic. Whatever answers GPT-3 gives to the questions related to COVID-19 are obtained with knowledge that was already available before the pandemic began.

This certainly raises some questions on whether advanced AI systems should be more widely used to define and implement policies important to the human race.

If you want more information bout GPT-3, it is easy to find in a multitude of sites with tutorials and demonstrations, such as TheNextWeb, MIT Technology Review, and many, many others.

Instantiation, another great collection of Greg Egan’s short stories

Greg Egan is a master of short-story telling. His Axiomatic collection of short stories is one of my favorites. This new collection of short stories keeps Egan’s knack for communicating deep concepts using few words and dives deeper into the concepts of virtual reality and the impacts of technology in society.

The first story, The discrete charm of the Turing machine, could hardly be more relevant these days, when the discussions on the economic impacts of Artificial Intelligence are taking place everywhere. But the main conducting line of the book is the series of stories where sentient humans who are, in fact, characters in virtual reality games, plot to break free of their slave condition. To find out whether they succeed or not, you will have to read to book yourself!

PS: As a joke, I leave here a meme of unknown origin

SIMULACRON-3: are we living in a computer simulation?

Are we living in a computer simulation? And, if so, how could we tell? This question became very popular in the last few years and has led to many articles, comments, and arguments. The simulation hypothesis which states that all of reality, including the Earth and the observable universe, could, in fact, be the result of a computer simulation is a hot topic of debate among philosophers, scientists, and SF writers.  Even the popular Saturday Morning Breakfast Cereal (SMBC) webcomic has helped clarify the issue, in a very popular strip. Greg Egan, the master of realistic SF, may have taken the matter to its ultimate consequences, with Permutation City and Instantiation, but the truth is that this question has been the subject of many books, including the famous Neuromancer, by William Gibson.

Still, to my knowledge, Simulacron-3, by Daniel Galouye, may have been the first SF book to tackle the issue head-on. For a book written more than half a century ago, the story is surprisingly modern and up-to-date. Not only the presentation of the simulated reality world is very convincing and the technology very believable, but it also turns out that the reasons why the simulated reality world (Simulacron-3) was created could be sold as a business plan for any ambitious startup today.

There is not much more that I can write about this book without depriving you of the pleasure of reading it, so let me just recommend that you get a copy from a website near you and take with you for the summer holidays.

Human Compatible: AI and the Problem of Control

Stuart Russell, one of the better-known researchers in Artificial Intelligence, author of the best selling textbook Artificial Intelligence, A Modern Approach addresses, in his most recent book, what is probably one of the most interesting open questions in science and technology: can we control the artificially intelligent systems that will be created in the decades to come?

In Human Compatible: AI and the Problem of Control Russell formulates and answers the following, very important question: what are the consequences if we succeed in creating a truly intelligent machine?

The question brings, with it, many other questions, of course. Will intelligent machines be dangerous to humanity? Will they take over the world? Could we control machines that are more intelligent than ourselves? Many writers and scientists, like Nick Bostrom, Stephen Hawking, Elon Musk, Sam Harris, and Max Tegmark have raised these questions, several of them claiming that superintelligent machines could be around the corner and become extremely dangerous to the humanity.

However, most AI researchers have dismissed these questions as irrelevant, concentrated as they are in the development of specific techniques and well aware that Artificial General Intelligence is far away, if it is at all achievable.  Andrew Ng, another famous AI researcher, said that worrying about superintelligent machines is like worrying about the overpopulation. of Mars.

There could be a race of killer robots in the far future, but I don’t work on not turning AI evil today for the same reason I don’t worry about the problem of overpopulation on the planet Mars

Another famous Machine Learning researcher, Pedro Domingos, in his bestselling book, The Master Algorithm, about Machine Learning, the driving force behind modern AI, also ignores these issues, concentrating on concrete technologies and applications. In fact, he says often that he is more worried about dumb machines than about superintelligent machines.

Stuart Russell’s book is different, making the point that we may, indeed, lose control of such systems, even though he does not believe they could harm us by malice or with intention. In fact, Russell is quite dismissive of the possibility that machines could one day become truly intelligent and conscious, a position I find, personally, very brave, 70 years after Alan Turing saying exactly the opposite.

Yet, Russell believes we may be in trouble if sufficiently intelligent and powerful machines have objectives that are not well aligned with the real objectives of their designers. His point is that a poorly conceived AI system, which aims at optimizing some function that was badly specified can lead to bad results and even tragedy if such a system controls critical facilities. One well-known example is Bostrom’s paperclip problem, where an AI system designed to maximize the production of paperclips turns the whole planet into a paperclip production factory, eliminating humanity in the process. As in the cases that Russell fears, the problem comes not from a machine which wants to kill all humans, but from a machine that was designed with the wrong objectives in mind and does not stop before achieving them.

To avoid that risk os misalignment between human and machine objectives, Russell proposes designing provably beneficial AI systems, based on three principles that can be summarized as:

• Aim to maximize the realization of human preferences
• Assume uncertainty about these preferences
• Learn these preferences from human behavior

Although I am not fully aligned with Russell in all the positions he defends in this book, it makes for interesting reading, coming from someone who is a knowledgeable AI researcher and cares about the problems of alignment and control of AI systems.

The Origin of Consciousness in the Breakdown of the Bicameral Mind

The origin of consciousness in the breakdown of the bicameral mind, a 1976 book by Julian Jaynes, is probably one of the most intriguing and contentious works in the already unusually controversial field of consciousness studies. This book proposed bicameralism, the hypothesis that the human mind once operated in a state in which cognitive functions were divided between one half of the brain, which appears to be speaking, and another half which listens and follows instructions. Julian Jaynes’ central claim is that consciousness in humans, in the form that is familiar to us today, is a relatively recent phenomenon, whose development followed the invention of writing, the evolution of complex societies and the collapse of bicameralism. According to Jaynes, in the bicameral eras, humans attributed the origin of the inner voices (which we presumably all hear) not to themselves, but to gods. Human behavior was, therefore, not conscious but automatic. Actions followed from strict obedience to these inner voices, which represented orders from a personal god, themselves conditioned by social and cultural norms.

In Jaynes view, consciousness is strongly connected with human language (an assertion hard to refute but possibly an insufficiently general description) and results, in large part, from our ability to introspect, and to hold conversations and dialogues with ourselves. The change in human’s perception of these voices, a process which, according to Jaynes, took place over a time span that lasted only a couple of millennia, during the Babylonian, Assyrian, Greek and Egyptian civilizations (the ones he studied) led to the creation of consciousness as we know it today. This implies that human consciousness, as it exists today, is a brand new phenomenon, in the evolutionary timescale.

Taken at face value, this theory goes totally against the very ingrained belief that humans have been fully conscious for hundreds of thousands or even millions of years, if we consider other species of hominids and other primates. It is certainly strange to think that consciousness, as we know it, is a phenomenon with only a few millennia.

And yet, Jaynes’ arguments are everything but naive. They are, in fact, very sophisticated and based on extensive analyses of historical evidence. The problem with the theory is not that it is simplistic or that there is a lack of presented evidence. The problem I have with this theory is that the evidence presented comes mostly from a very subjective and argumentative analysis of historical artifacts (books, texts, vases, ruins), which are interpreted, in a very intelligent way, to support Jaynes’ main points.

To give an example, which plays an important role in the argument, let’s consider the Iliad. In this text, which predates, according to Jaynes, conscious behavior, and has its origins in bicameral times, all human actions derive, directly, from the clear and audible instructions received from gods. In the Iliad, there is no space for reflection, autonomy, cogitations, hesitations or doubts. Heroes and plain humans act on the voices of gods, and that’s it. The Odyssey and posterior texts are progressively more elaborate on human thought and motivation and (according to Jaynes) the works of Solon are the first that can be viewed as modern, consistent with our current views of human will and human consciousness. Most significant of all, to Jaynes, is the Bible, in particular the Old Testament, which he sees as the ultimate record of the progressive evolution of men from bicameralism to subjective, conscious, behavior.  Analysis of these texts and of other evidence of the evolution of consciousness in Mesopotamia, Assyria, Greece, and Egypt, are exhaustively presented, and should not be taken lightly. At the least, Jaynes may have a point in that consciousness, today, is not the same thing as consciousness, five millennia ago. This may well be true, and it is hard for us to understand human thought from that time.

An yet, I remained unconvinced of Jaynes’ main point. True, the interpretation he makes of the historical evidence is from someone who has studied the materials deeply and I am certainly unable to counter-argue with someone who is so familiar with the topics. But, to me, the many facts (thousands, probably) that he brings to bear on his argument can all be the result of many other factors. Maybe the writers of the Iliad wanted to use god’s voices for stylistic effect, maybe the empty throne of the Assyrian king Tukulti-Ninurta depicted in a famous scene is not due to the disappearance and silence of the gods (as he argues) but to some other reasons. Jaynes proposes many interesting and ingenious interpretations of historical data, but in the end I was not convinced that these interpretations are sufficient to support his main thesis.

Despite missing his main objective, however, the book makes for a great read, presenting an interpretation of ancient history that is gripping and enlightening, if not fully convincing.

The mind of a fly

Researchers from the Howard Hughes Medical Institute, Google and other institutions have published the neuron level connectome of a significant part of the brain of the fruit fly, what they called the hemibrain. This may become one of the most significant advances in our understanding of the detailed structure of complex brains, since the 302 neurons connectome of C. elegans was published in 1986, by a team headed by Sydney Brenner, in an famous article with the somewhat whimsical subtitle of The mind of a worm. Both methods used an approach based on the slicing of the brains in very thin slices, followed by the use of scanning electron microscopy and the processing of the resulting images in order to obtain the 3D structure of the brain.

The neuron-level connectome of C. elegans was obtained after a painstaking effort that lasted decades, of manual annotation of the images obtained from the thousands of slices imaged using electron microscopy. As the brain of Drosophila melanogaster, the fruit fly, is thousands of times more complex, such an effort would have required several centuries if done by hand. Therefore, Google’s machine learning algorithms have been trained to identify sections of neurons, including axons, bodies and dendritic trees, as well as synapses and other components. After extensive training, the millions of images that resulted from the serial electron microscopy procedure were automatically annotated by the machine learning algorithms, enabling the team to complete in just a few years the detailed neuron-level connectome of a significant section of the fly brain, which includes roughly 25000 neurons and 20 million synapses.

The results, published in the first of a number of articles, can be freely analyzed by anyone interested in the way a fly thinks. A Google account can be used to log in to the neuPrint explorer and an interactive exploration of the 3D electron microscopy images is also available with neuroglancer. Extensive non-technical coverage by the media is also widely available. See, for instance, the article in The Economist or the piece in The Verge.

Image from the HHMI Janelia Research Campus site.