Mastering Starcraft

The researchers at DeepMind keep advancing the state of the art on the utilization of deep learning to master ever more complex games. After recently reporting a system that learns how to play a number of different and very complex board games, including Go and Chess, the company announced a system that is able to beat the best players in the world at a complex strategy game, Startcraft.

AlphaStar, the system designed to learn to play Starcraft, one of the most challenging Real-Time Strategy (RTS) games, by playing against other versions of itself, represents a significant advance in the application of machine learning. In Starcraft, a significant amount of information is hidden from the players, and each player has to balance short term and long term objectives, just like in the real world. Players have to master fast-paced battle techniques and, at the same time, develop their own armies and economies.

This result is important because it shows that deep reinforcement learning, which has already shown remarkable results in all sorts of board games,  can scale up to complex environments with multiple time scales and hidden information. It opens the way to the application of machine learning to real-world problems, until now deemed to difficult to be tackled by machine learning.

Advertisements

Deepmind presents Artificial General Intelligence for board games

In a paper recently published in the journal Science, researchers from DeepMind describe Alpha Zero, a system that mastered three very complex games, Go, chess, and shogi, using only self-play and reinforcement learning. What is different in this system (a preliminary version was previously referred in this blog), when compared with previous ones, like AlphaGo Zero, is that the same learning architecture and hyperparameters were used to learn different games, without any specific customization for each different game.
Historically, the best programs for each game were heavily customized to use and exploit specific characteristics of that game. AlphaGo Zero, the most impressive previous result, used the spatial symmetries of Go and a number of other specific optimizations. Special purpose chess program like Stockfish took years to develop, use enormous amounts of field-specific knowledge and can, therefore, only play one specific game.
Alpha Zero is the closest thing to a general purpose board game player ever designed. Alpha Zero uses a deep neural network to estimate move probabilities and position values. It performs the search using a Monte Carlo tree search algorithm, which is general-purpose and not specifically tuned to any particular game. Overall, Alpha Zero gets as close as ever to the dream of artificial general intelligence, in this particular domain. As the authors say, in the conclusions, “These results bring us a step closer to fulfilling a longstanding ambition of Artificial Intelligence: a general game-playing system that can master any game.
While mastering these ancient games, AlphaZero also teaches us a few things we didn’t know about the games. For instance, that, in chess, white has a strong upper hand when playing the Ruy Lopez opening, or when playing against the French and Caro-Kann defenses. Sicilian defense, on the other hand, gives black much better chances. At least, that is what the function learned by the deep neural network obtains…
Actualization: The NY Times just published an interesting piece on this topic, with some additional information.

The Evolution of Everything, or the use of Universal Acid, by Matt Ridley

Matt Ridley never disappoints but his latest book, The Evolution of Everything is probably the most impressive one. Daniel Dennett called evolution the universal acid, an idea that dissolves every existing preconception we may have about the world. Ridley uses this universal acid to show that the ideas behind evolution apply not only to living beings but to all sorts of things in the world and, particularly, to society. The universal acid is used by Ridley to deconstruct our preconceptions about history and to present his own view that centralized control does not work and that bottom-up driven evolution is the engine behind progress.

When Ridley means everything, he is not exaggerating. The chapters in this book cover, among many others, topics as different as the universe, life, moral, culture, technology, leadership, education, religion, and money. To all these topics Ridley applies the universal acid to arrive at the conclusion that (almost) all thas is planned and directed leads to bad results, and that all that evolves by the pressures of competition and natural selection provides advances and improvements in society. Bottom-up mechanisms, he argues, are what creates innovation in the world, be it in the natural world, in culture, in technology or in any other area of society. To this view, he gives explicit credit to Lucretius who, in his magnum opus The Rerum Natura from the fourth century BC, proposed essentially the same idea, and to Adam Smith’s who, in The Wealth of Nations, proposed the central role of commerce in the development of society.

Sometimes, his arguments look too farfetched like, for instance, when he argues that the state should stay out of the education business, or that the 2008 crisis was caused not by runaway private initiative but by wrong governmental policies. Nonetheless, even in these cases, the arguments are very persuasive and always entertaining. Even someone like me, who believes that there are some roles to be played by the state, ends up doubting his own convictions.

All in all, a must read.

 

Uber temporarily halts self-driving cars on the wake of fatal accident

Uber decided to halt all self-driving cars operations following a fatal accident involving an Uber car driving in autonomous mode, in Tempe, Arizona. Although the details are sketchy, Elaine Herzberg, a 49-year-old woman was crossing the street, outside the crosswalk,  in her bike, when she was fatally struck by a Volvo XC90 outfitted with the company’s sensing systems, in autonomous mode. She was taken to the hospital, where she later died as a consequence of the injuries. A human safety driver was behind the wheel but did not intervene.  The weather was clear and no special driving conditions have been reported but reports say she crossed the road suddenly, coming from a poorly lit area.

The accident raised concerns about the safety of autonomous vehicles, and the danger they may cause to people. Uber has decided to halt all self-driving car operations, pending investigation of the accident.

Video released by the Tempe police shows the poor light conditions and the sudden appearance of the woman with the bike. From the video, the collision looks unavoidable, by looking only at camera images. Other sensors, on the other hand, might have helped.

In 2016, about 1 person has died in traffic accidents, per each 100 million miles travelled by cars. Uber has, reportedly, logged 3 million miles in its autonomous vehicles. Since no technology will reduce the number of accidents to zero, further studies will be required to assess the comparative safety of autonomous vs. non-autonomous vehicles.

Photo credits: ABC-15 via Associated Press.

Nectome, a Y-combinator startup, wants to upload your mind

Y-combinator is a well known startup accelerator, which accepts and supports startups developing new ideas. Well-known companies, like Airbnb, Dropbox and Unbabel were incubated there, as were many others which became successful.

Wild as the ideas pitched at Y-combinator may be, however, so far no proposal was as ambitious as the one pitched by Nectome, a startup that wants to backup your mind. More precisely, Nectome wants to process and chemically preserve your brain, down to its most detailed structures, in order to make it possible to upload your mind sometime in the future. Robert McIntyre, founder and CEO of Nectome, and an MIT graduate, will pitch his company in a meeting in New York, next week.

Nectome’s is committed to the goal of archiving your mind, as goes the description in the website, by building the next generation of tools to preserve the connectome, the pattern of neuron interconnections that constitutes a brain. Nectome’s technology uses a process known as vitrifixation (also known as Aldehyde-Stabilized Cryopreservation) to stabilize and preserve a brain, down to its finer structures.

The idea is to keep the physical structure of the brain intact for the future (even though that will involve destroying the actual brain) in the hope that you may one day reverse engineer and reproduce, in the memory of a computer, the working processes of that brain. This idea, that you may be able to simulate a particular brain in a computer, a process known as mind uploading is, of course, not novel. It was popularized by many authors, most famously by Ray Kurzweil,  in his books. It has also been addressed in many non-fiction books, such as Superintelligence and The Digital Mind, both featured in this blog.

Photo by Nectome

 

 

AIs running wild at Facebook? Not yet, not even close!

Much was written about two Artificial Intelligence systems developing their own language. Headlines like “Facebook shuts down down AI after it invents its own creepy language” and “Facebook engineers panic, pull plug on AI after bots develop their own language” were all over the place, seeming to imply that we were just at the verge of a significant incident in AI research.

As it happens, nothing significant really happened, and these headlines are only due to the inordinate appetite of the media for catastrophic news. Most AI systems currently under development have narrow application domains, and do not have the capabilities to develop their own general strategies, languages, or motivations.

To be fair, many AI systems do develop their own language. Whenever a neural network is trained to perform pattern recognition, for instance, a specific internal representation is chosen by the network to internally encode specific features of the pattern under analysis. When everything goes smoothly, these internal representations correspond to important concepts in the patterns under analysis (a wheel of car, say, or an eye) and are combined by the neural network to provide the output of interest. In fact, creating these internal representations, which, in a way, correspond to concepts in a language, is exactly one of the most interesting features of neural networks, and of deep neural networks, in particular.

Therefore, systems creating their own languages are nothing new, really. What happened with the Facebook agents that made the news was that two systems were being trained using a specific algorithm, a generative adversarial network. When this training method is used, two systems are trained against each other. The idea is that system A tries to make the task of system B more difficult and vice-versa. In this way, both systems evolve towards becoming better at their respective tasks, whatever they are. As this post clearly describes, the two systems were being trained at a specific negotiation task, and they communicated using English words. As the systems evolved, the systems started to use non-conventional combinations of words to exchange their information, leading to the seemingly strange language exchanges that led to the scary headlines, such as this one:

Bob: I can i i everything else

Alice: balls have zero to me to me to me to me to me to me to me to me to

Bob: you i everything else

Alice: balls have a ball to me to me to me to me to me to me to me to me

Strange as this exchange may look, nothing out of the ordinary was really happening. The neural network training algorithms were simply finding concept representations which were used by the agents to communicate their intentions in this specific negotiation task (which involved exchanging balls and other items).

The experience was stopped not because Facebook was afraid that some runaway explosive intelligence process was underway, but because the objective was to have the agents use plain English, and not a made up language.

Image: Picture taken at the Institute for Systems and Robotics of Técnico Lisboa, courtesy of IST.

IEEE Spectrum special issue on whether we can duplicate a brain

Maybe you have read The Digital Mind or The Singularity is Near, by Ray Kurzweil, or other similar books, thought it all a bit farfetched, and wondered whether the authors are bonkers or just dreamers.

Wonder no more. The latest issue of the flagship publication of the Institute for Electrical and Electronic Engineers, IEEE Spectrum , is dedicated to the interesting and timely question of whether we can copy the brain, and use it as blueprint for intelligent systems.  This issue, which you can access here, includes many interesting articles, definitely worth reading.

I cannot even begin to describe here, even briefly, the many interesting articles in this special issue, but it is worthwhile reading the introduction, on the perspective of near future intelligent personal assistants or the piece on how we could build an artificial brain right now, by Jennifer Hasler.

Other articles address the question on how expensive, computationally, is the simulation of a brain at the right level of abstraction. Karlheinz Meier’s article on this topic explains very clearly why present day simulations are so slow:

“The big gap between the brain and today’s computers is perhaps best underscored by looking at large-scale simulations of the brain. There have been several such efforts over the years, but they have all been severely limited by two factors: energy and simulation time. As an example, consider a simulation that Markus Diesmann and his colleagues conducted several years ago using nearly 83,000 processors on the K supercomputer in Japan. Simulating 1.73 billion neurons consumed 10 billion times as much energy as an equivalent size portion of the brain, even though it used very simplified models and did not perform any learning. And these simulations generally ran at less than a thousandth of the speed of biological real time.

Why so slow? The reason is that simulating the brain on a conventional computer requires billions of differential equations coupled together to describe the dynamics of cells and networks: analog processes like the movement of charges across a cell membrane. Computers that use Boolean logic—which trades energy for precision—and that separate memory and computing, appear to be very inefficient at truly emulating a brain.”

Another interesting article, by Eliza Strickland, describes some of the efforts that are taking place to use  reverse engineer animal intelligence in order to build true artificial intelligence , including a part about the work by David Cox, whose team trains rats to perform specific tasks and then analyses the brains by slicing and imaging them:

“Then the brain nugget comes back to the Harvard lab of Jeff Lichtman, a professor of molecular and cellular biology and a leading expert on the brain’s connectome. ­Lichtman’s team takes that 1 mm3 of brain and uses the machine that resembles a deli slicer to carve 33,000 slices, each only 30 nanometers thick. These gossamer sheets are automatically collected on strips of tape and arranged on silicon wafers. Next the researchers deploy one of the world’s fastest scanning electron microscopes, which slings 61 beams of electrons at each brain sample and measures how the electrons scatter. The refrigerator-size machine runs around the clock, producing images of each slice with 4-nm resolution.”

Other approaches are even more ambitious. George Church, a well-known researcher in biology and bioinformatics, uses sequencing technologies to efficiently obtain large-scale, detailed information about brain structure:

“Church’s method isn’t affected by the length of axons or the size of the brain chunk under investigation. He uses genetically engineered mice and a technique called DNA bar coding, which tags each neuron with a unique genetic identifier that can be read out from the fringy tips of its dendrites to the terminus of its long axon. “It doesn’t matter if you have some gargantuan long axon,” he says. “With bar coding you find the two ends, and it doesn’t matter how much confusion there is along the way.” His team uses slices of brain tissue that are thicker than those used by Cox’s team—20 μm instead of 30 nm—because they don’t have to worry about losing the path of an axon from one slice to the next. DNA sequencing machines record all the bar codes present in a given slice of brain tissue, and then a program sorts through the genetic information to make a map showing which neurons connect to one another.”

There is also a piece on the issue of AI and consciousness, where Christoph Koch and Giulio Tononi describe their (more than dubious, in my humble opinion) theory on the application of Integrated Information Theory to the question of: can we quantify machine consciousness?

The issue also includes interesting quotes and predictions by famous visionairies, such as Ray Kurzweil, Carver Mead, Nick Bostrom, Rodney Brooks, among others.

Images from the special issue of IEEE Spectrum.