There have been a couple of brief discussions of this in the Open Thread, but it seems likely to generate more so here's a place for it.
The original paper in Nature about AlphaGo.
Google Asia Pacific blog, where results will be posted. DeepMind's YouTube channel, where the games are being live-streamed.
Discussion on Hacker News after AlphaGo's win of the first game.
'Yeah, we could maybe have AlphaGo learn everything totally from scratch and reach a superhuman level of knowledge just by playing itself, not using any human games for training material. Of course, reinventing everything that humanity has figured out while playing Go for the last 2,500 years, that's going to take quite a bit of time. Like a few months or so.'
Actually, the AlphaGo algorithm, this is something we’re going to try in the next few months — we think we could get rid of the supervised learning starting point and just do it completely from self-play, literally starting from nothing. It’d take longer, because the trial and error when you’re playing randomly would take longer to train, maybe a few months. But we think it’s possible to ground it all the way to pure learning.
http://www.theverge.com/2016/3/10/11192774/demis-hassabis-interview-alphago-google-deepmind-ai
I found this interesting: AlphaGo's internal statistics predicted victory with high confidence at about three hours into the game (Lee Sedol resigned at about three and a half hours):
For me, the key moment came when I saw Hassabis passing his iPhone to other Google executives in our VIP room, some three hours into the game. From their smiles, you knew straight away that they were pretty sure they were winning – although the experts providing live public commentary on the match weren’t clear on the matter, and remained confused up to the end of the game just before Lee resigned.
Hassabis’s certainty came from Google’s technical team, who pore over AlphaGo’s evaluation of its position, information that isn’t publicly available. I’d been asking Silver how AlphaGo saw the game going, and he’d already whispered back: “It’s looking good”.
And I realised I had a lump in my throat. From that point on, it was crushing for me to watch Lee’s struggle.
Towards the end of the match, Michael Redmond, an American commentator who is the only westerner to reach the top rank of 9 dan pro, said the game was still “very close”. But Hassabis was frowning and shaking his head – he knew that AlphaGo was definitely winning. And then Lee resigned, three and a half hours in.
Also this bit, suggesting that Lee might still win some matches:
Silver said that – judging from the statistics he’d seen when sitting in Google’s technical room – “Lee Sedol pushed AlphaGo to its limits”.
When I started hearing about the latest wave of results from neural networks, I thought to myself that Eliezer was probably wrong to bet against them. Should MIRI rethink its approach to friendliness?
Compared to its competition in the AGI race, MIRI was always going to be disadvantaged by both lack of resources and the need to choose an AI design that can predictably be made Friendly as opposed to optimizing mainly for capability. For this reason, I was against MIRI (or rather the Singularity Institute as it was known back then) going into AI research at all, as opposed to pursuing some other way of pushing for a positive Singularity.
In any case, what other approaches to Friendliness would you like MIRI to consider? The only other approach that I'm aware of that's somewhat developed is Paul Christiano's current approach (see for example https://medium.com/ai-control/alba-an-explicit-proposal-for-aligned-ai-17a55f60bbcf), which I understand is meant to be largely agnostic about the underlying AI technology. Personally I'm pretty skeptical but then I may be overly skeptical about everything. What are your thoughts? I don't recall seeing you having commented on them much.
Are you aware of any other ideas that MIRI should be considering?
Do you have a concise explanation of skepticism about the overall approach, e.g. a statement of the difficulty or difficulties you think will be hardest to overcome by this route?
Or is your view more like "most things don't work, and there isn't much reason to think this would work"?
In discussion you most often push on the difficulty of doing reflection / philosophy. Would you say this is your main concern?
My take has been that we just need to meet the lower bar of "wants to defer to human views about philosophy, and has a rough understanding of how humans want to reflect and want to manage their uncertainty in the interim."
Regarding philosophy/metaphilosophy, is it fair to describe your concern as one of:
My hope is that thinking and talking more about bootstrapping procedures would go a long way to resolving the disagreements between us (either leaving you more optimistic or me more pessimistic). I think this is most plausible if #1 is the main disagreement. If our disagreement is somewhere else, it may be worth also spending some time focusing somewhere else. Or it may be necessary to better define my lower bar in order to tell where the disagreement is.
It seems to be a combination of all of these.
EY was influenced by E.T. Jaynes, who was really against neural networks, in favor of bayesian networks. He thought NNs were unprincipled and not mathematically elegant, and bayes nets were. I see the same opinions in some of EY's writings, like the one you link. And the general attitude that "non-elegant = bad" is basically MIRI's mission statement.
I don't agree with this at all. I wrote a thing here about how NNs can be elegant, and derived from first principles. But more generally, AI should use whatever works. If that happens to be "scruffy" methods, then so be it.
But more generally, AI should use whatever works. If that happens to be "scruffy" methods, then so be it.
This seems like a bizarre statement if we care about knowable AI safety. Near as I can tell, you just called for the rapid creation of AGI that we can't prove non-genocidal.
I think that MIRI did a mistake than decided not be evolved in actual AI research, but only in AI safety research. In retrospect the nature of this mistake is clear: MIRI was not recognised inside AI community, and its safety recommendations are not connected with actual AI development paths.
It is like a person would decide not to study nuclear physics but only nuclear safety. It even may work until some point, as safety laws are similar in many systems. But he will not be the first who will learn about surprises in new technology.
I think that MIRI did a mistake than decided not be evolved in actual AI research [...] MIRI was not recognised inside AI community
Being involved in actual AI research would have helped with that only if MIRI had been able to do good AI research, and would have been a net win only if MIRI had been able to do good AI research at less cost to their AI safety research than the gain from greater recognition in the AI community (and whatever other benefits doing AI research might have brought).
I think you're probably correct that MIRI would be more effective if it did AI research, but it's not at all obvious.
Agreed on all points.
LW was one handshake away from DeepMind, we interviewed Shane Legg and referred to his work many times. But I guess we didn't have the right attitude, maybe still don't. Now is probably a good time to "halt, melt and catch fire" as Eliezer puts it.
https://gogameguru.com/alphago-defeats-lee-sedol-game-1/ has some (non-video) comments on the game, and promises more detailed commentary later.
9p Myungwan Kim's commentary (I much preferred this over the official commentary; he's also commenting tomorrow, so recommend following his stream then, though he might start an hour delayed like he did today).
Fun comment from him: "[AlphaGo] play like a god, a god of Go".
For me the most interesting part of this match was the part where one of the DeepMind team confirmed that because AlphaGo optimizes for probability of winning rather than expected score difference, games where it has the advantage will look close. It changes how you should interpret the apparent closeness of a game
Qiaochu Yuan, or him quoting someone.
Go champion Lee Se-dol strikes back to beat Google's DeepMind AI for first time in forth game 3:1 http://www.theverge.com/2016/3/13/11184328/alphago-deepmind-go-match-4-result
Several things I thought were interesting:
The commentator (on the Deepmind channel) calling out several of AlphaGo's moves as conservative. Essentially, it would play an additional stone to settle or augment some group that he wouldn't necessarily have played around. What I'm curious about is how much this reflects an attempt by AlphaGo to conserve computational resources. "I think move A is a 12 point swing, and move B is a 10 point swing, but move B narrows the search tree for future moves in a way that I think will net me at least 2 more points." (It wouldn't be verbalized like that, since it's not thinking verbally, but you can get this effect naturally from the tree search and position evaluator.)
Both players took a long time to play "obvious" moves. (Typically, by this I mean something like a response to a forced move.) 이 sometimes didn't--there were a handful of moves he played immediately after AlphaGo's move--but I was still surprised by the amount of thought that went into some of the moves. This may be typical for tournament play--I haven't watched any live before this.
AlphaGo's willingness to play aggressively and get involved in big fights with 이, and then not lose. I'm not sure that all the fights developed to AlphaGo's advantage, but evidently enough of them did by enough.
I somewhat regret 이 not playing the game out to the end; it would have been nice to know the actual score. (I'm sure estimates will be available soon, if not already.)
What I'm curious about is how much this reflects an attempt by AlphaGo to conserve computational resources.
If I understand correctly, at least according to the Nature paper, it doesn't explicitly optimize for this. Game-playing software is often perceived as playing "conservatively", this is a general property of minimax search, and in the limit the Nash equilibrium consists of maximally conservative strategies.
but I was still surprised by the amount of thought that went into some of the moves.
Maybe these obvious moves weren't so obvious at that level.
I'm quite interested in how many of the methods employed in this AI can be applied to more general strategic problems.
From talking to a friend who did quite a bit of work in machine composition, he was of the opinion that tools for handling strategy tasks like go would also apply strongly to many design tasks like composing good music.
Sure, you can model music composition as a RL task. The AI composes a song, then predicts how much a human will like it. It then tries to produce songs that are more and more likely to be liked.
Another interesting thing that alphago did, was start by predicting what moves a human would make. Then it switched to reinforcement learning. So for a music AI, you would start with one that can predict the next note in a song. Then you switch to RL, and adjust it's predictions so that it is more likely to produce songs humans like, and less likely to produce ones we don't like.
However automated composition is something that a lot of people have experimented with before. So far there is nothing that works really well.