If it's worth saying, but not worth its own post (even in Discussion), then it goes here.
Notes for future OT posters:
1. Please add the 'open_thread' tag.
2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)
3. Open Threads should be posted in Discussion, and not Main.
4. Open Threads should start on Monday, and end on Sunday.
http://mindhacks.com/2015/11/16/no-more-type-iii-error-confusion/#comments
Use "The Boy Who Cried Wolf" as a mnemonic. His first error is type 1 (claiming a wolf as present when there wasn't one). His second error is type 2 (people don't notice an existing wolf).
The latest New Yorker has a lengthy article about Nick Bostrom and Superintelligence. It contains a good profile of Bostrom going back to his graduate school days, his interest in existential threats in general, and how that interest became more focused on the risk of AGI specifically. Many concepts frequently discussed at LW are mentioned, e.g. the Fermi paradox and the Great Filter, the concept of an intelligence explosion, uploading, cryonics, etc. Also discussed is the progress that Bostrom and others have made in getting the word out regarding the threat posed by AGI, as well as some opposing viewpoints. Various other AI researchers, entrepreneurs and pundits are mentioned as well (although neither EY nor LW is mentioned, unfortunately).
The article is aimed at a general audience and so it doesn't contain much that will be new to the typical LWer, but it is an interesting and well-done overview, IMO.
LW displays notifications about replies and private messages in the same place, mixed together, looking the same. Note that top-level comments on your articles are also considered replies to you (this is a default behavior, you could turn it off, but it makes sense so you will probably leave it turned on).
This has the disadvantage that when you post an article which receives about 20 comments and someone sends you a private message, it is very easy to miss the message. Because in your inbox you just see 21 entries that look almost the same.
Suggestion: The easiest fix would probably be to change the appearance of private messages in your inbox. Make the difference obvious, so you can't miss it. For example, add a big icon above each private message.
As soon as I have two Karma points, I will post a 2000 word article on bias in most LW posts (which I would love to have your feedback on) with probably more to follow. However, I don't want to search for some more random rationality quotes to meet that requirement. Note to the administrators: Either you are doing a fabulous job at preventing multiple accounts or registration is currently not working (tried multiple devices, email addresses, and other measures).
I've been hearing about all this amazing stuff done with recurrent neural networks, convolutional neural networks, random forests, etc. The problem is that it feels like voodoo to me. "I've trained my program to generate convincing looking C code! It gets the indentation right, but the variable use is a bit off. Isn't that cool?" I'm not sure, it sounds like you don't understand what your program is doing. That's pretty much why I'm not studying machine learning right now. What do you think?
ML is search. If you have more parameters, you can do more, but the search problem is harder. Deep NN is a way to parallelize the search problem with # of grad students (by tweaks, etc.), also a general template to guide local-search-via-gradient (e.g. make it look for "interesting" features in the data).
I don't mean to be disparaging, btw. I think it is an important innovation to use human AND computer time intelligently to solve bigger problems.
In some sense it is voodoo (not very interpretable) but so what? Lots of other solutions to problems are, too. Do you really understand how your computer hardware or your OS work? So what if you don't?
In some sense it is voodoo (not very interpretable)
There is research in that direction, particularly in the field of visual object recognising convolutional networks. It is possible to interpret what a neural net is looking for.
I did my PhD thesis on a machine learning problem. I initially used deep learning but after a while I became frustrated with how opaque it was so I switched to using a graphical model where I had explicitly defined the variables and their statistical relationships. My new model worked but it required several months of trying out different models and tweaking parameters, not to mention a whole lot of programming things from scratch. Deep learning is opaque but it has the advantage that you can get good results rapidly without thinking a lot about the problem. That's probably the main reason that it's used.
The problem is that it feels like voodoo to me.
Clarke's Third Law :-)
Anyway, you gain understanding of complicated techniques by studying them and practicing them. You won't understand them unless you study them -- so I'm not sure why you are complaining about lack of understanding before even trying.
RNNs and CNNs are both pretty simple conceptually, and to me they fall into the class of "things I would have invented if I had been working on that problem," so I suspect that the original inventors knew what they were doing. (Random forests were not as intuitive to me, but then I saw a good explanation and realized what was going on, and again suspect that the inventor knew what they were doing.)
There is a lot of "we threw X at the problem, and maybe it worked?" throughout all of science, especially when it comes to ML (and statistics more broadly), because people don't really see why the algorithms work.
I remember once learning that someone had discretized a continuous variable so that they could fit a Hidden Markov Model to it. "Why not use a Kalman filter?" I asked, and got back "well, why not use A, B, or C?". At that point I realized that they didn't know that a Kalman filter is basically the continuous equivalent of a HMM (and thus obviously more appropriate, especially since they didn't have any strong reason to suspect non-Gaussianity), and so ended the conversation.
it sounds like you don't understand what your program is doing
That is ambiguous. Do you mean the final output program or the ML program?
Most ML programs seem pretty straight-forward to me (search, as Ilya said); the black magic is the choice of hyperparameters. How do people know how many layers they need? Also, I think time to learn is a bit opaque, but probably easy to measure. In particular, by mentioning both CNN and RNN, you imply that the C and R are mysterious, while they seem to me the most comprehensible part of the choices.
But your further comments suggest that you mean the program generated by the ML algorithms. This isn't new. Genetic algorithms and neural nets have been producing incomprehensible results for decades. What has changed is that new learning algorithms have pushed neural nets further and judicious choice of hyperparameters have allowed them to exploit more data and more computer power, while genetic algorithms seem to have run out of steam. The bigger the network or algorithm that is the output, the more room for it to be incomprehensible.
http://boingboing.net/2015/11/16/our-generation-ships-will-sink.html
Kim Stanley Robinson, author of the new scifi novel Aurora and back in the day the Mars trilogy, on how the notion of interstellar colonization and terraforming is really fantasy and we shouldnt let it color our perceptions of the actual reality we have, and the notion of diminishing returns on technology.
He doesnt condemn the genre but tries to provide a reality check for those who take their science fiction literally.
Um, no, we cannot colonise the stars with current tech. What a surprise! We cannot even colonise mars, antarctica or the ocean floor.
Of course you need to solve bottom up manufacturing (nanotech or some functional eqivalent) first, making you independent from eco system services, agricultural food production, long supply chains and the like. This also vastly reduces radiation problems and probably solves ageing. Then you have a fair chance.
So yes, if we wreck earth the stars are not plan B, we need to get our shit together first.
If at this point there is still a reason to send canned monkeys is a completely different question.
I found out about Omnilibrium a couple months ago, and I was thinking of joining in eventually. I was also thinking of telling some friends of mine who might want to get in on it even more than I do about it. However, I've been thinking if I told lots of people, or they themselves told lots of people, then suddenly Omnilibrium might get flooded with dozens of new users at once. I don't know how big that is compared to the whole community, but I was thinking Omnilibrium would be averse to it growing it too big, as well-kempt gardens die by pacificism and all that. But then, Slate Star Codex linked to it a few weeks ago. So, that's plausibly hundreds of new users flooding it.
I'm wondering, how do the admins of Omnilibrium feel about this? Are you happy to have many new users? Are you upset SSC linked to Omnilibrium, bringing it to the attention of so many people who may not necessarily maintain the quality of discourse current users of Omnilibrium have gotten accustomed to?
The whole point of the site is to do automated moderation and curation. At the moment, it is so small that it is serving no purpose better than a human dictator would. The whole point is that algorithms can scale. Maybe the algorithms aren't yet ready for prime time and maybe it's better if it grows slowly so that they have time to understand how to modify the algorithms. In particular, I believe that it currently grades users on a single political axis, while with more users it would probably be better to have a more complicated clustering scheme. But you probably won't cause it to grow rapidly, anyhow.
Recommended: a conversation between Tyler Cowen and Cliff Asness about financial markets. Especially recommended for people who insist that markets are fully efficient.
Samples:
A momentum investing strategy is the rather insane proposition that you can buy a portfolio of what’s been going up for the last 6 to 12 months, sell a portfolio of what’s been going down for the last 6 to 12 months, and you beat the market. Unfortunately for sanity, that seems to be true.
and
One thing I should really be careful about. I throw out the word “works.” I say “This strategy works.” I mean “in the cowardly statistician fashion.” It works two out of three years for a hundred years. We get small p-values, large t-statistics, if anyone likes those kind of numbers out there. We’re reasonably sure the average return is positive. It has horrible streaks within that of not working.
and
...what’s the actual vision of human nature? What’s the underlying human imperfection that allows it to be the case, that trading on momentum across say a 3 to 12 month time window, sorry, investing on momentum, will work? What’s with us as people? What’s the core human imperfection?
Disinformation review, a weekly publication, which collects examples of the Russian disinformation attacks.
The main aim of this product is to raise the awareness about Russian disinformation campaign. And the way to achieve this goal is by providing the experts in this field, journalists, academics, officials, politicians, and anyone interested in disinformation with some real time data about the number of disinformation attacks, the number of countries targeted, the latest disinformation trends in different countries, the daily basis of this campaign, and about the coordination of the disinformation spread among many countries.
Our global network of journalists, government officials, academics, NGOs, think tanks (and other people / initiatives dealing with this issue) provides us with the examples of current disinformation appearing in their countries. East StratCom Task Force compiles their reports and publishes a weekly review of them. The document with the data collected is public and free for further use – journalists may use it as a source for their products, decision makers and government officials may find relevant information about the latest state of events, experts may find data for their analysis, NGOs and think tanks may share the knowledge about this issue with the rest of the world.