I've just bumped into a fun link about guessing the teachers password. It is only 184 words so here it is:

I do a game called k9 nosework with my corgi. You take a qtip dipped in certain scents and hide it in boxes in the beginning then progressing to furniture and outside areas. The game is for the dog to find the qtip and identify; there's even a contest / sport. If you decide to do the sport, the handler has to read the dog and decide if the dog is identifying a given bit of car or whatever as the location of the goal scent or if the dog is just sniffing. The point of this rambling is even the plain trainers that do k9 nosework know that the handler has to go blind and can't know where the scent is or the dog will learn to read the handler instead of learning to use its nose to find the scent. The idea that the police are unaware of this is ludicrous, since this sport grew out of nose training for police dogs, and it's a major training obstacle.

The meat of the comment is "the handler has to go blind". It took me a while to work out what is being said here. My understanding:

You take up the sport. You hide the scented qtip yourself. You train your dog to sniff it out. Your dog does good. You go to the competition. The judge hides the scented qtip and you don't know where. Your dog does bad. Explanation: Your dog never learned to sniff out the scented qtip, he learned to read you and the tells you gave away when he sniffed near where you knew the qtip to be.

Why is this fun? Well, John Holt's book How Children Fail has a lovely story about guessing the teachers password

... since it was clear to me from the way the children were talking and acting that they hadn't a notion of what Nouns, Adjectives, and Verbs were. Finally one child said, 'Miss ---, you shouldn't point to the answer each time.' The teacher was surprised, and asked what she meant. The child said, 'Well, you don't exactly point, but you kind of stand next to the answer.' This was no clearer, since the teacher had been standing still. But after a while, as the class went on, I thought I saw what the girl meant. Since the teacher wrote each word down in its proper column, she was, in a way, getting herself ready to write, pointing herself at the place where she would soon be writing. From the angle of her body to the blackboard the children picked up a subtle cue to the correct answer.

Dogs do this too!

The relation of my comment to Eliezer's point is delicate. It is not correct to decenter and imagine yourself as the teacher; Eliezer is not writing pedagogy. You have to decenter and imagine yourself as the sniffer dog. You train hard, winning lots of praise and treats from your owner. You wag your tail happily. Then you go to the competition and discover that you haven't learned the skill that you were supposed to learn, at all!

To expand on my point, I think there is not enough testing in schools, and what testing there is is too associated with grades. Suppose that you are teaching a child to ride a bike, but suppose that each time he falls you give him a bad grade. At the end of it he knows how to ride a bike really well, and any honest assessment of him should be, "great, he learned well, he's done". Unfortunately, that is not what happens in school. Instead, all his stumbles on the way to learning are summarized and then put into the report card that his parents see.

Once the child has learned, why should it matter how many times he failed in the process of learning? What should matter is whether he knows now.

Because children are keen not to get poor grades, they are worried about never stumbling even once in any of the tests of their knowledge, because they know that each stumble will make it into their permanent record. In order to make it at least possible for students never to stumble, the tests must be designed so that with enough preparation students will not stumble. This limits what can be tested.

The test of getting on a bike and trying to ride will invariably result in stumble after stumble until the child eventually learns to ride. This kind of test is therefore necessarily excluded from the tests that the school will give children. And since such tests are key to learning important aspects of knowledge, schools will fail to teach those aspects of knowledge.

Imagine a bike riding course in which children are taught the theory of bike riding but not actually placed on a bike until the end of the semester. In fact, to ensure that at least some students will pass the test, even the final exam cannot place them on a bike, but will instead need to be a form where they regurgitate the bike riding theory they learned.

I am exaggerating of course, but I do see this as a pronounced tendency in learning.

I think of politics here: trying to guess the electorate's password. An elected official is rewarded for giving the answer that will get him the most votes. If that happens to be a well-conceived policy decision, that is a happy coincidence.

The hard part about teaching students not to guess the password is teaching teachers not to accept password guesses, and how to distinguish verbal behavior from thought.

"Then you may think that "Light is arglebargle" is a good explanation, that "arglebargle" is the correct password. It happened to me when I was nine years old - not because I was stupid, but because this is what happens by default. This is how human beings think, unless they are trained not to fall into the trap. Humanity stayed stuck in holes like this for thousands of years."

Okay, but there's one innocent interpretation even here. People learn language, and when we learn language we copy the verbal behavior of other people. Maybe "arglebargle" is a synonym for light in some language, or maybe it's a supercategory of light (a category that includes light among other things). Maybe the teacher is still in the process of explaining to us what arglebargle means and the first step is to say that light is arglebargle - later on the teacher will tell us what else is arglebargle so that we will gradually build a good concept of it but initially we need to retain the point that light is arglebargle while not yet knowing what arglebargle is, because this is a step in learning what arglebargle is. In that case, we're learning new language when we learn that "light is arglebargle". That's innocent, it's not a mistake.

This suggests that the error may not be learning the teacher's passwords per se, as such, but learning the teacher's passwords when we should ideally be learning something else. The context matters.

But the student is the student and therefore ignorant by assumption, so it may in many cases be too much to expect the student to know when it is time to learn the passwords and when it is time to learn something else. If the student experiences academic success only from learning the passwords, then it may be that the student is not at fault, it's the curriculum that is at fault - the teacher.

So the right recommendation may not be to tell the student to stop learning passwords. Passwords are a legitimate thing to learn, sometimes, and the student, being a student, doesn't know ahead of time which times. The right recommendation may be to adjust the curriculum so that only the right kind of learning yields academic success.

That's probably not easy.

In 1990, after seven years of teaching at Harvard, Eric Mazur, now Balkanski professor of physics and applied physics, was delivering clear, polished lectures and demonstrations and getting high student evaluations for his introductory Physics 11 course, populated mainly by premed and engineering students who were successfully solving complicated problems. Then he discovered that his success as a teacher “was a complete illusion, a house of cards.”

The epiphany came via an article in the American Journal of Physics by Arizona State professor David Hestenes. He had devised a very simple test, couched in everyday language, to check students’ understanding of one of the most fundamental concepts of physics—force—and had administered it to thousands of undergraduates in the southwestern United States. Astonishingly, the test showed that their introductory courses had taught them “next to nothing,” says Mazur: “After a semester of physics, they still held the same misconceptions as they had at the beginning of the term.”

The students had improved at handling equations and formulas, he explains, but when it came to understanding “what the real meanings of these things are, they basically reverted to Aristotelian logic—thousands of years back.” For example, they could recite Newton’s Third Law and apply it to numerical problems, but when asked about a real-world event like a collision between a heavy truck and a light car, many firmly declared that the heavy truck exerts a larger force. (Actually, an object’s weight is irrelevant to the force exerted.)

Mazur tried the test on his own students. Right at the start, a warning flag went up when one student raised her hand and asked, “How should I answer these questions—according to what you taught me, or how I usually think about these things?” To Mazur’s consternation, the simple test of conceptual understanding showed that his students had not grasped the basic ideas of his physics course: two-thirds of them were modern Aristotelians. “The students did well on textbook-style problems,” he explains. “They had a bag of tricks, formulas to apply. But that was solving problems by rote. They floundered on the simple word problems, which demanded a real understanding of the concepts behind the formulas.”

http://harvardmagazine.com/2012/03/twilight-of-the-lecture

Uh, Eliezer, when you were 9, had you seen a wave? Did you have a sense that a wave was different in several ways from other kinds of things, even from other kinds of fluid behaviors -- different from a current, say?

Now, when you were 9, had you seen an arglebargle? Did you have a sense of how arglebargles differed from other things? (If so, how?)

And are there ways in which the characteristics you did recognize in waves, when you were 9, also do in fact apply to light? On the other hand, are there ways in which the characteristics you recognize in arglebargles apply to light?

I would venture to say that you did learn something meaningful by repeating your teacher's explanation that light is "made of waves". You are overly discounting what counts as experimental evidence of the real-world phenomena which words refer to. You had a lot of experimental evidence of waves at age 9. You'd seen them, for a start. When your teacher told you that light is made of waves, he was helping you to understand something about light by explaining it in terms of something you had real-world experience of.

Similarly, when my 3-year-old asks whether stars are burning but planets are not, I don't tell him that actually stars aren't "burning" as such. I say yes, and what he said counts as learning, not "guessing my passwords" -- even though he doesn't fully understand yet what I would understand by the word "burning".

Disagreeing with the original post is a bit like disagreeing with the statement that there's too much crime. What I find remarkable, though, is how useful password knowledge is. I mean you might think it would have zero use, but it doesn't. For instance, all knowledge of geography is password knowledge, unless you have actually been to the place. Yet, most people will argue that knowledge of geography is a good thing because it makes you form little mental boxes that are useful for organizing future knowledge (e.g. there are some people from a place called China, and they're probably very different from people from a place called Botswana).

"But concise is not always precise, and without precision, concision is just vague at best, and misleading at worst. Several years ago, a student wanted to contest the scoring of one of his test answers in my introductory psychology course. The test question was something to the effect of “What is the primary advantage of an experimental study over a correlational study?” and an example sufficient answer would have been, “Causal conclusions may be drawn from an experiment, but not from a correlational study.”

The student’s answer was, “In an experiment, you actually test something” (the word ‘test’ was underlined twice). When he questioned why his answer earned no points, I explained that it failed to distinguish the two study types at all, as both correlational and experimental studies “test something” (i.e., in introductory terms, a relationship between variables and cause-and-effect, respectively). He looked at me earnestly and rebutted, “But you said some of the questions could be answered in one sentence, and I underlined ‘test’ twice.”

After a moment of silence (to contain my disbelief at his statement), I asked him what it would have meant if he had only underlined “test” once, or not at all, and how was I to know those distinctions in meaning. He had no answer."

--"Precision First", L. Kimberly Epting

I remember discovering this secret through a few related events. The one that comes forefront is my brother and I laughing at a magazine advert showing a sumo wrestler ski jumping. I made the comment, "Ha, he's going to fall like a rock!" My father was there and asked why? Didn't I know that the weight of the object doesn't determine its falling speed? Apparently, whenever that concept was taught to me it didn't stick and so I wasn't using it to make predictions. But based on the tone of his voice I knew I was supposed to know this and, even though I had no clue what he was talking about, I immediately responded by saying, "Because of drag!"

This was stupid of me. It got my dad off my back and so I had correctly guessed the teacher's password and the rest of the day I tried to figure out why the weight of an object wouldn't make it fall faster. It wasn't until years later that I saw proof in a science museum experiment and I accepted the theory as fact.

The event caused me to notice, however, that I had switched theories to try explaining my prediction. I made a bad prediction, but instead of saying oops I started desperately grabbing for evidence that backed up my result. How evil of my younger self. But now that I noticed I was doing this I was able to stop it. It also caused me to start seeing this happen in others. I am still surprised at how often people do this and never think that they could be doing something wrong.

"Constant, just because something is part of how humans learn language, does not make it okay."

Learning the passwords is okay in the context of learning language (I think). Take it away and you impede the ability to learn language. More generally, I think that learning empty slogans isn't wrong, it's merely incomplete. Part of learning is unintelligent aping, where we first learn to go through some of the motions without necessarily understanding them. You have to start somewhere. Understanding is complex and so necessarily requires passage through stages which do not each one constitute understanding, just as an office building as it is being built passes through many stages where what it is is just a useless structure that serves no immediate purpose. What is bad is not learning the passwords, it's failing to learn more beyond the passwords. Similarly, what is bad is not laying a foundation under a building, what is bad is abandoning the construction of the building after the foundation is laid but before the building is complete.

"a floating, non-anticipation-controlling belief that feels like ordinary knowledge, and is not labeled as a hint about someone else's word-associations"

It is almost necessarily the case that people who do not understand yet (e.g., students who are in the process of learning), do not yet have the understanding that it takes to understand that they do not understand yet. Knowing that you did not know is more easily done in hindsight once you know, than while you still do not know.

So the burden is probably better placed on the teacher to ensure that the students' education is not left incomplete.

I myself find it hard to see what the gaps are in my knowledge unless that knowledge is tested. So I think a good place to start in fixing whatever problem it is that you see, is in the testing of the students.

Although this is an old article I came to it from the Theory of Knowledge article (link below). I'm commenting because this crystallizes my objections to a repeated theme at LW: that irrationality comes from unquestioned cached thoughts, and that modern education systems exacerbate this tendency. In other words, I'm questioning whether password-guessing and memorization in education are actually avoidable, even at the highest levels of optimization, and whether this isn't in fact the result of the expansion of knowledge and the limits of human cognition.

I'm not even going to argue whether those are true statements and whether they are bad. But my own background has raised questions about what the solution might be. My idealistic assumptions about how people should learn used to track the predominant ones on LW much more closely. I'm currently in a technical graduate program for a profession that is different from most other types of graduate programs because in this field, the first few years are spent memorizing huge reams of information and taking multiple choice tests (have a guess as to what I'm studying?) My fellow students and I would genuinely love to really understand from a critical perspective all the information that is getting shoveled into our brains. But doing that, rather than memorizing lists of word correspondences, takes time that we just don't have. Therefore in reality our tests are really just exercises in figuring out which bar to press so we get the food pellet. (I like that analogy better than "guessing passwords".) And perhaps most scandalous, once we're "in the system", we students recognize the need for it to be this way even though it's frustrating.

Scandal! Unethical? Stupid? Cynical, at least? Maybe yes to all. But the pragmatic reality is that there's a trade-off. There's such a massive amount of material that no human who ever lived would be able to critically digest it all unless the training lasted far longer than it does, and while the system is certainly not optimized, the body of knowledge itself is so rife with detail not predictable from first principles that until the limits on cognition fundamentally change, this will continue to be the case.

So, regarding the strategy students can adopt, the two ends of the spectrum are: 1) You can just memorize details long enough to regurgitate them in the lever-pressing experiments, and understand nothing critically; or 2) You can insist on trying to critically understand everything, and you will certainly fall behind and fail. You. Yes, you, reading this, because you're human, and this applies to all humans. ("Not me! I'm special! I still care about my fellow man, the human spirit, etc." Sorry, can you tell I've had this conversation before?)

Unfortunately, given the pace the material is presented and tested, you'll end up much closer to #1 than #2. Even if you disagree with my assessment that a body of knowledge can be such that large amounts of memorization are necessary, this still raises the omnipresent problem of how to maximize whatever it is you want to maximize while surrounded by irrational humans (who are running the schools, and who believe that memorization is necessary). If you want to do the thing you're being trained for, you need the piece of paper. To get that paper, you have to pass the tests. To pass the tests you need to just memorize and not think. I guess I could just declare the whole enterprise unethical and forget about this profession and move out in the woods somewhere. So if you can tell me how to maximize career satisfaction and income out in a nice forest away from everyone instead of memorizing reams of B.S., I'm all ears. Seriously.

As knowledge accumulates, this problem is only going to get worse, and extend to more fields. Certainly education has not yet been optimized but all fields are not math and physics, and there's enough unpredictable detail in the world that as knowledge accumulates, so will the memorization required to learn that knowledge. And if we can't always use critical thinking at every step - which we can't - then password-guessing is better than nothing.

Post I came in from: http://lesswrong.com/lw/70d/theory_of_knowledge_rationality_outreach/

It seems to me that passwords and placeholders occupy about the same space, and probably look fairly similar when they're first being taught. Knowing that light, sound, and matter are all waves tells me they have something in common. It means once I learn what a wave is, I ought to be able to predict some behaviors of all three based on this new information. If I also know that some things are not waves, I'll have a decent foundation for when to apply the wave equation, once I learn it.

Indeed, "password" itself seems to be a password for the concept of a placeholder. Now that I've puzzled that out, though, the new question arises: is it actually useful to learn a placeholder, or should we teach the wave equation first, and then go in to "light is a wave, and here is why", "sound is a wave and here is why"?

Certainly, being up-front about it when we're using a placeholder or an approximation would be wise. I must have had an odd experience in school, since there was generally at least one other student that would ask "what does 'wave' MEAN?" and then we'd all either learn what it meant, or that it was a placeholder to be learned about later.

"Imagine a bike riding course in which children are taught the theory of bike riding but not actually placed on a bike until the end of the semester. In fact, to ensure that at least some students will pass the test, even the final exam cannot place them on a bike, but will instead need to be a form where they regurgitate the bike riding theory they learned."

You think that passage is a joke, but it's an exact description of my high school's driver's education unit.

An interesting little demonstration is to pose the "How old is the shepherd?" word problem / riddle to kids at school: http://robertkaplinsky.com/how-old-is-the-shepherd/

When I was a kid I had this book called "Thinking Physics", which was basically a book of multiple choice physics questions (such as "an elephant and a feather are falling, which one experiences more air resistance ?", or "Kepler and Galileo made telescopes around the same time and Kepler's was adopted widely, why ?") aimed to point out where our natural instincts or presuppositions go against how physics actually work, and explaining, well, how physics actually work.

Really, the simple idea that physics are a habit of thought that have to be worked on because our defaults are incorrect (or, as I realized much later, are correct only in the special case of the everyday life of a social hominid) has been helpful to me ever since, and too few people have it or realize it's important.

I think it gets to what you're saying : one shouldn't learn physics (or anything for that matter) as a list of facts or methods to apply in the classroom, one should work to integrate them into one's mental model of the world. Which is not as easy as it sounds.

"If you aren't using the diffusion equation - putting in numbers and getting out results that control your anticipation of particular experiences - then the connection between map and territory is severed as though by a knife. What remains is not a belief, but a verbal behavior."

I'm surprised noone's commented on this before, but I think this is overly restrictive. If I'm familiar with the process of diffusion, I intuitively know what's going to happen, without actually plugging in numbers. I don't need to do math to reach the right conclusion, I can just sort of visualize what's happening. I think you progress from mechanically plugging in numbers to generalizing the behavior into a pattern that you can apply, like people can catch a ball in the air without actually calculating a parabola. That doesn't mean that there is no connection between map and territory. Of course, you have to take some care to make sure you're applying the right pattern, but that's no different than knowing to apply the right equation.

On the other hand, if techers make first-graders intuitively predict what 2+2 equals, they'll never get around to making them anticipate who it was that discovered America. There are a lot of passwords. Plus, the kids just won't work that hard.

There's a lovely bit in Egan's Diaspora showing the viewpoint character understanding a concept from physics by applying it in various contexts.

More generally, I don't know if much is known about how people get from input to understanding.

Possibly of interest: Mathsemantics, which grew out of a project to find employees who understood what numbers mean. The book (about a questionaire for the purpose) is very interesting, the articles listed mostly look minor except for the one about grokduelling (you win if you understand the other side better), and they're looking for research ideas.

my summary:

• For abstract concepts we need to drill down to relate them to empirical testing

• We shouldn’t be satisfied with just saying the keywords, but seek to genuinly understand what’s going on.

• The human brains tends to value what it gives social approval than which it matches physical reality.

• Test as much as possible.

Some of the critics of Eliezer's point are falling in the trap of justifying a functional system rather than looking how to optimize it. I think Constant's point that we often regurgitate on the path to understanding. But still, I think primacy should be given to figuring out how to optimize these learning processes, rather than justify the functional elements of the learning processes we currently have.

Constant, just because something is part of how humans learn language, does not make it okay. We know that something is arglebargle without knowing what arglebargle is, but, this is not labeled as "a hint about someone else's mental associations to a word", it is labeled as knowing that something is arglebargle. That's the error, right there: a floating, non-anticipation-controlling belief that feels like ordinary knowledge, and is not labeled as a hint about someone else's word-associations. Just because this is part of how human minds learn language, still doesn't make it okay. The human mind has a crappy design. That's what blogs like Overcoming Bias are for.

I notice that I am confused by this post.

Is the claim that this is a school thing or a life thing? I can see how this behavior might happen if a student is more interested in getting good grades than in actual learning. In such a situation "learning the teachers password" might be a short cut to get to your actual goals.

If the claim is that this is a life thing, could some one give me some more non-classroom example? Organized religion counts as classroom.

When I fist heard that light is a wave, then I interpreted that sentence in my brain an gave it meaning. I can't say for sure that I gave it the correct meaning. But I defiantly know that I did not just save a way the sound pattern, as truth. Because I don't think that way, and I can't even imagine thinking that way.

I can, on the other hand, imagine thinking: "If I write 'because of heat conduction' on a test, I have a chance of getting points." This is not how I went though school, because I was interested in accusal learning, but I can model a student who thinks this way.

"If I write 'because of heat conduction' on a test, I have a chance of getting points." is an anticipation controller.

Eliezer, thank goodness you exist.