Once upon a time, there was an instructor who taught physics students. One day she called them into her class, and showed them a wide, square plate of metal, next to a hot radiator. The students each put their hand on the plate, and found the side next to the radiator cool, and the distant side warm. And the instructor said, Why do you think this happens? Some students guessed convection of air currents, and others guessed strange metals in the plate. They devised many creative explanations, none stooping so low as to say "I don't know" or "This seems impossible."
And the answer was that before the students entered the room, the instructor turned the plate around.
Consider the student who frantically stammers, "Eh, maybe because of the heat conduction and so?" I ask: is this answer a proper belief? The words are easily enough professed—said in a loud, emphatic voice. But do the words actually control anticipation?
Ponder that innocent little phrase, "because of", which comes before "heat conduction". Ponder some of the other things we could put after it. We could say, for example, "Because of phlogiston", or "Because of magic."
"Magic!" you cry. "That's not a scientific explanation!" Indeed, the phrases "because of heat conduction" and "because of magic" are readily recognized as belonging to different literary genres. "Heat conduction" is something that Spock might say on Star Trek, whereas "magic" would be said by Giles in Buffy the Vampire Slayer.
However, as Bayesians, we take no notice of literary genres. For us, the substance of a model is the control it exerts on anticipation. If you say "heat conduction", what experience does that lead you to anticipate? Under normal circumstances, it leads you to anticipate that, if you put your hand on the side of the plate near the radiator, that side will feel warmer than the opposite side. If "because of heat conduction" can also explain the radiator-adjacent side feeling cooler, then it can explain pretty much anything.
And as we all know by this point (I do hope), if you are equally good at explaining any outcome, you have zero knowledge. "Because of heat conduction", used in such fashion, is a disguised hypothesis of maximum entropy. It is anticipation-isomorphic to saying "magic". It feels like an explanation, but it's not.
Supposed that instead of guessing, we measured the heat of the metal plate at various points and various times. Seeing a metal plate next to the radiator, we would ordinarily expect the point temperatures to satisfy an equilibrium of the diffusion equation with respect to the boundary conditions imposed by the environment. You might not know the exact temperature of the first point measured, but after measuring the first points—I'm not physicist enough to know how many would be required—you could take an excellent guess at the rest.
A true master of the art of using numbers to constrain the anticipation of material phenomena—a "physicist"—would take some measurements and say, "This plate was in equilibrium with the environment two and a half minutes ago, turned around, and is now approaching equilibrium again."
The deeper error of the students is not simply that they failed to constrain anticipation. Their deeper error is that they thought they were doing physics. They said the phrase "because of", followed by the sort of words Spock might say on Star Trek, and thought they thereby entered the magisterium of science.
Not so. They simply moved their magic from one literary genre to another.
Part of the sequence Mysterious Answers to Mysterious Questions
Next post: "Guessing the Teacher's Password"
Previous post: "Hindsight Devalues Science"
I disagree. The proper response to not knowing the answer is to admit to not knowing and then give your best guess, not to try to hide your ignorance, because if you succeed then the teacher doesn't know you need help. A student who is more concerned with not displaying ignorance than with not being ignorant is not trying to learn, which is not rational. That which can be destroyed by the truth should be, and it probably won't be if you try to avoid finding out what the truth is.
The key phrase here is "on average". If you guess at random from all possible explanations of a given phenomenon, you will, on average, die before guessing the correct answer. There is a reason the monkeys with typewriters are given infinite time to reproduce Hamlet.
Moreover, as the set of answers considered increases in size, the expected utility from giving any one answer tends towards the expected utility of a wrong answer. As long as giving the wrong answer gives less utility than admitting ignorance, admitting ignorance is almost always the utility maximising option if you don't know.
If I write down a number and then take a number from a table of random numbers, and the numbers are the same, does this mean that I'm psychic? Because if getting the correct answer means that I have useful anticipation controllers then I must be.
"I don't know" is not an excuse for not knowing. That makes no sense whatsoever. "I don't know" is a statement about whether I know something or not, not a statement about whether I ought to know. If you can't admit fallibility then you will never learn anything.
The points you make about the benefits of testing students' knowledge are true. Unfortunately, they miss the point - while it is important not to penalise guessing incorrectly, so as not to dissuade admitting ignorance, it is much better to actively reward admitting that you have tried and failed. If a confused student does not always seek an explanation, the reward for seeking explanations isn't large enough yet. If students are content to remain ignorant, something is seriously wrong with your system for making students less ignorant.