How could I have better predicted this?

I think your problem is not that you couldn't predict someone's political views, but rather that you had far too much confidence in your predictive ability.

To fix this, you should generally have only weak expectations about what other people believe. Unless you know someone quite well, you shouldn't be "shocked" to hear them express any view with a small but significant base rate in the population.

I consider it likely that among those reading this, more of you grew up in families of intelligent, educated people than the national average. It is also likely that the number of readers who grew up in liberal and nonreligious families exceeds the norm. Not all, certainly. Quite possibly not even the majority. However, I think there must be many among you who understand the shock of leaving your family and community, perhaps to go to college, and slowly discovering that there are people who don't think like you.

Those of us who grew up in religious, working class families and were the first in our families to attend college also had the experience of learning that there are people who don't think like us. We just experienced it earlier in life. And then when we went off to college, we slowly discovered that there are people who do think like us.

My experience dictated that the conversation would start with "Isn't this a terrible thing?" and proceed to "Oil companies shouldn't be allowed to make a mess they can't clean up." or "Shouldn't we invest in clean energy?" However, though the conversation began as I expected, I was subsequently informed that the oil companies were fully capable of cleaning it up, and that the reason it had not been cleaned up already was a conspiracy on the part of President Obama.

This was particularly shocking to me because there were no warning signs. These were people who were all educated to a Master's Degree level. I had spoken to several on more innocuous topics, and they seemed both interesting and intelligent. ...

How could I have better predicted this?

Why were you shocked? Unless the degrees were in Petroleum Extraction Engineering, you have no reason to expect them to be any better informed on the relevant issues than the general population. (And incidentally, unless your own degrees are in those fields, you have no particular reason to be confident about your own opinions.) On this kind of issue, we all get our opinions and factoids from the media. Our choice of which media. Fox or MSNBC. And if it shocks you that some intelligent people choose to get their news fix from conservative sources, then you really have led a sheltered life. If the thing that shocked you is that the conservative media were saying bad things about the federal regulators, then maybe you ought to sample from right wing sources more often, if only to keep your finger on the national pulse.

I am not only confused, I am viscerally uncomfortable. How do we model for people whose cultural contexts and information delivering authorities are fundamentally different from our own, without lumping them into a faceless group?

By getting to know those people and by becoming familiar with those cultural contexts and information delivering authorities. Duh! It is really not difficult. Actually, if you analyze, you will probably discover that you are already expending considerable effort trying to insulate yourself from those people and those sources of information. Just stop expending all that effort building walls to maintain your accustomed comfort level. And then, after a short period of discomfort, you will find that your comfort range has become extended.

And you may discover something else by occasionally placing yourself in the 'silent minority' position. Which is that some of the people in your own select group of friends may secretly harbor unorthodox positions on some political issues, but keep their mouths shut to avoid trouble. There is no way the opinions you encountered upstate could have been "shocking" to you unless your usual circle practices a particularly powerful kind of censorship-by-exclusion.

I don't have direct answers to your questions, but the main point that I would make here is that such stated beliefs don't necessarily run very deep.

I would guess the people who you were teaching with probably are quite similar to you and that their stated beliefs which seem so foreign function as belief as attire, serving primarily to bind them together. Such beliefs tend to be compartmentalized and need not have a strong impact on their views about things overall.

There's an incident I heard about at the time: a Belgian (Danish? Dutch? I don't remember exactly) dredging company offered to lend dredging ships, got a "no thanks," and went to their local press to blame it on the Jones Act, which is protectionist legislation that requires only using American-made ships unless the President grants an exception. It got picked up by the conservative blogosphere, made it to conservative TV as a "look at how pro-union legislation hampers our emergency response and destroys our environment, and how Obama doesn't really care because otherwise he would have granted an exemption" and then got responded to by administration officials and at about this point I stopped paying attention.

So, there's a way to have factual support for that position, but it's obviously unclear whether or not they had that in mind.

How do we model for people whose cultural contexts and information delivering authorities are fundamentally different from our own, without lumping them into a faceless group?

A giant question mark. Don't try to extend your beliefs to cover as much as possible but as little as you can defend.

See also:

• Generalizing From One Example

Are you looking for a better answer than talking to them to explore what they believe and why they believe it?

Certainly there are better answers, but I don't know of a better way to reach those answers than through the exercise of doing precisely that, and the hope that over time my generic model of people broadens to include the communities that at first seem alien, confusing and discomfiting, so that they start to seem... well, human, I guess.

To some extent, that does involve modeling a faceless group... or, rather, a lot of different and more-or-less overlapping faceless groups. It also involves constructing those models out of my experiences with individuals, and grounding out my thinking about those models in terms of those individuals. There's nothing wrong with that as far as I can see.

I think elitism is valuable in that it helps prevent this error (amongst other reasons). I despise false humility from the highly intelligent in this pro egalitarian age.

There is no contradiction in believing that a prototypical human is smarter than most humans. Perhaps the variance in human intelligence is mostly explained by different degrees of divergence from the prototype due to developmental errors.

(I realize that this reveals a potential bias on my part regarding a correlation between intelligence and a liberal bent).

I wouldn't expect an intelligent conservative to posit this:

the reason it had not been cleaned up already was a conspiracy on the part of President Obama

So I don't see a bias if that idea shocked you (unless I share your bias). On the other hand, if you expected this:

"Oil companies shouldn't be allowed to make a mess they can't clean up." or "Shouldn't we invest in clean energy?"

simply because they were intelligent, then maybe there's a bias there.

As for the answer to the question that you asked, I agree with TheOtherDave and knb (not that I disagree with multifoliaterose either). Recognise that there is more variety of opinion, even among people who seem intelligent, than you previously realised, and the only way to find out what people believe is to talk to them.

Perhaps you're drawing the wrong moral. I don't know why it's generally important to predict disagreements, but I can think of other reasons the encounter could be, probably should be, profoundly disturbing.

If you take Robert Aumann's theorem seriously--which, put crudely, says you have no right to be cocksure of your position if your epistemic equals disagree--learning that some (perhaps numerous) intelligent, educated people have fundamentally different beliefs ought to induce (assuming you seek truth) serious doubt about your own views' accuracy. Maybe the reactionaries are correct! Or more seriously, you have acquired (assuming many similar others exist) strong evidence tending to show that you're as wrong as they.

I have met intelligent people who believe in all sorts of nonsense. 9/11 conspiracy, God, whatever. They usually have one or two beliefs of that kind and are very reasonable in the rest of their lifes.

Were you surprised because you literally hadn't expected that an educated intelligent person can believe in different facts, or because they all believed the same?

I consider it likely that among those reading this, more of you grew up in families of intelligent, educated people than the national average. It is also likely that the number of readers who grew up in liberal and nonreligious families exceeds the norm. Not all, certainly. Quite possibly not even the majority. However, I think there must be many among you who understand the shock of leaving your family and community, perhaps to go to college, and slowly discovering that there are people who don't think like you.

I think you are actually generalizing from one example about the tendency to generalize from one example here. :)

Or maybe it's just that I grew up in the South, but really, not until college?