A Gamification Of Education: a modest proposal based on the Universal Decimal Classification and RPG skill trees

While making the inventory of my personal library and applying the Universal Decimal System to its classification, I found myself discovering a systematized classification of fields of knowledge, nested and organized and intricate, many of which I didn't even know existed. I couldn't help but compare how information was therein classified, and how it was imparted to me in engineering school. I also thought about how, often, software engineers and computer scientists were mostly self-thought, with even college mostly consisting of "here's a problem: go forth and figure out a way to solve it". This made me wonder whether another way of certified and certifiable education couldn't be achieved, and a couple of ideas sort of came to me.

It's pretty nebulous in my mind so far, but the crux of the concept would be a modular structure of education, where the academic institution essentially established what information precisely you need from each module, and lets you get on with the activity of learning, with periodic exams that you can sign up for, which will certify your level and area of proficiency in each module.

A recommended tree of learning can be established, but it should be possible to not take every intermediate test, if passing the final test proves that you've passed all the others behind it (this would allow people coming from different academic systems to certify their knowledge quickly and easily, thus avoiding the classic "Doctor in Physics from Former Soviet Union, current Taxi Driver in New York" scenario).

Thus, a universal standard of how much you have proven to know about what topics can be established.

Employers would then be free to request profiles in the format of such a tree. It need not be a binary "you need to have done all these courses and only these courses to work for us", they could be free to write their utility function for this or that job however they would see fit, with whichever weights and restrictions they would need.

Students and other learners would be free to advance in whichever tree they required, depending on what kind of profile they want to end up with at what age or point in time. One would determine what to learn based on statistical studies of what elements are, by and large, most desired by employers of/predictors of professional success in a certain field you want to work in.

One would find, for example, that mastering the peculiar field of railway engineering is great to be a proficient railway engineer, but also that having studied, say, things involved with people skills (from rhetoric to psychology to management), correlates positively with success in that field.

Conversely, a painter may find that learning about statistics, market predictions, web design, or cognitive biases correlates with a more successful career (whether it be on terms of income, or in terms of copies sold, or of public exposure... each one may optimize their own learning according to their own criteria).

One might even be able to calculate whether such complimentary education is actually worth their time, and which of them are the most cost-efficient.

I would predict that such a system would help society overall optimize how many people know what skills, and facilitate the learning of new skills and the updating of old ones for everyone, thus reducing structural unemployment, and preventing pigeonholing and other forms of professional arthritis.

I would even dare to predict that, given the vague, statistical, cluster-ish nature of this system, people would be encouraged to learn quite a lot more, and on a quite wider range of fields, than they do now, when one must jump through a great many hoops and endure a great many constraints in space and time and coin to get access to some types of educations (and to the acknowledgement of their acquisition thereof).

Acquiring access to the actual sources of knowledge, a library (virtual or otherwise), lectures (virtual or otherwise), and so on, would be a private matter, up to the learner:

  • some of them already have the knowledge and just need to get it certified,
  • others can actually buy the books they want/need, especially if keeping them around as reference will be useful to them in the future,
  • others can subscribe to one or many libraries, of the on-site sort or by correspondence
  • others can buy access to pre-recorded lectures, peruse lectures that are available for free, or enroll in academic institutions whose ostensible purpose is to give lectures and/or otherwise guide students through learning, more or less closely
  • the same applies to finding study groups with whom you can work on a topic together: I can easily imagine dedicated social networks could be created for that purpose, helping people pair up with each other based on mutual distance, predicted personal affinity, mutual goals, backgrounds, and so on. Who knows what amazing research teams might be borne of the intellectual equivalent of OK!Cupid.

A thing that I would like very much about this system is that it would free up the strange conflicts of interest that hamper the function of traditional educational institutions.

When the ones who teach you are also the ones who grade you, the effort they invest in you can feel like a zero-sum game, especially if they are only allowed to let a percentage of you pass.

When the ones who teach you have priorities other than teach (usually research, but some teachers are also involved in administrative functions, or even private interests completely outside of the university's ivory tower1), this can and often does reduce the energy and dedication they can/will allocate to the actual function of teaching, as opposed to the others.

By separating these functions, and the contradictory incentives they provide, the organizations performing them are free to optimize for each: 

  • Testing is optimized for predicting current and future competence in a subject: the testers whose tests are the most reliable have more employers requiring their certificates, and thus more people requesting that they test them
  • Teaching is optimized for getting the knowledge through whatever the heck the students want, whether it be to succeed at the tests or to simply master the subject (I don't know much game theory, but I'd naively guess that the spontaneous equilibrium between the teaching and testing institutions would lead to both goals becoming identical).
  • Researching is optimized for research (researchers are not teachers. dang it, those are very different skill-sets!). However researchers and other experts get to have a pretty big say in what the tests test for and how, because their involvement makes the tests more trustworthy for employers, and because they, too, are employers.
  • And of course entire meta-institutions can spring from this, whose role is to statistically verify, over the long term,
    • how good a predictor of professional success in this or that field is passing the corresponding test, and
    • how good a predictor of passing the test is to be taught by this or that teaching institution.
    • how good a predictor of the test being reliable is the input of these or those researchers and experts
  • It occurs to me now that, if one wished to be really nitpicky about who watches the watchmen, I suspect that there would be institutions testing the reliability of those meta-institutions, and so on and so forth... When does it stop? How to avoid vested interests and little cheats and manipulations pulling an academic equivalent of the AAA certification of sub-prime junk debt in 2008?

Another discrepancy I'd like to see solved is the difference between the official time it is supposed to take to obtain this or that degree, to learn this or that subject, and the actual statistical distribution of that time. Nowadays, a degree that's supposed to take you five years ends up taking up eight or ten years of your life. You find yourself having to go through the most difficult subjects again and again, because they are explained in an extremely rushed way, the materials crammed into a pre-formatted time. Other subjects are so exceedingly easy and thinly-spread that you find that going to class is a waste of time, and that you're better off preparing for it one week before finals. Now, after having written all of the above, my mind is quite spent, and I don't feel capable of either anticipating the effect of my proposed idea on this particular, nor of offering any solutions. Nevertheless, I wish to draw attention to this, so I'm leaving this paragraph in until I can amend it to something more useful/promising.

I hereby submit this idea to the LW community for screening and sound-boarding. I apologize in advance for your time, just in case this idea appears to be flawed enough to be unsalvageable. If you deem the concept good but flawed, we could perhaps work on ironing those kinks together. If, afterwards, this seems to you like a good enough idea to implement, know that good proposals are a dime a dozen; if there is any interest in seeing something like this happen, we can need to move on to proprely understanding the current state of secondary/superior/higher education, and figuring out of what incentives/powers/leverages are needed to actually get it implemented.

 



 

1By ivory tower I simply mean the protected environment where professors teach, researchers research, and students study, with multiple buffers between it and the ebb and flow of political, economical, and social turmoil. No value judgement is intended.

 


 

EDIT: And now I look upon the title of this article and realize that, though I had comparisons to games in mind, I never got around to writing them down. My inspirations here were mostly Civilization's Research Trees, RPG Skill Scores and Perks, and, in particular, Skyrim's skills and perks tree.

Basically, your level at whatever skill improves by studying and by practising it rather than merely by levelling up, and, when you need to perform a task that's outside your profile, you can go and learn it without having to commit to a class. Knowing the right combination of skills at the right level lets you unlock perks or access previously-unavailable skills and applications. What I like the most about it is that there's a lot of freedom to learn what you want and be who you want to be according to your own tastes and wishes, but, overall, it sounds sensible and is relatively well-balanced. And of course there's the fact that it allows you to keep a careful tally of how good you are at what things, and the sense of accomplishment is so motivating and encouraging!

Speaking of which, several netwroks and consoles' Achievement systems also strike me as motivators for keeping track of what one has achieved so far, to look back and be able to say "I've come a long way" (in an effect similar to that of gratitude journals), and also to accomplish a task and have this immediate and universal acknowledgement that you did it dammit (and, for those who care about that kind of thing, the chance to rub it the face of those who haven't).

I would think our educational systems could benefit from this kind of modularity and from this ability to keep track of things in a systematic way. What do you guys think?

Comments

sorted by
magical algorithm
Highlighting new comments since Today at 7:01 PM
Select new highlight date
All comments loaded

Testing is optimized for predicting current and future competence in a subject:

What if this finds that the best predictors for most employers consists of two tests (1) a <10 minute IQ test, and (2) a test showing a student's willingness to submit to authority and learn what he was told, in the manner under which he was told without getting too far ahead or behind his classmates?

I am willing to bet money (up to €100, I am a student) that these will not, in fact, be the best predictors, just as grades in college have been proven not to be good predictors of how well a Google employee does, IQ tests have been proven not to be a good predictor of how well a salesman does, and higher salaries have proven to, past a certain point, be detrimental to productivity for employees whose jobs require creativity, imagination or otherwise intellectual exertion.

My hypothesis is that a <10 min IQ test will be so inaccurate as to be useless, and that the current school system already favors people who do what they are told as they are told without getting too far ahead or behind, with the destructive and wasteful results we all know: they are the ones it grades best, and yet, time and again, tested against the workplace, those grades have proven little.

IQ test scores are massively correlated with workplace performance.

I find it plausible that among the set of people Google has hired grades don't predict workplace success, but I bet if Google were to randomly select employees from among the U.S. population (or even from just among computer science majors) grades would be hugely correlated with performance.

I bet if

How much are you betting?

I can cite my studies, if you want. It may take me some time to find them (an afternoon or two). Can you cite yours?

What are the terms of the bet? I would be willing to bet €100 depending on the terms. Full disclosure: I read a lot of academic research on IQ to help me write this book, one of whose chapters is about IQ.

There is a lot of silly stuff written about IQ. No offense, but do you think you have the needed background to separate the high from the low quality IQ scholarship?

I only read Talent is Overrated, which cited a few studies to the effect that IQ was a bad predictor of professional performance beyond the short term; to become a world-class performer at some specific skill, you need to spend 20000 hours of your life deliberately practicing it (trying things outside your comfort zone, failing to get it right, figuring out what to improve and how, getting it right, trying something harder, rinse and repeat, and it's a pain in the neck). Most people, no matter how smart, tend to stop improving at their jobs after around one year and a half of practice: this includes highly skilled professionals like doctors and lawyers.

And, on the anecdotal side, in my own experience and what I could get from the people that frequent Lesswrong and TVTropes, a high IQ commonly results in huge childhood issues, a miserable social life, and a boatpload of akrasia, which seriously hamper whatever base utility it might have. Perhaps there's a selection bias and only the incompetent brains have the time to hang out here, while competent smart people are too busy actually getting things done?

in my own experience and what I could get from the people that frequent Lesswrong and TVTropes, a high IQ commonly results in [...] a miserable social life, and a boatpload of akrasia [...] Perhaps there's a selection bias and only the incompetent brains have the time to hang out here[?]

There is massive selection bias going on here.

I don't think it has as much to do with free time as with target audience, though. LW attracts a few different clusters of people, but the ones you're seeing in this context are those who feel their thinking is flawed in some way, and who believe they have a decent chance of fixing it with a cognitive science toolset and vocabulary. The site's native idiom and interaction style -- basically a founder effect -- imposes a few more filters, underrepresenting some problems and overrepresenting others. Akrasia and social problems are precisely the issues I'd expect to see a lot of, given those constraints.

TV Tropes... well, that might have more to do with free time. Almost everyone likes media, but if you want to make many original contributions, you need unusual knowledge of media and an analytical attitude towards it. Moreover, the media best represented there tend to be the most time-consuming ones -- TV, anime, doorstopper fantasy novels. I don't think I need to go into too much detail regarding the people most likely to share those requirements.

Offtopic post, but a discussion I wish to pursue nonetheless:

Regarding TVT: it used to be so. Nowadays school study media and the "literary canon" are beginning to find their way in... and all those tropes with silly names, built from mass media, are proving their usefulness as tools of analysis. Of course, getting a movie adaptation or a TV miniseries is one of the best ways to draw troper attention to a work, but classics always end up getting those with some regularity. So let's just say that the user base has widened. Oh, and many classics are as doorstoppery as modern fantasy sagas, especially stuff from the XIXth century, when novels where published as long-running serials in magazines and authors were paid by the word. When people call Eliezer Yudkowsky a terrible writer because of MoR's lack of tightness or his using it as a vehicle for ideas and lectures, I feel half-tempted to point at the likes of Victor Hugo or Alexandre Dumas or Dickens or Benito Pérez Galdós, just to name a few... surely if those are the traits of terrible writing, it means that those books have room for improvement, if only by way of abridging them?

Regarding LW: The filters the site imposes on its demographics (language and mode of interaction) worry me: what's the point of translating MoR to Spanish or French, if afterwards hispanophones and francophones don't get to read their sequences and have a site to be a community in? I'm trying to seed a rationalist community at my school, and the language barrier is proving to be troublesome; I often find myself not even knowing how a bias would be called in Spanish.

what's the point of translating MoR to Spanish or French, if afterwards hispanophones and francophones don't get to read their sequences and have a site to be a community in?

There have been a few Sequences translation projects discussed, proposed, or started, in various languages, plus a few more for MoR. Of these, only one (diegocaleiro's, in Portugese) seems aimed at a persistent community hub. I might have missed some, though.

With regard to the specific issue of bias names, if you don't have access to cognitive science faculty, I think a good place to start might be finding a translation of Daniel Kahneman's Thinking, Fast and Slow; Kahneman is a major source for the Sequences, his book's been widely translated, and there should be a lot of overlap. After that, you might want to try digging into XiXiDu's resource list. Though there are a few original LW coinages floating around, and I'm afraid you're on your own with those.

That is junk science.

The correlation between IQ and workplace performance is extremely robust and very well established.

See Intelligence: Knowns and Unknowns

Well thank goodness I'm on a holiday and can actually afford to read all that. Nevertheless, such a violent flipflop of opinions would make me dangerously closer to epistemic learned helplessness, i.e. giving up on learning new stuff because of not feeling confident that I can tell good arguments from bad ones when they come from experts after getting burned one time too many.

That is a dangerous state for a rationalist to be in. so would you please be a dear and have a look at that book to figure out how precisely it junk-scienced me and the rest of its readers? It would really help me out and I'd really be grateful for that.

Also, I have a vested interest in believing in the "only deliberate practice matters" thesis. I used to feel having a high IQ obliged me to get excellent results with little effort, and every time that didn't happen I felt disappointed in myself and that I didn't live up to some weird standard. Nowadays I only believe in working as hard as possible for as long as possible, and it serves me much better.

That is a dangerous state for a rationalist to be in. so would you please be a dear and have a look at that book to figure out how precisely it junk-scienced me and the rest of its readers? It would really help me out and I'd really be grateful for that.

I'm afraid I haven't read that exact popularization, but if it's drawing on Ericsson's research as it sounds like, the explanation is easy enough: Ericsson's points are valid largely because the studies are correlational, do not control for underlying factors or Matthew effects, and suffer from heavy range restriction in he's already looking at people who are selected or self-selected to be elites.

(ie. suppose someone studied MIT physicists with a mean IQ of 150 and discovered that in this group of physicists, Conscientiousness predicted better than IQ which would go on to win Nobels. This is a possible result, and what this has actually demonstrated is "you have to be incredibly brainy to be a MIT physicist in the first place, but once you've gotten that, then other things are also important; which is another way of saying that if we look at the general population, like all the people from IQ 60 to 150, IQ is the overwhelming most important trait" but this is easily popularized to "IQ doesn't matter!")

Also, I have a vested interest in believing in the "only deliberate practice matters" thesis. I used to feel having a high IQ obliged me to get excellent results with little effort, and every time that didn't happen I felt disappointed in myself and that I didn't live up to some weird standard. Nowadays I only believe in working as hard as possible for as long as possible, and it serves me much better.

As a practical matter, it's probably a good idea to believe hard work matters and deliberate practice matters. If you could somehow improve your intelligence, then it might be important to hold correct beliefs about IQ being far more important than practice; but unfortunately, IQ is pretty much fixed and all that's left is to make the best use possible. Given people matched on IQ and other traits, and Conscientiousness will be pretty important.

However, in other contexts, it's very important to hold the correct beliefs about the relative value of intelligence and 'just work harder' - if we were discussing iodization or immigration or whether someone should go into debt for college, for example. Many population-level questions will rest far more on intelligence than other traits.

Understanding which context we're in can be a hard balancing act, and especially difficult when reading papers making statistical claims (did they control for IQ? Should they control? Or for education? Or for range restriction? Or for reliability of their metrics?); I try to be consistent and clear in any discussions of Conscientiousness or IQ which context we're in and which we should value or ignore, but I don't think I always succeed.

Well thank you very much!

.. sigh This may be the first time I find some actual use for compartmentalization, and I can't do it anymore...

Nowadays I only believe in working as hard as possible for as long as possible, and it serves me much better.

Imagine a world where 50% of your results are genetically determined and 50% of your results are hard work. What would be the best strategy for success in that world, assuming that you already have decent genes? It would be working hard. Not working 50% hard, but working 100% hard.

Seems like you found the right strategy for the wrong reasons. You can keep the strategy; you don't have to blindly reverse your decisions.

I dunno, at a certain point the marginal utility of one unit of hard work will be less than the marginal utility of one unit of leisure, and it's well possible that the point at which that happens depends on how genetically good you are.

See Hambrick et al, Deliberate practice: Is that all it takes to become an expert?, for a recent critical assessment of the claim that deliberate practice is sufficient for attaining expertise. Gwern provides some quotes.

Even if (1) was possible (which I kind-of doubt), it'd be quickly Campbelled into uselessness if employers started to use it.

It's very useful and, in the U.S., illegal. Employers, I suspect, often favor applicants with degrees from prestigious colleges as a way to get around the direct ban on the use of IQ tests.

in the U.S., illegal.

Given recent Supreme Court decisions, I'm not sure how much longer this will be a true statement of US law.

The most recent case held that fear of disparate impact lawsuits was not a justification for a disparate treatment (throwing out a promotion test because no minority passed). In a concurrence (agree on result, not reasoning), one justice noted that disparate impact treatment law for employment is in considerable tension with the constitutional doctrine on equal protection - ConLaw equal protection does not have a disparate treatment component, while you correctly note that private employment discrimination law does.

There are two possible resolutions of this legal tension:
a) Employers are screwed - any choice subjects them to either (1) a valid disparate impact claim or (2) a valid disparate treatment claim.
b) Disparate impact claims will be tightened substantially - to the point that one would likely need to prove a facially neutral test was selected primarily for its disparate impact. This is functionally equivalent to lowering the legally required evidence to show disparate treatment.

My read of the legal landscape is that (b) is massively more likely than (a). (> .99). The resolution of that tension might even make private disparate impact claims practically impossible to prove. Or disparate impact liability might even be ruled unconstitutional (in the absence of disparate treatment evidence).

Although IQ tests were initially outlawed by the courts on disparate impact grounds, I believe that Congress enacted these IQ test restrictions directly into civil rights laws so they can't be overturned by courts except on constitutional grounds, which seem unlikely.

The test makers in the Ricci v. DeStefano case you cite went out of their way to have their tests not be just IQ tests.

Who knows what amazing research teams might be borne of the intellectual equivalent of OK!Cupid.

This sounds like an awesome idea. Does anyone know if such a thing already exists? If not, I would be willing to commit $500 to someone who wants to make it (assuming they already have the appropriate skill set, of course).

I'm reminded of a couple of students at a German university last year who studied all the material by dividing up the classes between them and exchanging notes, took all the exams, and passed in a few months. The university then turned around and sued them for studying too fast.

http://www.thelocal.de/education/20120703-43517.html

It is not in a University's interests to do a good job. It's like any other company: The aim is to extract the most $$$ from you while they give up the least value in return.

In highly competitive markets this leads to fairly marginal profits. But formal education is not a competitive market. Not at all. Very tightly regulated.

The competition for education comes almost entirely around costs for most people. They view education as an expense, not as an investment - which is quite reasonable when you consider the likely quality they're going to get - and aim to minimise that expense. Offer people a better education and the majority of them won't be willing to pay much for it, offer them a cheaper education though, or a faster one - which amounts to more or less the same thing....

That's the financial side of things anyway, and one of the reasons I think your idea's just never going to happen.

#

Assorted

One would determine what to learn based on statistical studies of what elements are, by and large, most desired by employers of/predictors of professional success in a certain field you want to work in.

  1. I don't know, but I wouldn't think this data's going to be available or reliable. Google did their own research on this, which suggests to me that the data wasn't available when they looked for it elsewhere. One thing's just that everyone's going to record things slightly differently - there's not an industry standard for measuring this sort of stuff that I'm aware of. The other is that most places are not particularly rational about their HR procedures - there are a set of skills that go into even the basic recording of data that I'd not expect them to have. I'd expect it to be more like they sit there with a sheet of paper that they're marking 1-10 based on their subjective opinion of your answer to a question. That's probably the extent of any data most people will be expected to generate and, perhaps, keep.

In the case of desired by employers I suspect you might find the data was actually contradictory to predictors of success a lot of the time too. I know, when I go for interviews, I tone down a lot of the qualities that let me do things easily - the corollary of don't dress better than the boss is don't act smarter/more skilled than the boss - most people like to hire people they can use as tools for their own success, not colleagues who they have to work with. Especially considering lots of economic behaviour seems to be self-justifying/rent-seeking.

================

That said, I would imagine that what you really want to do is to follow up with people who've left university, both those who won and those who lost, and ask them what they wish they'd known then that they know now. Certainly, a few years out of the gate, I could give my university a great deal of feedback.

================

2 . The incentives in HR seem to be to be risk averse. To find reasons not to take people on. If you hire someone and they do really well then you don't get rewarded, whereas if you hire someone and they're awful then your job is at risk. If that's heavily weighted in the decision maker's consideration, then giving people more granular data will result in dramatically fewer hires, since the probability of a concurrent set of criteria being fulfilled is the multiple of its individual probabilities. Even if the individual probabilities are quite high, HR managers will be able to talk their way into using a great number of individual criteria and then you'll be hung.

3 . It makes the market more competitive, not necessarily in a good way. Students take on a great deal of the financial risk in education these days. Which honestly seems the wrong way around, but there you go. If rich people can afford to grind their way higher up the tech tree, then you're essentially pricing poor people out of the market. The rich person only has to be ahead by one or two points - you're probably going to pick someone who's even slightly better on paper over someone worse. So there's an incentive to have large steps in your pricing.

I'm reminded of a couple of students at a German university last year who studied all the material by dividing up the classes between them and exchanging notes, took all the exams, and passed in a few months. The university then turned around and sued them for studying too fast.

That in turn reminds me of Nick Bostrom:

As an undergraduate, I studied many subjects in parallel, and I gather that my performance set a national record. I was once expelled for studying too much, after the head of UmeƄ University psychology department discovered that I was concurrently following several other full-time programs of study (physics, philosophy, and mathematical logic), which he believed to be psychologically impossible.

there's not an industry standard for measuring this sort of stuff that I'm aware of

Thank you for bringing this up. One of the huge deals of the Industrial Revolution, together with the American System of Manufacturing (basically, building in spare parts) and the Patent System, was the extensive development of Standards: originally, everyone agrees to do things the same way because it makes it easier to divvy up tasks between competing msnufacturers, and makes things cheaper.

Then there were safety standards and regulations, which, in practice, are often about who to blame if anything goes wrong; like taxes, there's a strong sense of "it ain't cheating unless you're caught".

And then there's this more recent movement for getting ISO certifications in the workplace and in universities. I haven't been been in the front-lines for that, but it would appear that it's being kind of a huge deal: standardized organization, standardized customer service, and so on and so forth. The buzzword is !quality".

To sum it up, the standardization of stuff is a work in progress, and people do realize the value in it.

tone down a lot of the qualities that let me do things easily - the corollary of don't dress better than the boss is don't act smarter/more skilled than the boss - most people like to hire people they can use as tools for their own success, not colleagues who they have to work with.

And that's terrible! Do we have a name for that bullgum, or shall we just call it an instance of the Tragedy of Commons? Aren't there studies on how this attitude is wrong for a company's future and how to disincentive it from within the company? Or maybe pay a third party to do the hiring?

If you hire someone and they do really well then you don't get rewarded.

?! Why is-- Huh? I'm sorry, I'm dumbfounded by how freaking stupid this is.

giving people more granular data will result in dramatically fewer hires

That doesn't sound economically sustainable for me: the law of conservation of work force (which I just made up) dictates that, in a steady state of work output and productivity, what comes out must come in.The number of hires needed is the number of hires needed. Unless we're in an economic crunch and companies are explicitly downsizing, adding hoops to jump through is an expensive zero-sum game between HR people and departments, not something prospective employees should be worried about.

If rich people can afford to grind their way higher up the tech tree, then you're essentially pricing poor people out of the market.

Isn't that already so, though? Only the well-off, and those so brilliant that they get scholarships (and how many full scholarships are there still?) can bear to go through the excruciatingly long, overly detailed and wide studies we have nowadays. I suspect that things wouldn't change much from today: middle classes and rich immigrants would invest in education because they can't live off capital alone, lower classes usually won't bother taking the risk and will stay where they are, and upper classes won't bother making the effort, or will just study whatever they please at their own leisure.

The main issue in implementing such a system is testing and certification. If you use quizzes then there is nothing to stop the student from looking up the answers in another window. The system can still be made but the tests need to be designed so that students can't just look up the answers. The solution is applied knowledge problems.

For example instead of asking someone about the physical properties of a metal you would ask them what metal they would use for a certain construction and why. Student programmers would be required to write short scripts. Student geographers would interpret maps. Student lawyers could say how a client should behave is different situations.

Basically you would test whether people can use their knowledge instead of whether they can communicate it to avoid students second-hand communication. The downside is that questions are harder to make.

The uspide is that it tests for the actual skills (expertise at) rather than the ability to talk about them (expertise on), which is something that should only be required of communicators such as teachers/professors, journalists, lawmakers, and so on.

I'm starting a job in an "adaptive learning" startup soon, and many of the points you make here remind me of the things this company does or plans to do. The basic idea of the company is that it collects data about the student as he or she interacts with an electronic course, then uses this to personalize the course and make recommendations for the student's education path. This isn't quite the same as what you're suggesting, where a student independently finds educational content and then gets certified in those areas. However, there are several similarities.

The tree of learning definitely exists. In your idea, the students find their own paths through this tree, but the adaptive learning way is to recommend paths to each student based on their performance in previous courses.

Both ideas allow each student to have some type of profile that showcases exactly what knowledge they have obtained. Adaptive learning could also add to this profile things that are specific to how the student learns (e.g. is it better to introduce this person this math concept by offering a rigorous proof or by explaining several revealing examples). Something like that could be helpful for the "intellectual OK!Cupid."

You envision a scheme wherein the student acquires access to real or virtual libraries of educational content, with varying degrees of completely independent learning and institutional guided learning. Right now, the company where I will work partners with education content companies (which seem to be mostly textbook publishers), and is only able to offer adaptive learning to students learning through the specific classes created around these textbooks. However, I believe there are plans to make the platform more open, so any course could be built with it.

The main disadvantage of adaptive learning, which your system does better with, is the problem where the teaching and grading is mostly done by the same entity. This seems to be a necessary aspect of adaptive learning, because the data collected about the student as they are taking a test, quiz, or exam is used to learn more about how the student thinks and learns. Still, there is the possibility that an adaptive course can use a test or quiz as a personalized teaching tool, and not only as a way to evaluate performance.

TL;DR- the difficulty of solving this problem is that availability of good data, NOT the lack of decent skill-set models.

So lets say you finish an outline of skills in these trees, as you suggest. Now, we want to make the statistical models that are at the core of your proposal- "One would determine what to learn based on statistical studies of what elements are, by and large, most desired by employers of/predictors of professional success in a certain field you want to work in." Where exactly do you plan to get the data to actually do this? As you "level up" down your skill tree, the number of people who have skill combinations is going to get very, very thin. I might be the only person in the world who has both calculated a Next-to-Next-to-Leading-Order cross-section at Tevatron energy levels AND who has used restricted-Boltzmann-machines to categorize medical billing codes to triage health insurance claims. Backing up a bit, and "quantum field theory" + "health care informatics" + "statistics" might have what, a dozen people, all of whom are going to be outlier-ish for the general field of insurance data analysis.

Lets look at data from physics (because I know it well) here is the AIP's focus on physics bachelor holders. Bachelors outnumber phds by quite a bit, so this is the densest you are going to get for certain types of physics skills. http://www.aip.org/statistics/trends/reports/empinibs0910.pdf. If you subtract out the exclusions (unemployed, people who went back to their pre-college job, and part-time employed), the data is already cut down to 2/3. Of those, the two largest chunks are highschool teachers (about 6% of the total physics bachelors) and engineers (about 7% of the total physics bachelors).

So we have about 350 people who teach highschool, probably highschool science, so you can maybe compare physics-educated general science teachers to other educated general science teachers. A bit more then that are in "engineering" fields, but there are tons of subfields/'skill-trees' in engineering. You'll have <100 per skill-tree at the trees base! The data is already very thin, and its much thinner for all other career paths.

Career organizations that have much larger memberships (i.e. most career organizations) have much less detailed information because collecting it becomes more of a chore. These larger datasets probably won't have the information you'd really need to fill out your skill-trees.

Also, I'm willing to bet that the world moves fast enough in a number of fields that you can't use very deep historical data to do predictions. i.e. two decades ago a physics phds was a decent way to get a quant job on wallstreet, BUT two decades ago a masters of financial engineering degree didn't exist. Maybe in another decade the MFEs squeeze out the phds the way that CS squeezed out physicist programmers in the 70s and 80s.

How about starting in a supportive domain that you happen to be interested in where existing taxonomies and skill progressions seem to exist?

Tech/skill trees help with the visualization process, and that goes a long way to motivation. Much of my struggle battling akrasia and learning math & beginner rationality is to try to stop imagining forbidding mountains of concepts, nomenclature and future practice, and instead focus on images of cartoonish/animated tech trees that are more appealing, tractable and familiar (for those of us with gaming backgrounds); ultimately, anything less intimidating than an impassible mountain to climb.

Added bonus: You will probably not find a more willing group of participants to enthusiastically input, rate and measure themselves on their cognitive and rationalist skills.

Added bonus 2: I speculate that CFAR may be able to integrate this type of system as additional value-add for post-workshop alumni.

edit: "Data Scientist" seems to be a buzz word used to trigger competence and desirability in job candidates. Here's an example of a visual roadmap: http://nirvacana.com/thoughts/becoming-a-data-scientist/. I think other people are willing to create skill trees, but the missing ingredient is the ability to animate and infuse them with meaning over time (i.e. referring back to your ideas of tests for each node/module with clear achievement and progress indicators).