I can roughly imagine a division for considering domains of knowledge and areas of work and practice, based on how proficiency or fluency with them is assessed (or not assessed).

Assessable fields are things like:

  • mathematics (early on)
  • track and field
  • biology
  • computer programming
  • physics (early on)
  • drawing 'craft' (drawing straight lines without aid, creating gradients, true-to-life portraiture)

Non-Assessable fields are things like:

  • literary criticism
  • figure skating
  • pop music creation
  • politics and policy-making
  • economics
  • sociology and psychology
  • drawing 'art' (choosing subjects, non-representational art)

Evaluation in assessable fields tends objective, and in non-assessable, subjective. In figure skating a panel of judges works from some rubric to evaluate a skater's skating with a numeric rating. Usually they won't all have the same evaluation. There may even be an appeals process. Contrast to a track race. A keen observer or a camera at the finish line determines race finish time.

A biology exam tests fact recall and a lab report tests successful replication of an experiment. An english class paper or a piece of criticism tests whether the writer can persuade a professor or grader (or other audience) that the writer is fluent with a more nebulously-defined skill like 'organizing information'

One rule of thumb that could help illustrate the distraction is whether something is evaluatable by a machine or not. Paul Graham famously relates hackers and painters because they both supposedly involve intuitive creative thinking. But anyone who's seen the error output of a python interpreter when his or her input was off by a letter knows that's not something that an art show does.

But this machine-evaluation criteria looks increasingly untenable with advances in computer image-recognition or other machine learning. One can imagine a machine learning model that evaluates figure skating video, replacing judges. Before the creation of photography, sometimes it would be difficult to call a close track race. Or a machine learning model may be able to evaluate the essay portion of a standardized college admissions exam.

In fact, automated essay scoring already exists, and with controversy. In a gross oversimplification of how machine learning works: the model is trained with an set of inputs and their classification. Input an essay that's been classified already by a human as A grade or F grade. Repeat tens, hundreds, or thousands of times. Now the model is trained. The problem is that human evaluation criteria may be quite inconsistent over the time period represented by these samples. And to some degree it's cementing the evaluations of the subjective evaluators. The machine didn't define the rubric of assessment, it just amplified it.

So is this assessable/non-assessable division the same as objective and non-objective? Not quite. Automated essay scoring can make essays assessable, but they're assessed by what the grading organization agree on. In contrast there's not much room for a difference of opinions on the products of a chemical reaction.

Another light on the divide is consideration of the stakes of what's being evaluated. Some of the fields that are assessable can be evaluated on whether they aid survival. But art or friendship can't be evaluated in this way. From C.S. Lewis:

Friendship is unnecessary, like philosophy, like art, like the universe itself (for God did not need to create). It has no survival value; rather it is one of those things which give value to survival.

With non-assessable fields, often the stakes are either lower, or higher depending on your point of view. Someone explains how a great painting saved his life

Actor and comedian Bill Murray claims this painting saves his life.

That is a bold claim, that may be true. And it is subjective. On the other hand you can often see more objectively when a doctor gives someone a necessary drug or suture. You can see faulty engineering in a bridge collapse, but is it easy to identify the collapse of a novel? And what happens if the novel collapses? In most cases, very little because few have read most novels (and that's getting to be more the case). But in a few cases, and from a certain point of view, a novel's collapse is something more diffuse and visible downstream in culture. If a lauded novel subverts a long-standing myth, there's a risk it throws away any wisdom contained in that myth.

There does seem to be a trend that the deeper a practitioner gets into a skill or domain, the more non-assessable it becomes. Early on in math education, simple computation can be evaluated. The practitioner (student) either wrote 2+2=4 and it was correct, or they wrote 2+2=1 and it was incorrect. But a mathematician writes elaborate, pages-long proofs that can't be read by an automated grading system. They're instead evaluated by a network of peer readers. However this may be subject to some change with automated theorem proving. And unlike automated essay scoring, there seems to be less controversy over automated theorem proving. And yet, there are still limitations to what is provable (if I understand right, this is one of the things demonstrated by Gödel among others).

Another distinction both orthogonal to and overlapping with the assessable/non-assessable division is among how readily subskills are acquired. With literature, if you can't write in the language it's gibberish and that's obvious and readily assessed. But once this is met, you're in more subjectively assessed territory. In fact, once enough people (assessors) believe you've mastered literature enough, they may still assess your skill as great even as you turn the formerly objective subskill of 'writing in the language' into something subjective. Famously, the novelist James Joyce's novel appears to the uninitiated as gibberish. To take another example, portraiture and representational art have some threshold of technical skill, where it's obvious to an observer if the artist can hardly wield a brush or charcoal stick. But beyond some threshold, portraiture is 'in the eye of the beholder'. And again, artists like Picasso turn what was formerly a technical subskill of portraiture fundamentals into more subjective art. You can see this in the gripe of the casual visitor to a modern art museum: "What's the big deal? My kid could paint that!".

I have an aversion to assessable fields, and I suspect many people do. It's sometimes easier to feel safe and comfortable in a modern or abstract art movement and convince yourself of the merits of your 3-tone rectangle paintings.

I actually quite like Mark Rothko paintings.

If a critic doesn't like it, you can tell yourself the critic doesn't understand. On the other hand, a collection of peers can't dispute that your solution to an equation is wrong for very long, but a film producer may insist your script is trash. So if you're not able to convince yourself that the producer doesn't understand your work, you're in for a big pride hit. There's a component of approval-seeking in non-assessable fields that may be less prevalent in assessable ones.

What can a person do who is interested in becoming competent in an assessable field, but hesitant because of perceived danger and ego exposure? For one, focus on the advantages: many an engineering student is happy they don't have to please an instructor. Or, expanding that, that they don't have to cater to an editorial board when designing a bridge. For another, seek supportive people, who view even assessable fields with an eye to human flourishing. Francis Su is a math professor who is so engrossed in his subject, he can't wait to share it with anyone and everyone-suggesting it is not only for a lone few naturals who can win competitions, but benefits all. Thirdly, one can view assessability as an opportunity for rapid feedback loops, and an aide to self-teaching. Ability to get rapid feedback is a necessary element identified for fast learning by the blogger Scott H. Young[1].

What of a demonstration of work invested over time to master a non-assessable skill, even if the skill is difficult to assess? This is the James Joyce example, or painter Piet Mondrian. Both developed a body of work that seem to show an evolution toward something different from before and more complex, though still not easily assessed. Perhaps it's appropriate to think of this as some third category entirely.


[1]Scott also applies his techniques to self-portraiture, but notable the technical elements. He assesses his portraits against a photo or his image in the mirror. Notably he does not give any mention of ideas for rapidly learning non-representational art.