blog

Meanings and belief states

Fri, 27 Jul 2018 00:00:00 +0000

Should AI research be concerned with explicit representations of the meanings of utterances? By “explicit representations of meaning” I mean structured variables with a pre-defined interpretation—the kind of thing that semanticists are concerned with. For a long time, such meaning representations were central to successful efforts aimed at linking language to other tasks involving reasoning, perception, and action (a line of work that runs from SHRDLU to modern semantic parsers). Structured meaning representations were also central to unsuccessful work on machine translation, syntax, etc. This work uses lots of different representation formalisms—proper neo-Davidsonian logical forms [AZ13], combinator logics [LJK11], other non-logical structures [TK+11]—but if you squint they’re all basically predicate-argument structures implementing a model-theoretic semantics, with perhaps a few free parameters in the bodies of the predicates.

These kinds of approaches seem to be disappearing. Now that everything is end-to-end all the time, it’s rare to see models with explicit latent variables taking values in hand-designed logical languages. Utterances come into our models, behaviors come out, and we don’t worry too much about the structure of the computation that gets performed in the middle.
There are still certain kinds of generalization and inductive bias we used to get for free with the old models that we haven’t totally figured out how to recreate. The success of hybrid approaches like structured regularizers [OS+17] and our NMN work [AR+16] suggest we’ll get there eventually. This is, with some qualifications, a good thing: in more formal approaches, tight coupling between the machine learning and the representation means that there’s always a risk that some new semantic phenomenon shows up in the data and suddenly your model is useless. Sufficiently generic machinery for learning (non-logical) representations makes this a little less scary.

But the attitude of the end-to-end world seems to be that since we’re no longer doing logical inference, there’s no need to think about meaning at all. Suddenly everyone loves to cite Wittgenstein to argue that we should be evaluating “language understanding” in terms of success on downstream tasks rather than predicting the right logical forms [WLM16] [GM16] [LPB16]—which is great!—but underlying this seems to be a philosophy that “meaning is use, so if we can predict use with high accuracy we’ve understood everything we need to about meaning”. I’ve never understood this to be the claim in Philosophical Investigations—even if use (rather than reference) is the primary thing we should be trying to explain, PI is very interested in the kinds of ~~mental representations~~ representations of the processes (?) in virtue of which language use is possible. Particularly given that we have not actually solved “use”, I think machine learning has both lots to learn and lots to say about the meaning side of the equation as well.

In this post I want to motivate the use of explicit representations of belief states of the form $p(\textsf{world state} \mid \textsf{utterance})$ as representations of meaning suitable for “unstructured” machine learning models. These kinds of representations arise naturally in the sorts of decision-making tasks the community is excited about about these days, but they also look a lot like classical representational theories in linguistics. The synthesis suggests ways of both training and interpreting language processing models.

Belief states and intensions

Consider the problem of trying to act in a partially observed world where people talk to you in order to help reduce your uncertainty. How should you choose the best possible action to take? Given a single utterance $w$ , and possible true states of the world $x$ , the min Bayes risk action is

$\arg\min_a \int_x p(x \mid w) \; R(a;\,x)$

for some risk function $R$ . Any listener who hopes to succeed in the world needs to do a good job of at least approximately solving this optimization problem, and in practice the listener will probably need to represent the distribution $p(x \mid w)$ , at least implicitly. In POMDP-land we call $p(x \mid w)$ a belief state; for a given $w$ , it’s a function that maps from possible worlds $x$ to scalar plausibility judgments—how likely is it that $x$ is the true world given that we observed someone saying $w$ about it?

Compare this to the notion of an intension in Montague semantics: “a function from possible worlds and moments of time to truth values” [J11]. Most (model-theoretic) semantics programs represent intensions using logical expressions (rather than e.g. tabularly). But a logical form is just one way of expressing a function of the right type; at the end of the day, an “explicit representation of meaning” to the Montagovian tradition is precisely an intension—that is, something that looks like a discretized version of our $p(x \mid w)$ .

A belief state is an intension with probabilities. Intensional representations of meaning are useful not just because they help us solve linguistic problems, but also because they approximate a quantity that we know helps language users do useful things with the information they’ve acquired from language. From the other side, what the POMDP tells us we have to compute upon hearing an utterance is approximately the thing linguists have been telling us to compute all along. Or almost the thing linguists have been telling us about—what would be even better than a black box for answering $p(x \mid w)$ queries would be something with a little structure, maybe some kind of factorized representation that let us find the MBR action efficiently by making it possible to inspect the set of properties that all plausible worlds have in common. Perhaps a product of assertions about individuals, their properties, and their relations…. If logical semantics didn’t exist we would have had to invent it.

$p(x \mid w)$ qua “meaning” should be precisely understood as a listener meaning: an accurate belief state already accounts for Gricean speaker-meaning-type effects (e.g. implicatures) and also further inferences the speaker may not have wanted the listener to draw (e.g. the possibility that $w$ is a lie). Our story here doesn’t care where $p(x \mid w)$ comes from, so it might be computed via something like RSA [FG12] with a distinct notion of sentence meaning embedded inside.

One last adjustment: real-world listeners don’t start with a tabula rasa: every utterance is interpreted in the context of an existing belief state $p(x)$ , and we really want to think of the meaning of a sentence as an update function; i.e. $p(x) \mapsto p(x \mid w)$ rather than just $p(x \mid w)$ . For sentences of the “Pat loves Lou” variety I think this update is basically always conjunctive; i.e. $p(x) \mapsto (1/Z) \cdot p(x) \cdot p(x \mid w)$ . But the general version is necessary for dealing with indexicals and Quine’s problems with the denotation of bachelor.

Practical implications

All very nice, but we led by noting that explicit denotational meaning representations (logical, probabilistic, or otherwise) don’t actually show up in the kinds of models that work well in practice. So why does any of this matter?

For language understanding systems to work well, they must be choosing something close to the min Bayes risk action. Hand-wavingly: a suffix of a deep network is a function from input representations to output actions via a fixed circuit; if this suffix can pick a good action for every input representation it’s implementing something like an MBR decoding algorithm (though perhaps approximate and specialized to the empirical distribution over representations); whatever representation of language-in-context is presented to this part of the network must then be sufficient to solve the optimization problem, so be something like a representation of $p(x \mid w)$ .

This is not a great argument: there may in fact be no clear distinction between the “sentence representation” and “optimization” parts of the model. But in practice we do see meaning-like sentence representations emerge (especially in models where the sentence representation is computed independent of whatever initial information the listener has about the state of the world [DP+18]). When using specialized optimization modules within larger networks [TW+17] [LFK18] we can be sure of the distinction.

In any case, the knowledge that some intermediate representation in our model is (or should be) decodable into a distribution over world states gives us two tools:

Interpretability: test whether representations are capturing the right semantics (or identify what weird irregularities they’re latching onto) by estimating $p(x \mid \textrm{rep}(w))$ , where $\textrm{rep}(w)$ is the model’s learned representation of the utterance $w$ . Determine whether this corresponds to the real (i.e. human listener’s) denotation of $w$ . We got a bunch of mileage out of this technique in our neuralese papers [ADK17] [AK17] and other students in the group have been using it recently to analyze pretraining schemes for instruction-following models. But in some ways it’s even more natural to apply it to the learning of representations of natural language itself rather than a learned space of messages / abstract actions.

Auxiliary objectives: the normal objective for an instruction following / QA problem is $p(\textsf{action} \mid \textsf{utterance}, \textsf{listener obs})$ . But if overfitting is an issue, it’s easy enough to tack on an extra term of the form $p(\textsf{speaker obs, listener obs} \mid \textsf{utterance})$ if it’s available. For some problems (e.g. GeoQuery-style semantic parsing) there isn’t a meaningful distinction between “speaker observation” and “action”; for others it looks like a totally different learning problem. For referring expression games the denotational auxiliary problem is “generate / retrieve pairs of images for which this would be a discriminative caption”; for instruction following models it’s “generate the goal state (but not necessarily the actions that get me there)”.

Conclusions

Thinking about POMDP-style solutions to language games results in an account of meaning that looks suspiciously like model-theoretic semantics. This analogy provides tools for interpreting learned models and suggests auxiliary objectives for improving their accuracy.

Fake language: two perspectives

Sat, 04 Nov 2017 00:00:00 +0000

Is there a role for fake language datasets in the AI ecosystem? (By “fake language” I mean things like bAbI, CLEVR, Karthik’s “Home” world [pdf link], and this DeepMind paper.) Claims that various learning architectures can “do language processing” based on results from these datasets have caused a lot of hand-wringing in the NLP community. While a lot of this is due to good old-fashioned overclaiming, I’ve become persuaded some of it is miscommunication between two groups that mean totally different things by “language data”.

I’m going to focus here on instruction following, but I think there’s a similar story to tell about lots of other grounded tasks like question answering, generation, etc.¹ For a long time, an AI researcher’s view of an instruction following problem was something like this:

Language -> Abstraction -> Behavior

That is, we start with whatever utterances people are generating, map them to some kind of clean, structured representation, and then make decisions about how to act on the basis of that structure. Because this whole pipeline was too hard to tackle all at once, the community mostly started working on it from different ends.²

“Language people” worked on this:

Language -> Abstraction

In this picture, language comes to you from the outside world—you don’t control the distribution. You get to design the language of abstractions, but it had better be able to handle (or at least fail gracefully on) whatever utterances the world throws at you. Linguists give us a nice abstraction formalism in the form of logic, and that the way to get from abstraction to behavior is just logical interpretation. So it’s very easy to say “abstraction = formal semantics” and treat the Abstraction -> Behavior edge as someone else’s problem.

Data is collected from human speakers who don’t necessarily know anything about logical forms. Indeed, decisions about details of the logical language are typically worked out after collecting initial annotations. What distinguishes “language data” from other kinds of data is precisely the fact that it was generated by human users. (If we’re generating data from a fake grammar and mapping it onto logical forms, we generally haven’t learned anything about language that we didn’t write down in the first place.)

“Policy people” (broadly understood to include everything from RL to planning to classical control) worked on this:

Abstraction -> Behavior

In this picture, the scope of acceptable abstractions is up to the system designer—it’s in behavior where details of the real world (physics, etc.) intervene. Abstraction languages range from “do one of these 10 specific things” to “satisfy this STRIPS goal”. In particular, an abstraction language that doesn’t support all possible goals is no more problematic than a remote control that doesn’t operate all appliances at once.³ The thing that I originally found difficult to appreciate is just how hard some of these problems are even when we have complete control over the input distribution. Reinforcement learning is hard. Planning is hard. There’s still lots of room for interesting compositionality in these abstraction languages—if I have some kind of structured representation of the goal, and I train on a subset of structures, do I generalize to the rest? There’s lots we still can’t do.

To come up with problems that are within reach of current methods, data is generated rather than collected. The distribution over abstractions and their induced behaviors is hand-designed. There’s no language data here; what distinguishes “language data” from what does get used is that language has no precise execution semantics, but e.g. STRIPS does.

In the last few years these two communities have run together, because the world looks like this:

Language -> Behavior

Everything is end-to-end all the time. Abstraction isn’t gone, but now it lives in some uninterpretable representation space rather than a formalism we’ve designed by hand. This is a great thing! Language people no longer have to limit themselves to worlds where they’re clever enough to construct a good enough logical language.

And policy people (here’s where the trouble starts) no longer have to describe their task inventory in terms of any particular formalism: they just need some way of generating reward functions / goal tests synchronously with some kind of (compositional?) identifier that describes them. So they generate interpretable strings made of sequences of words. No execution semantics, uses English words: natural language. Whence the confusion.

I think there’s a lesson in this for people in both communities:

For “policy people” as researchers: please please please signpost explicitly when your input strings were generated synthetically. The word language is hopelessly overloaded at this point, but the bigram natural language is not: avoid using the word natural unless people were involved. (A couple of the papers linked in the first paragraph are guilty of this.)
For “language people” as reviewers: respond to appropriately qualified fake language datasets by asking “Is an interesting Abstraction -> Behavior problem is being solved? Do strings index the target class of behaviors in an interesting way?” There are lots of problems out there for which this is an appropriate standard.

I think we’re still at a stage where there’s something to learn from fake language, even those of us who ultimately care about the distribution of sentences produced by humans.

Explicitly excluded from this discussion is work on doing linguistic analysis of fake language data. Such work is not about language at all, and is either about analyzing the formal expressive capacity of certain model classes, or is garbage. ↩
Not everyone! Both Stefanie Tellex and Branavan had work that was trying to tackle the complete pipeline well before the current end-to-end craze. I think this research is a model for what the field should be trying to accomplish, but for lots of problems our techniques just aren’t there yet. ↩
There’s a sense in which Jonathan Berant and Percy Liang’s work on paraphrasing for semantic parsing [pdf link] actually belongs in this category, rather than Language -> Abstraction. But they still hold themselves to a “real language” evaluation standard. ↩

Learning to reason with neural module networks

Wed, 17 May 2017 00:00:00 +0000

A neural network is a monference, not a model

Thu, 18 Feb 2016 00:00:00 +0000

The distinction between models and inference procedures is central to most introductory presentations of artificial intelligence. For example: HMMs are a class of model; the Viterbi algorithm is one associated inference procedure, the forward–backward algorithm is another, and particle filtering is a third.

Many people describe (and presumably think of) neural networks as a class of models. I want to argue that this view is misleading, and that it is more useful to think of neural networks as hopelessly entangled model–inference pairs. “Model–inference pair” is a mouthful, and there doesn’t seem to be good existing shorthand, so I will henceforth refer to such objects as “monferences”. My claim is that we should think of a neural network as an example of a monference. (An implementation of the Viterbi algorithm, equipped with the parameters of some fixed HMM, is also a monference.)

I’m about to cite a bunch of existing papers that follow naturally from the neural-nets-as-monferences perspective—it seems like this idea is already obvious to a lot of people. But I don’t think it’s been given a name or a systematic treatment, and I hope others will find what follows as useful (or at least as deconfounding) as I did.

What are the consequences of regarding a neural net as a model? A personal example is illustrative:

The first time I saw a recurrent neural network, I thought “this is an interesting model with a broken inference procedure”. A recurrent net looks like an HMM. An HMM has a discrete hidden state, and a recurrent net has a vector-valued hidden state. When we do inference in an HMM, we maintain a distribution over hidden states consistent with the output, but when we do inference in a recurrent net, we maintain only a single vector—a single hypothesis, and a greedy inference procedure. Surely things would be better if there were some way of modeling uncertainty? Why not treat RNN inference like Kalman filtering?

This complaint is wrong. Our goal in the remainder of this post is to explore why it’s wrong.

Put simply, there is no reason to regard the hidden state of a recurrent network as a single hypothesis. After all, a sufficiently large hidden vector can easily represent the whole table of probabilities we use in the forward algorithm—or even represent the state of a particle filter. The analogy “HMM hidden state = RNN hidden state” is bad; a better analogy is “HMM decoder state = RNN hidden state”.

Let’s look at this experimentally. (Code for this section can be found in the accompanying Jupyter notebook.)

If we think about the classical inference procedure with the same structure as a (uni-directional) recurrent neural network, it’s something like this: for $t = 0..n$ , receive an emission $x_t$ from the HMM, and immediately predict a hidden state $y_t$ . You should be able to convince yourself that if we’re evaluated on tagging accuracy, the min-risk monference (if HMM parameters are known) is to run the forward algorithm, and predict the tag with maximum marginal probability at each time $t$ .

I generated a random HMM, drew a bunch of sequences from it, and applied this min-risk classical procedure. I obtained the following “online tagging” accuracy:

62.8

Another totally acceptable (though somewhat more labor-intensive) way of producing a monference for this online tagging problem is to take the HMM, draw many more samples from it, and use the (observed, hidden) sequences as training data (x, y) for a vanilla RNN of the following form:

(where each arrow is an inner product followed by a ReLU or log-loss). In this case I obtained the following accuracy:

62.8

Is it just a coincidence that these scores are the same? Let’s look at some predictions:

true hidden sequence: 1 0 0 1 0
classical monference: 1 0 1 1 0
neural monference:    1 0 1 1 0

true hidden sequence: 1 0 1 2 0
classical monference: 1 0 1 0 0
neural monference:    1 0 1 0 0

So even when our two monferences are wrong, they’re wrong in the same way.

Of course, we know that we can get slightly better results for this problem by running the full forward-backward algorithm, and again making max-marginal predictions. This improved classical procedure gave an accuracy of:

63.3

better than either of the online models, as expected. Training a bidirectional recurrent net

on samples from the HMM gave a tagging accuracy of:

63.3

A sample prediction:

true hidden sequence: 0 1 0 0 1
classical monference: 0 1 0 0 1
neural monference:    0 1 0 0 1

Notice: the neural nets we’ve used here don’t encode anything about classical message-passing rules, and they definitely don’t encode anything about the generative process underlying an HMM. Yet in both cases, the neural net managed to achieve accuracy as good as (but no better than) the classical message passing procedure with the same structure. Indeed, this neural training procedure results in a piece of code that makes identical predictions to the forward–backward algorithm, but it doesn’t know anything about the forward–backward algorithm!

Neural networks are not magic—when our data is actually generated from an HMM, we can’t hope to beat an (information-theoretically optimal) classical monference with a neural one. But we can empirically do just as well. As we augment neural architectures to match the algorithmic structure of more powerful classical inference procedures, their performance improves. Bidirectional recurrent nets are better than forward-only ones; bidirectional networks with multiple layers between each “real” hidden vector might be even better for some tasks.

Better yet, we can perhaps worry less about harder cases, when we previously would have needed to hand-tune some approximate inference scheme. (One example: suppose our transition matrix is a huge permutation. It might be very expensive to do repeated multiplications for classical inference, and trying to take a low-rank approximation to the transition matrix will lose information. But a neural monference can potentially represent our model dynamics quite compactly.)

So far we’ve been looking at sequences, but analogues for more structured data exist as well. For tree-shaped problems, we can run something that looks like the inside algorithm over a fixed tree or the inside–outside algorithm over a whole sparsified parse chart. For arbitrary graphs, we can apply repeated “graph convolutions” that start to look a lot like belief propagation.

There’s a general principle here: anywhere you have an inference algorithm that maintains a distribution over discrete states, instead:

replace {chart cells, discrete distributions} with vectors
replace messages between cells with recurrent networks
unroll the “inference” procedure for a suitable number of iterations
train via backpropagation

The resulting monference has at least as much capacity as the corresponding classical procedure. To the extent that appproximation is necessary, we can (at least empirically) learn the right approximation end-to-end from the training data.

(The version of this that backpropagates through an approximate inference procedure, but doesn’t attempt to learn the inference function itself, has been around for a while.)

I think there’s at least one more constituency parsing paper to be written using all the pieces of this framework, and lots more for working with graph-structured data.

I’ve argued that the monference perspective is useful, but is it true? That is, is there a precise sense in which a neural net is really a monference, and not a model?

No. There’s a fundamental identifiability problem—we can’t really distinguish between “fancy model with trivial inference” and “mystery model with complicated inference”. Thus it also makes no sense to ask, given a trained neural network, which model it performs inference for. (On the other hand, networks trained via distillation seem like good candidates for “same model, different monference”.) And the networks-as-_models_ perspective shouldn’t be completely ignored: it’s resulted in a fruitful line of work that replaces log-linear potentials with neural networks inside CRFs. (Though one of the usual selling points of these methods is that “you get to keep your dynamic program”, which we’ve argued here is true of suitably organized recurrent networks as well.)

In spite of all this, as research focus in this corner of the machine learning community shifts towards planning, reasoning, and harder algorithmic problems, I think the neural-nets-as-monferences perspective should dominate.

More than that—when we look back on the “deep learning revolution” ten years from now, I think the real lesson will be the importance of end-to-end training of decoders and reasoning procedures, even in systems that barely look like neural networks at all. So when building learning systems, don’t ask: “what is the probabilistic relationship among my variables?”. Instead ask: “how do I approximate the inference function for my problem?”, and attempt to learn this approximation directly. To do this effectively, we can use everything we know about classical inference procedures. But we should also start thinking of inference as a first-class part of the learning problem.

Thanks to Matt Gormley (whose EMNLP talk got me thinking about these issues), and Robert Nishihara and Greg Durrett for feedback.

Also Jason Eisner for this gem: “An awful portmanteau, since monference should be a count noun like model, but you took the suffix from the mass noun. Not that I can claim that infedel is much better…”

Planning in representation space

Sun, 17 Jan 2016 00:00:00 +0000

Agents parameterized by neural nets (Atari players etc.) seem to universally suffer from an ability to plan. This is obvious in the case of Markov reflex agents like vanilla deep Q learners, and seems to be true even of agents with some amount of hidden state (like the MemN2N paper at NIPS). Nevertheless planning-like behaviors have been successfully applied to other deep models, most notably text generation—beam decoding, and even beam-aware training, seem to be essential for both MT and captioning. And of course, real planning is ubiquitous among people working on non-toy control problems.

Task and motion planning is a good example. At the end of the day, we need to solve a continuous control problem, but attempting to solve this directly (either with a generic control policy or TrajOpt-like procedure) is too hard. Instead we come up with some highly simplified, hand-specified encoding of the problem—perhaps a STRIPS representation that discards geometry. We solve the (comparatively easy) STRIPS planning problem, and then project back down into motion planning space. This projection might not correspond to a feasible policy! (But we want things feasible in task space to be feasible in motion space as much as possible.) We keep searching in planning space until we find a solution that also works in task space.

This is really just a coarse-to-fine pruning scheme—we want a cheap way to discard plans that are obviously infeasible, so we can devote all of our computational resources to cases that really require simulation.

We can represent this schematically:

Here we have a representation function $r$ , a true cost function $c$ (which we may want to think of as a 0–1 feasibility judgment), and a “representation cost” $k$ . We want to ensure that $r$ is “close to an isomorphism” from motion costs to task costs, in the sense that $c(s_1, s_2) \approx k(r(s_1), r(s_2))$ .

For the STRIPS version, we assume that $r$ and $k$ are given to us by hand. But can we learn a better representation than STRIPS for solving task and motion planning problems?

Learning from example plans

First suppose that we have training data in the form of successful sequences with motion-space waypoints $(s_1, s_2, \ldots, s^*)$ . Then we can directly minimize an objective of the form

$L(\theta) = \sum_i \left[c(s_i, s_{i+1}) - k_\theta(r_\theta(s_i), r_\theta(s_{i+1}))\right]^2$

for $r$ and $k$ parameterized by $\theta$ . Easiest if representation space (the codomain of $r$ ) is $\mathbb{R}^d$ ; then we can manipulate $d$ to control the tradeoff between representation quality and the cost of searching in representation space.

Problem: if we only ever observe constant $c$ (which might be the case if we only see good solutions), there’s no pressure to learn a nontrivial $k$ . So we also want examples of unsuccessful attempts.

Decoding

Given a trained model, we can solve new instances by:

Sample a cost-weighted path through representation space $(r_1, r_2, ..., r_n)$ such that $r(s^*) \approx r_n$ .
Map each representation space transition $r_1 \to r_2$ onto a motion space transition $s_1 \to s_2$ such that $r(s_2) \approx r_2$ . (Easily expressed as an opt problem if $r$ is differentiable, but harder as a policy.)
Repeat until one of the motion-space solutions is feasible.

At every step that involves computing a path (whether in $r$ -space or $s$ -space, we can use a wide range of possible techniques, whether optimization-based (TrajOpt), search-based (RRT, though probably not in high dimensions), or by learning a policy parameterized by the goal state.

Learning directly from task feedback

What if we don’t have good traces to learn from? Just interleave the above two steps—starting from random initialization, roll out to a sequence of predicted $r$ and $s$ , then treat this as supervision, and again update $k$ to reflect observed costs.

Informed search

So far we’re assuming we can just brute-force our way through representation space until we get close to the goal. There’s nothing to enforce that closeness in representation space corresponds to closeness in motion space (other than the possible smoothness of $r$ ). We might want to add an additional constraint that if $r_i$ is definitely three hops from $r_n$ , then $||r_i - r_n|| > ||r_{i+1} - r_n||$ or something similar. This immediately provides a useful heuristic for the search in representation space.

We can also use side-information at this stage—maybe advice provided in the form of language or a video demonstration. (Then we need to learn another mapping from advice space to representation space.)

Modularity

It’s common to define several different primitive operations in the STRIPS domain—e.g. just “move” and “grasp”. We might analogously want to give our agent access to a discrete inventory of different policies, with associated transition costs $k_1, k_2, \ldots$ . Now the search problem involves both (continuously) choosing a set of points, and (discretely) choosing cost functions / motion primitives for moving between them. The associated motions of each of these primitives might be confined to some (hand-picked) sub-manifold of configuration space (e.g. only move the end effector, only move the first joint).

Thanks to Dylan Hadfield-Menell for useful discussions about task and motion planning.

Programs made of neural networks

Sun, 06 Sep 2015 00:00:00 +0000

A crude history of applied machine learning:

Whenever we have a low-capacity model with hand-engineered structural constraints, and replace it with a high-capacity model with simple features and few structural constraints, model quality improves [models are smaller, take less time to develop, and generalize better to unseen data].

See (in NLP): linear models replacing decision lists, Jelinek’s infamous linguists, statistical machine translation, the recent flurry of papers whose entire substance is “replace this log-linear model (a two-layer neural net) with a three-layer neural net”.

A crude history of programming languages:

Whenever we have a programming language with lots of simple constructs, and we replace them with a few high-level constructs, program quality improves [programs of equivalent complexity are shorter, take less time to develop, and are less likely to contain bugs].

First everyone stopped writing assembly, then everyone stopped writing C.¹

Notice: in each case we’re moving the same slider—just moving it in different directions. Machine learning and programming language design ultimately have the same goal: making it as easy as possible for certain problem-solving machines (whether those are humans or optimization algorithms) to produce correct code according to some specification. Out in the real world, we don’t favor machine learning because it is inherently more pure or beautiful than writing code by hand—we use it because it’s effective. If somebody released a library today with a bunch of composable vision primitives, and suddenly Facebook to could solve all their image-labeling problems more effectively with interns rather than neural nets, then neural nets would be out the door tomorrow.

Actually, can we write this library now?

To be clear, I don’t mean something like OpenCV, where you take a bunch of pre-implemented models for particular tasks and then do whatever stitching-together you want in postprocessing. Instead, again, some notion of little vision primitives from which it would be possible to write a classifier as

    load(image) andThen 
      detectObjects andThen 
      orderBy(salience) andThen 
      head andThen 
      name

or a captioner as

    load(image) andThen 
      detectObjects andThen 
      describeAll

or a face detector as

    load(image) andThen 
      detectObjects andThen 
      filter(name(_) == Face) andThen 
      drawBoundaries

What do the functions detectObjects, describeAll, etc. look like? Current experience suggests that they should be neural nets, but neural nets of a very particular kind: rather than being trained to accomplish some particular task (like image captioning), they’re trained to be freely composable: describeAll promises to take anything “like a list of detections” (whether directly from detectObjects or subsequently filtered) and produce a string. Note in particular that the inputs and outputs to these functions are all real vectors. There is no way to structurally enforce that a thing “like a list of detections” actually has the desired semantics, and instead we rely entirely on the training procedure.

In current real-world implementations, there’s a notion of layers as modular, pre-specified units, but networks as monolithic models customized for specific tasks (and requiring end-to-end training). Once we move to modular networks, though, we can start to perform tasks for which no training data exists. For example, “write a caption about the people in this image”:

    load(image) andThen 
      detectObjects andThen 
      filter(name(_) == Face) 
      andThen describeAll

using the same primitives specified above.

Steps we are already taking in this direction: the fact that people use a prefix of an image classification network to initialize models for basically every other vision task; the fact that “attention” is suddenly being treated as a primitive in model descriptions even though it’s a complicated sequence of operations for combining multiple layers. Roger Grosse’s beautiful paper on grammars over matrix factorization models also kind of looks like this, and Christopher Olah has a discussion of the type-theoretic niceties of neural nets understood as collections of reusable modules (though to me this seems largely secondary to the practical question of what these types are).

To bring this back to the earlier programming language discussion, we observe that:

It’s hard for a person to write down a person-detector by hand, but easy for a neural net.
Given appropriate functional vision primitives, it’s easy for a person to write down a person describer. But training a neural net to do this from scratch requires a lot of examples of people descriptions to do this. (We might then say it’s “easy” for people but “hard” for a neural net.)

To take this yet a step further, we can note that there are lots of machine learning techniques that are more human-like than neural-net-like, in the sense that they do well with tiny data sets and a good pre-specified inventory of primitives (e.g. program induction, semantic parsing). If we really just care about minimal human intervention, we can figure out our vision primitives and then hand them off to a machine learning subsystem of an entirely different kind.

So let’s write this library! There are research questions here: First, what is the right set of functional primitives to give people (or models for program induction)? Next, can these shared representations actually be learned? How do we find parameter settings for these modules using the kinds of labeled data currently available?

Disclosure: I already have a model like this working on a bunch of simple question-answering tasks about images—I think it’s a really exciting proof-of-concept, and I’ll hopefully be able to show it off soon. But it’s not a comprehensive solution (esp. if we want to interface between vision / language / control applications), and I think there’s a really interesting systems problem here too.

Followup:

Obviously this is a gross overstatement, since lots of people do continue to write assembly and C. But I think it’s less controversial to say that fewer people write in low-level languages, and that it’s harder to do so correctly. ↩