NLP · 9 compared
NLP tasks
Nine tasks arranged by the one thing that separates them — how large the space of acceptable answers is. Read the “Scored by” column first.
Comparison
| Name What the task is | Takes | Returns | Scored by | Breaks when |
|---|---|---|---|---|
| Text classification One text in, one label out. Sentiment, topic, spam, intent. The oldest and still the most deployed shape in NLP, and the only one whose metric nobody argues about. Full page → | one text | a label | accuracy, F1 | classes are rare or shift |
| Natural language inference Two texts in, one label out — does the first entail the second, contradict it, or neither. The standard probe for whether a model handles relations between sentences rather than topics. Full page → | a pair of texts | a label | accuracy | the hypothesis alone predicts it |
| Sequence tagging A label per token — named entities, parts of speech, chunk boundaries. The output is still discrete, but it is now structured, so a single wrong token can invalidate a whole span. Full page → | one text | a label per token | span-level F1 | the tokenizer splits an entity |
| Extractive question answering A question and a passage in, a span of that passage out. The answer is constrained to text that already exists, which is exactly what makes the task scorable and what limits it. Full page → | question + passage | a span | exact match, token F1 | the answer is not in the passage |
| Semantic search A query in, a ranked list out. Precompute a vector per document, compare with cosine, and the collection can be arbitrarily large because nothing is quadratic. Full page → | a query | a ranking | recall@k, MRR, nDCG | relevance is unlabelled |
| Clustering and deduplication No query at all — just the geometry. Group or de-duplicate a collection by distance between embeddings. The task a cross-encoder structurally cannot do. Full page → | a collection | a partition | against a labelling, or not at all | the number of groups is unknown |
| Machine translation A sequence in, a different-language sequence out, with no alignment between them. The task that produced the encoder-decoder, attention, and the first honest arguments about evaluating generated text. Full page → | one text | new text | BLEU, chrF, human | one reference is not enough |
| Summarisation A long text in, a short one out. Extractive summarisation picks sentences and stays scorable; abstractive summarisation writes new ones and does not. Full page → | one long text | new text | ROUGE, human | the summary is right and unlike the reference |
| Open-ended generation A prompt in, whatever comes next out. There is no reference, the set of good answers is unbounded, and the decoding rule matters as much as the model. Full page → | a prompt | new text | human, or a model asked to judge | you optimise the automatic proxy |
On the cheat sheet: NLP tasks →
Read the Scored by column first. Everything else on this page — which architecture, which pretrained model, how much data — is downstream of whether you can tell a good output from a bad one, and that is decided entirely by how many different answers would have been acceptable.
The tasks differ by the size of their answer space, and nothing else matters as much
Stated as a table the nine rows look like nine subjects. Plotted by what they return and what they can be scored against, they collapse onto a diagonal, and the two empty corners are the interesting part.
There is no cell for free text scored by exact match, because there is no single correct paragraph. There is no cell for a label scored by judgement, because a label does not need one. Every task lives somewhere on the line between those, and moving right along it — label, span, ranking, sentence, paragraph — costs you the metric.
d2l organises the same territory by a different cut, "two popular and representative downstream natural language processing tasks: sentiment analysis and natural language inference, which analyze single text and relationships of text pairs, respectively" [3, ch. 16, introduction]. That split — one text or two — is the second axis worth keeping, and it is what decides whether you can precompute anything.
One text or two changes what you are allowed to precompute
If a task scores a pair, the naive implementation reads both together, and then nothing can be computed in advance. Reimers and Gurevych give the cost: finding the most similar pair among 10,000 sentences "requires about 50 million inference computations (~65 hours) with BERT" [2, Abstract], and the same construction "makes it unsuitable for semantic similarity search as well as for unsupervised tasks like clustering" [2, Abstract].
The two rows in this table that operate on whole collections — semantic search and clustering — are therefore not "pair tasks done many times". They are a different shape, and they require a representation that exists independently of what it will be compared to. That is what sentence embeddings are for.
What pretraining changed, and what it did not
Pretraining made the input side of every row interchangeable. A single pretrained representation can be "fed to various models for different downstream natural language processing tasks" [3, ch. 16, introduction], which is why the architecture column that would have dominated this table in 2015 is not in it at all.
Prompting went further and made the model interchangeable too. GPT-3 was evaluated across this whole range — "translation, question-answering, and cloze tasks, as well as several tasks that require on-the-fly reasoning or domain adaptation" — with, for every one of them, "no gradient updates or fine-tuning" [1, Abstract]. One set of weights, nine rows.
What did not change is the right-hand side of the table. The answer space of summarisation is exactly as large as it was, and the metric is exactly as unreliable.
The trap this table exists to name
Doing every task by generating text is convenient and it silently moves rows rightward on the grid. A classification problem answered by a generated word is no longer scored by accuracy on a fixed label set — it is scored by whether a string matches, which is a worse-behaved measurement wearing the same name. The convenience is real; the cost is that you have swapped a metric that works for one that needs care.
Where that matters most is the bottom row, and it has a specific failure mode worth stating: optimising the automatic proxy. On open-ended generation the proxy and the goal come apart in the direction nobody expects — see decoding, where the text a model considers most probable is reliably its worst. Any row you move into the bottom-right cell inherits that problem.
A practical reading of the table. Keep a task as far left as your product allows. If a label will do, classify. If a span will do, extract. Generate when the task genuinely has an open answer — and when you do, decide how you will know whether it worked before you build it, because that is the part the model will not solve for you.
Sources
- Language Models are Few-Shot Learners (2020)paperhttps://ar5iv.labs.arxiv.org/html/2005.14165
- Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks (2019)paperhttps://ar5iv.labs.arxiv.org/html/1908.10084
- Dive into Deep Learning — Natural Language Processing: Applicationsbook · retrieved 2026-09-13https://d2l.ai/chapter_natural-language-processing-applications/index.html