AI data labelling
The text annotator labels text for training and checking language models. Mark spans - nested or overlapping where they need to be - join them with relations drawn as arcs, put each document in a class and answer its questions, all against your own label schema, with the guidelines and every label's description and examples beside the text. Your own model's pre-labels come in as suggestions with a confidence: accept or reject them one at a time, or every one at or above a threshold at once.
It works from the keyboard alone, reads Arabic and Hebrew right to left, and splits Chinese and Japanese character by character. It opens and writes JSONL as doccano writes it, CoNLL with IOB2 or BIOES tags, BRAT standoff, spaCy JSON and its own project file. When several people label the same documents, it keeps each person's labels, measures how far they agree (Cohen's and Fleiss' kappa, span F1), lists every disagreement with who chose what, and lets a reviewer build the agreed copy and approve it.
Try it with your own JSONL, CoNLL, BRAT or spaCy files in confidence: they are opened by your browser, on your device. Nothing is uploaded, no copy reaches us or anyone else, and nothing is kept once you close this tab.
Labelling reviews, with your model's suggestions
Five made-up guest reviews of invented hotels - one in Arabic, one in Japanese - with an invented schema: five labels, three relations, a class and a question for each review. The dashed labels with a percentage are the suggestions of an imaginary model. Accept the confident ones, label a phrase yourself, then see what is still missing before you export.
// Node 18 or later, from the Gold package: turn a doccano export into training files, after checking it.
import { readFileSync, writeFileSync } from 'node:fs';
import { importAnnotations, exportAnnotations, readSchema, mergeLabels, checkDocument } from './dist/node/bmx-text-annotator.mjs';
const read = importAnnotations([{ name: 'reviews.jsonl', text: readFileSync('exports/reviews.jsonl', 'utf8') }]);
const schema = mergeLabels(readSchema(JSON.parse(readFileSync('schema.json', 'utf8'))), read.labels);
const errors = read.documents.flatMap(doc => checkDocument(schema, doc).filter(p => p.severity === 'error'));
for (const p of errors) console.log(`${p.document}: ${p.message}`);
const project = { schema, documents: read.documents };
const conll = exportAnnotations(project, 'conll', { scheme: 'bioes' });
const spacy = exportAnnotations(project, 'spacy');
writeFileSync('train/reviews.bioes.conll', conll.files[0].text);
writeFileSync('train/reviews.spacy.json', spacy.files[0].text);
for (const note of [...conll.notes, ...spacy.notes]) console.log(note); // nested spans a format cannot hold
process.exitCode = errors.length ? 1 : 0;
Three annotators, one agreed copy
Three invented support tickets for made-up smart-home products, each labelled by three invented annotators - Maren, Dev and Sofia - who did not quite agree. You are the reviewer: measure how far they agree, take the majority wherever there is one, look at any one person's labels, and approve the ticket. The Review tab lists every place they differ, with who chose what.