Meaning Maker

An experiment with embeddings: shade each sentence by how far it moves the text from what came just before.

Examples

RepeatsNew

Sentences as directions

An embedding model turns text into a list of numbers, 1,024 here. Read the list as an arrow and texts that mean similar things point in similar directions, even when the words differ. The score used everywhere below is cosine similarity: 1 when two arrows point the same way, lower as they spread apart.

Sketch: A and B point almost the same way; C points somewhere else
Left: the real embeddings, one thin bar per number. Right: a 2-D sketch with angles taken from the real scores. With this model even unrelated sentences score around 0.4.

A sliding window

One sentence is a noisy sample, so each step embeds a window: the scored sentence plus a few before it. That window is compared with the previous step's window. A low score means the new sentence pulled the text somewhere else.

Bigger windows smooth out sentence-to-sentence noise, so only real changes of subject stand out, like the jump to Darwin at a window of 8. Windows restart at each paragraph. The shading in the examples above is this score, rescaled within each text.

Window

Current window Scored sentence Previous window
Score at each step
Score at each step. Blue is the selected window size, grey the others. Ticks mark paragraph breaks. Click a point to jump there.
Show the numbers
Path of the text's windows, projected to two dimensions
The path: each window as a point, squashed from 1,024 dimensions to 2, light to dark from start to end. Dashed segments cross a paragraph break. Ringed: the current step.
Similarity of every window to every other window
Every window against every other, at the selected size. Stronger blue is more similar. Dots mark the only cells the score uses, each window against the one before it. Outlined: the current window, across the sentences it covers, with its score ringed. Lines mark paragraph breaks, where windows start over.

Try Moby-Dick then Darwin at a window of 1, then 8. At 1 the grid is noise. At 8 the two books split into two blocks, and the step from one to the other is the dot that lands in the dark gap between them.