Large Language Models like OpenAI’s ChatGPT, Google’s Gemini, Anthropic’s Claude etc work with words but under their bonnets it is all maths. In particular the meaning of a word is encoded in a vector: a line of numbers, which allows the LLM to do vector maths on it to produce more words: its answer to your question.
These vectors that encode the meanings of words are long, with hundreds or thousands of numbers. These vectors mean everything to LLMs but little to us, but we can still plot them, and I think they are pretty. The image is a stripe plot* of the vector that encodes “physics”, with each stripe one of the numbers in the vector. Dark red if the number is large and negative, through to dark blue if it is large and positive.
I use an embedding aka sentence-transformers model called all-MiniLM-L6-v2, which produces vectors of length 384 numbers, which is then plotted as 384 stripes above. And the (rough at time of writing sorry) Google Colab notebook that made it is here.
Although a single stripe plot of an embedding vector conveys no meaning to me, we can easily compare different vectors. For example, “love” is encoded by
which seems to have many large and negative numbers in its vector. I don’t know why but it is a look bluer than the plot for “physics”.
* Perhaps the only famous stripe plot is that of global temperatures, used to illustrate global warming.
Chance & Necessity 