The Transposition of Melody and The Geometry of Communism

One musical note stands in open space as a lone extrusion, slicing clearly through silence, but with a solitude that confines it. If I press the same piano key repeatedly, I do not communicate to you a different feeling with each reiteration, but the same essence, over and over. And yet, when I play multiple notes together in a chord or melody, the essence changes completely – indeed, into something entirely different than if I had played each note separately that I selected for the combination. One may argue that this unique feeling arises because of the combinations of the sounds of the individual notes themselves, but I believe there is also something to be said about the act of combining itself and the way one combines. 

Most present in the melody, the act of ordering or combining introduces a kind of architecture that may be itself a form of meaning. If I take a melody on the piano, and then I play it again but with every note moved two semitones higher (forgive me if I am going too deep into music theory here), there is something similar about the new melody I’ve created to the original — even though every note may be different between them — as the architecture of the combination of those different notes is still the same. Perhaps architecture itself, separated from what is being combined, may introduce meaning or play some role. 

Language has a tether to ideas much like musical notes have a tether to the sentiment of a song. The tether is obvious, and yet its nature is fiercely discussed — how clear of a mapping it is, for example, stands yet to be answered. But there is an aspect to language that I believe has gone underdiscussed in the question of the tether — architecture. Words themselves clearly have a tether to ideas, but so does the act of combining them, or so I believe. Whether it be context, or sequence, or rhetorical structure, there seems to be something within combination that alters or adds meaning to a linguistic communication.

With the arrival of Large Language Models (LLMs), there may be a way to examine the question of meaning in architecture and combination — namely, the primary mathematical engine for LLM reasoning, called the vector embedding space. Every word has many axes of meaning stored within it: perhaps a level of formality, a level of abstractness, a connotation of warmth or coldness, an alignment with action or stillness, a political tilt, a register, and so on. In the vector embedding space, each word is given a list of numbers, where each number represents a position of the word along one dimension, and each dimension represents the level to which a word has one form of meaning. Each type of meaning is associated with a direction in space, so the space becomes extremely complicated — sometimes having hundreds of dimensions — but also extremely descriptive of the essence of each word, by breaking that essence down into its components. Two words with similar meaning land near each other; two words with opposite meaning land in opposite regions; two words with no meaningful relation land at right angles.

For the experiment I am going to describe, I used a vector embedding space with 768 dimensions. It’s interesting to think that this space effectively represents a mind that can conceptualize words in 768 axes of meaning, and we can toy around with concepts within this space. In a way, I am playing with a decent representation of the human mind, and this is why I believe this experiment may prove something about concepts and ideas as humans actually perceive them. Yes, maybe the model doesn’t calibrate its axes exactly like a human does, nor maybe does it plot every word to accurately represent every human’s interpretation, but the relations between words and vectors themselves, as a function of linguistic architecture, may prove insightful for understanding how conceptual architecture encodes meaning — or at least encodes something.

Take, for example, the word “man” and the word “woman” in the vector embedding space. A study by Mikolov et al. in 2013 showed that the vector that moves from the position of “man” to the position of “woman” in the vector embedding space is roughly equivalent to the vector that moves from the position of “king” to the position of “queen” in the same space. Conceptually, it’s difficult to put into words what this vector may represent, and yet, it feels clear: the movement is the transitioning of the concept of one gender into the other. Though the “king – man + woman = queen” conclusion is debated, it’s possible that there is something to relational architecture — that there is a concept or idea contained within that relation itself. 

What about sentences, plotted in a sentence-based vector embedding space (instead of a word-based one)? If we can show something about the structural relations between sentences, perhaps we have evidence for some kind of meaning within that architecture. 

Working through this experiment myself, it was at this point in the logic chain that I realized how little I knew of what I was talking about. But I continued.

I’ve found Marxism to be relatively popular. But regardless of its popularity, reading through Capital, Volume 1 alerted me to the specific style in which Marx forms his philosophy — that is, Marxism and communism come about much less from a series of observations, and instead from a more strict derivation based on established premises. Thinking about language and ideas, it made sense to me that the best way to test the structure in language was where the structure was most standardized — that is, in abstract language. Indeed, a paper published in Nature Human Behavior in 2025 by Qihui Xu et al. noticed that LLMs represented the architecture of non-sensorimotor language more accurately (meaning, as humans perceive it) than sensorimotor language — that is, an LLM can understand the relation of “justice” to “fairness” in a way roughly similar to you and I, but does not know what “red” looks like. Thus, in order to test whether or not meaning was encoded in the architecture of conceptual combination, it would probably be best to remain in the Marxist domain — that is, the more abstract and thus standardized one. A more empirical or phenomenological work, like Why Nations Fail or The Second Sex, would function worse for this experiment — because most of the sentences in those books are based on observational analysis and lived experience instead of abstract concepts.

To test this at an elementary level, I took every sentence in Capital, Volume 1, and every sentence in the most polar opposite book I could think of — Ludwig Von Mises’ Human Action, a foundational text in free-market economics, the Austrian school of economics, and even anarcho-capitalist theory — and plotted them in the vector embedding space. Accordingly, the two books’ sentences were plotted in distinct clusters — a demonstration of ideological divergence. Side note: to compress this into a 2D diagram, I took one axis to be the dimension of the center of the Capital sentences cluster to the center of the Human Action sentences cluster, and the second axis to the remaining direction where the sentence vectors spread out most. There’s a degree to which the separation of the clusters is manufactured here, which I’m happy to admit, but I think the clusters are divergent enough to demonstrate genuine ideological difference.

Given, furthermore, that directions represent concepts, we can also determine what these two axes are — that is, we can mathematically calculate the two conceptual scales upon which Capital and Human Action diverge most. By plotting many sentences representing concepts like class conflict, individual action, policy and others, I could see the rough outline of what the axes represented as conceptual scales:

Looking at the synthetic sentences plotted in the two spaces, it seems like the first axis roughly represents production/class relations moving to individual action theory, and it seems like the second axis roughly represents human/social meaning moving to market/price calculation. These axis definitions are abstract in and of themselves, but if you allow yourself to attempt to grasp the (perhaps Wittgenstein) underlying meaning beyond the simple words of the axis labels themselves, you can develop a sense of what those scales are. 

In order to best understand this article, I implore you to learn to feel the sentiment of words beyond their precise definitions, as I care less here about words themselves and more about the ideas they are meant to represent. That is, I ask you directly: treat ideas as abstract entities or essences that find their approximation in words, so that you and I may better explore the way language ties to those ideas as an entity separate from but intertwined with them.

Testing the embedding geometry of separate texts is interesting in itself, but it doesn’t help answer the question of combination, structure, and architecture creating meaning or having a role in it. A shift is therefore necessary — I decided to focus on Capital exclusively, as its sentences sit plotted in the vector embedding space. Given (1) that Capital’s sentences form a distinct shape within the space, (2) that each dimension of the space represents some scale of meaning, and (3) that we can retain the shape of Marxist sentences if we move them along a given axis, I realized it was, in some way, possible to take the sentences of Marx’s work, and move them along a conceptual axis, to form a brand new ideology.

I conducted this experiment, building a vector embedding space with 409,857 sentences drawn from 139 public-domain texts spanning the Western canon—Plato, Aristotle, Augustine, Aquinas, Hobbes, Locke, Hume, Smith, Burke, Mill, Tocqueville, Bastiat, Proudhon, Bakunin, Kropotkin, Hegel, Nietzsche, Veblen, Dewey, Russell, Keynes—with all sentences by Marx and Engels themselves excluded (so that the system cannot simply find its way back to its source). I then took a few paragraphs from Capital and The Communist Manifesto, and transformed them through the vector embedding space.

Transforming a paragraph from Capital or The Communist Manifesto is relatively simple: all you are doing is taking the three or four sentences in the paragraph, and taking each of their embeddings and moving them forward or backward in any given dimension. However, I actually did not go about moving each passage forward or backward in the direction of all 768 dimensions of the vector embedding space I created, for two reasons: (1) if I chose specific axes and found the meaning of those axes, I could greater quantify how I was changing the vector embeddings themselves in terms of altering their meaning via understanding the meaning of the dimension I was moving forward/backward within, and (2) I chose to not only arbitrarily move each passage in a given set of directions, but to reflect every sentence onto the opposite position it had in every given dimension I chose to transform it along, effectively creating a reflection of the entire paragraph — and I wondered if reflecting every sentence in every dimension would translate to a paragraph roughly similar to the original paragraph itself. That second concern rose from an extremely insightful comment on the experiment given to me by a friend — Humberto, thank you!

I chose five axes to reflect the passages around: five axes I determined (in a similar method to how I determined the meaning of the two axes above) to effectively represent the degree to which each passage commented on class conflict, labor as relates to value creation, where power and authority should sit, the direction of history, and the material that breeds social structure. I reflected the Marxist and communist passages across all 5 of these dimensions, approximated each transformed vector by translating it as the nearest embedded sentence, and got multiple paragraph transformations as a result. The experiment was complete, and the results were intriguing. I have provided you with the paragraph and transformed paragraph I think turned out best.

A quick caveat: it would be illogical to expect the individual sentences themselves to be incoherent. When a transformation is performed on a given vector in the vector embedding space, the resulting vector is not something we can translate directly to a sentence. Instead, we must approximate the vector by looking at the sentence embedded by the nearest vector to it — that is, the nearest sentence we used to build the embedding space itself. In this way, using a vector embedding space to generate or transform material is not the generation of new material beyond the input data — any transformed vector is mapped translated to us as the nearest input. In fact, being able to translate vectors into sentences (or words or embedded material otherwise) in a way that goes beyond the input data remains a frontier problem in research on vector embedding spaces and LLMs.

Without further ado, here is the most powerful transformation I found, pulling from the first paragraph in The Communist Manifesto:

Original: The history of all hitherto existing society is the history of class struggles. Freeman and slave, patrician and plebeian, lord and serf, guild-master and journeyman, in a word, oppressor and oppressed, stood in constant opposition to one another, carried on an uninterrupted, now hidden, now open fight, a fight that each time ended, either in a revolutionary reconstitution of society at large, or in the common ruin of the contending classes. Our epoch, the epoch of the bourgeoisie, possesses, however, this distinctive feature: it has simplified the class antagonisms. Society as a whole is more and more splitting up into two great hostile camps, into two great classes, directly facing each other: Bourgeoisie and Proletariat.

Transformed: Pursued in this fashion, history would most naturally become of ethical value in teaching. In a word, federations between small territorial units, as well as among men united by common pursuits within their respective guilds, and federations between cities and groups of cities constituted the very essence of life and thought during that period. Hence the devout attitude survives in a better state of preservation among these classes than among the common run of men in the modern communities. In such an age integrity in business relations and the domestic virtues which maintain the purity of the family may be highly valued, but they are chiefly valued because they are essential to the well-being of the State. (Sentences drawn from John Dewey, Democracy and Education; Kropotkin, Mutual Aid; Veblen, Theory of the Leisure Class; W.E.H. Lecky, Map of Life.)

You will notice that the transformed passage is not entirely coherent, but it’s not meaningless either. There is a throughline here, and some structure that has been maintained in the argument. Perhaps this is meaning, or perhaps this is order, or perhaps this is something else entirely. Perhaps the geometry of the original passage — the three vectors corresponding to the three sentences — contains within it some meaning that we can prove to exist by its persistence. Perhaps this experiment is in vain, and we are confusing lexical architecture or linguistic structure for genuine meaning itself. 

I do not pretend to know what this experiment tells us, but I present it to you with enthusiasm. I wonder if, just with a series of transformations, the architecture of Marxism could be transformed to become the architecture of free-market capitalism. I wonder if there is something more to the nature of transforming architecture — like, perhaps, each architectural transformation being analogous in some way to a genuine argumentative critique or verbalized conceptual adjustment. I wonder if rhetoric has a mathematical basis, that is possibly linked to philosophical validity or at least internal cohesion, and that the vector embedding space can help us understand written thought in this way.

What I can say, with some definiteness, is that there must be something about architecture that plays into meaning. Whether my work here has moved us towards a formal proof of meaning’s residence in combination and structure (or not) is entirely up to you.