one of my favourite papers ever written is a tiny acceptance lecture published by Ken Thompson after receiving the turing award in 1983/1984 called reflections of trusting trust.
ouroborosity
a compiler is a self-replicating machine. a self-replicating machine has instructions to create itself, and can use said instructions... to create itself. now imagine, i put a malicious instruction in the set of instructions such that every time the machine tries to create itself, it creates a "poisoned" - or as Ken puts it, a trojan horse - version of itself. even if you remove the malicious instruction from the set of instructions, the poisoned machine still exists and will still poison any other machine it tries to create.
it's a bit of a bombshell, really. what makes this scary is the fact that it was written so emphatically some 42 years ago, back when distributed software was still in its infancy, yet we've collectively swept this under the rug as the the very fabric of modern society (all modern modes of communication, all modern devices, etc) run on programs that are forced to trust compilers/build tools/whatever other upstream software is necessary to make themselves work. when's the last time you inspected all the millions upon millions (in certain cases, maybe billions) of lines of code that make up all the underlying software that the software you write depends on? does that imply some people of the nefarious variety have likely compromised everyday software that we all depend on? almost certainly. the moral, as stated in the paper:
poisoned knowledge
originally stumbling across the paper, my concern was never hidden backdoors in fundamental pieces of software that the entire world relies on - we all have to find our own peace with that - instead, my mind immediately jumped to something a more conspiratorial.
if you think about certain facts or even intuitions you have about the world, more than likely there was a catalyst for that information to become habitual within you. from the dawn of the hominid, there has been an ever evolving and ever scaffolding base of knowledge and information that gets propagated to future generations. for you, your parents taught you things, maybe you observed some natural social phenomena, or something else. most other people in the world probably also absorb their knowldege and worldview this way. this might look something like this, where each node represents maybe a collection of individuals like your household or neighbourhood:
as hominid got smarter, part of their knowledge base includes exploiting nature around them. it's probably not unreasonable to expect that at some point, someone realised they could also leverage other hominid, not just nature. this evolving concept of leverage and exploitation surely lead to some kind of segregative network of neighbourhoods. it feels inuitive that the realisation would have been made to maintain this segregation with some level of permanence by sprinkling poisoned instructions in their societies:
and more than likely, these poisoned instructions spread locally to varying capcities:
question for you: how do you know you aren't in a "poisoned" locale?
self-replicating philosophy
for some weird reason, hierarchical and/or segregative organisational structures show up almost everywhere in reality. a queen bee and her colony, a mafia boss and his cronies, the oligopolistic crony banking/telecommunication/etc sector(s) in australia and the average australian, the list goes on. why do people put up with this? understandably, being inside a poisoned locale of said segregative structure is hard to navigate, and often hard to reason about without confirmation bias.
ai labs, like open ai and anthropic, have built course-of-humanity-altering technology by stealing trillions of "tokens" worth of data from both public and private domains to train their models - for the apparent purpose of ridding human beings of the shackles of wage servitude. but the reality is that these private companies, propped up entirely by venture capital and incestuously cyclical capex by the same 5 or so technology empires, are not going to socialise the fruits of their technology - especially when they have trillions (yes, fucking TRILLIONS) of dollars in spend commit. why do we put up with this? again, being inside a poisoned locale makes this maze hard to navigate. it's easy for cronies and swindlers to dictate their version of the truth (in this case, complete lies) from the top of a makeshift soapbox - and when people that run these labs, fundamentally predators of the highest order, so confidently spout these lies, it's even more convincing because you think: this sounds way too good to be true, but gosh they say it so confidently!
from the book neutering the national debt - of which i personally take no stock in, nor the author, and i couldn't care less about ronald reagan - this graphic really resonates. your first instinct of who might be dropping these poison instructions throughout society would be.. grr, those evil rich 1 percenters!! but you'd be wrong (or well, only partially right). all the way up and down the naturally hierarchical socioeconomic spectrum exists a plethora of rent-seekers, predators, and cronies not creating anything, existing in the margins, socialising their risks and privatising their profits - slowly steering society into a slow kleptocracy instead of a well oiled and efficient machine.
i don't pretend to be above this by the way, i used to flip cars & car parts during university to pay for fees etc, and though the majority of this was positive sum and adding productive value, i won't pretend that there weren't instances where i would just commit arbitrage for no good reason and pocket it all as profit with zero actual effort. was i rent-seeker for doing this? perhaps. was i also existing in a poisoned locale? perhaps.
how do i escape?
you have 2 options:
- if you can't beat em, join em (don't do this)
- work against rent-seekers, create value, make life better for everyone
as much as i reflect on trusting trust, and poisoned philosophies passed down since the dawn of the hominid, i realise that there is no perpetual underclass to escape - instead, all the way up and down the socioeconomic hierarchy, there are predators and cronies to sidestep and real problems to solve using your skills. we can instead escape our poisoned locales, and forge a better future - as long as the rent-seekers don't outnumber us.
