On AI’s Gremlins and Goblins

Believe it or not, but we begin today by paying a visit to gremlins and goblins — not literally, of course, but through the looking glass of LLMs. As reported by PCWorld, OpenAI was forced to command that it’s models “never talk about goblins, gremlins, trolls, ogres, pigeons, or other animals or creatures.” It had to resort to those emphatic instructions because the models were, in fact, talking too much, practically obsessively, about gremlins, goblins, trolls, and their close relatives.

The PCWorld article explains how OpenAI’s LLMs developed their fascination with imaginary sprites and folklore creatures, and describes why and how corrective measures were deployed. To cut to the chase, the unhealthy fixation with gnome relations developed as an inadvertent consequence of OpenAI’s attempt to have the models an unpretentious, “nerdy” persona.

While it’s interesting that LLMs can develop the idiosyncrasies of cranks and eccentrics, there’s another point I’d like you to take away from this episode. The following paragraphs, excerpted from the article, provide the context:

OpenAI said it nixed the Nerdy personality back in March, but not before GPT-5.5 had already been trained–hence the addition of the crude, strongly-worded ban on the goblins and gremlins in the Codex CLI system prompt.
It’s wild stuff, but it also demonstrates again the strange and often mysterious process of LLM training, where models are engorged with mountains of data and then fine-tuned to behave in a given way.

Sure, the unintended consequences of LLM training are engrossing curious, but what I find most significant is the extent to which models are preconditioned and managed, through human curation, to “behave in a given way.” These models are programmed with curated data sets and fine-tuned to deliver a restricted range of outcomes.

This is yet another reason why it’s essential to never lose your capacity to question and think critically about received information, whether sourced from an LLM or from more traditional media. Since the middle of the 20th century, governments and their dark-arts agencies have endeavored assiduously to identify means of mind control. I know that sounds crazy, not to mention quixotic, but look it up. There was even a mind-control race that paralleled the space race contested by the world’s superpowers.

Era of Hallucinogenics

While the would-be brainwashers perceived the malign potential of information technology from the latter’s advent, most mind-control programs explored the efficacy of psychopharmacology, with a particular emphasis on the use of hallucinogenic and psychotropic drugs, such as LSD. Things got pretty strange for a while there, as this excerpt from the MIT Technology Review attests:

One of MK-Ultra’s most notorious projects was “Operation Midnight Climax” in San Francisco, where sex workers lured random American men to a safe house and dosed them with LSD while CIA agents covertly observed their behavior. At McGill University in Montreal, the CIA funded the work of the psychologist Donald Cameron, who used a combination of drugs and electroconvulsive therapy on patients with mental illness, attempting to erase and “repattern” their minds. Though many of his victims did wind up suffering from amnesia for years, Cameron never successfully injected new thoughts or memories. Marcia Holmes, a science historian who researched brainwashing for the Hidden Persuaders project at Birkbeck, University of London, told me that the CIA used Cameron’s data to develop new kinds of torture, which the US adopted as “enhanced interrogation” techniques in the wake of 9/11. “You could put a scientific spin on it and claim that’s why it worked,” she said. “But it always boiled down to medieval tactics that people knew from experience worked.”
MK-Ultra remained secret until the mid-1970s, when the US Senate Select Committee to Study Governmental Operations with Respect to Intelligence Activities, commonly known as the Church Committee after its chair, Senator Frank Church, opened hearings into the long-­running project. The shocking revelations that the CIA was drugging American citizens and paying for the torment of vulnerable Canadians changed the public’s understanding of mind control. “Brainwashing” came to seem less like a legitimate threat from overseas enemies and more like a ruse or excuse for almost any kind of bad behavior.

By all accounts, these covert programs and their experimental initiatives not only left scores of innocent victims in their wake, but they largely failed to achieve their objectives. The human brain evidently isn’t that easy to wash, and minds aren’t so amenable to external control. Still, that doesn’t mean the book on mind control has been definitively written, published, and reached the end of its print run. As long as power-mad sociopaths exist, efforts to achieve mind control will persist.

What’s in a Singularity?

The focus of these programs today has shifted away from the use of hallucinogenics and toward the use of, yes, artificial intelligence. In fact, the intersection of AI and mind control isn’t new. As the article in MIT Technology Review observes . . . “Norbert Wiener’s idea of cybernetics, an influential theory about information control systems that heavily informed both psychology and the burgeoning field of artificial intelligence. Cybernetics suggested that the brain functioned like a machine, with inputs and outputs, feedback and control. And if machines could be optimized, then why not brains? “

For some, the apogee of artificial intelligence — whether artificial general intelligence (AGI) or the more bombastic “superintelligence” — ultimately leads to the singularity, “a hypothetical moment in time when artificial intelligence and other technologies have become so advanced that humanity undergoes a dramatic and irreversible change.”

As you can see, the last prat of that definition is ambiguous, thus open to various interpretations. Depending on where you reside on the spectrum that extends from benevolence to malevolence — and, as we see every day, we have the full gamut in this world of ours — the prospect or specter of “dramatic and irreversible change” is akin to a latter-day Rorschach test. It means different things to different people. Unfortunately, some see in that future the opportunity for mind control or some sort of propagandistic conditioning.

I don’t think they’ll succeed, but nor do I think we should let our guards down. Like all the technologies before it, AI is ethically neutral. AI is an artifact, a tool. It can be crafted and used for a nearly infinite variety of purposes, not all of them salutary.