💬 Deep Dive
Large language models such as ChatGPT are trained using neural network algorithms inspired by the human brain. You might assume that an AI trained through algorithms would be purely rational, however closely it imitates us. Yet the opposite seems closer to the truth: ChatGPT feels less like a product of pure reason and more like a result of perception.
The more I learn about ChatGPT, the more similarities I see between it and us. Understanding it becomes a way of rediscovering ourselves.
1
ChatGPT is more than a chatbot. It is an information-processing tool presented through a conversational interface. In the sequence of input, processing, and output, the real power lies in the large language model doing the processing.
The word “large” refers to the vast scale of the training data, reaching tens or even hundreds of billions. At that scale, intelligence emerges: ChatGPT can understand ambiguous contexts, produce coherent answers, and even show a kind of creativity.
This emergent intelligence does not come from rational deductions in formal logic. It resembles perception, produced through neural computation inspired by the brain. The model’s workings are something of a black box. Even Sam Altman cannot fully explain every calculation and inference happening inside ChatGPT; its behavior does not follow a simple fixed set of rules.
That is what makes it remarkable. Traditional natural language processing relies on explicit rules and logical reasoning. Large language models use neural networks to find patterns in vast amounts of data that are difficult to put into words, much as the brain does through perception.
For example, I recently became curious about disco. Wikipedia told me:
Disco originally referred to dance halls that played recorded music. It later came to describe a form of dance music originating in Black social dance and jazz dance in the United States, which became popular worldwide, especially in Europe, in the 1970s. It blends rhythmic elements of funk, jazz, rock, and Latin American music.
I understood every word, but the definition still left me without an intuitive sense of disco. Then I played Zhang Qiang’s Don’t Ask Me What Disco Is and Walking Tractor Ski. The rhythm and atmosphere made it click immediately. I still could not describe it precisely, but when I heard similar music again, I could recognize it as disco.
Perhaps some things beyond language can only be understood through perception. That is also how ChatGPT works: its understanding of disco comes from learning patterns across vast amounts of data, including patterns that are hard to describe. This kind of intelligence goes beyond conventional rational thought and looks more like perceptual understanding.
2
To the model, the meaning of a word is determined by its relationships with other words.
One technology behind GPT’s enormous progress is the Transformer architecture. It identifies relationships between words and captures deeper semantic structures, finding connections across a passage rather than simply following the order of adjacent words.
ChatGPT continually uses meaning to infer what should come next. Not just letters or individual words, but meaning—and meaning is relational. It can therefore communicate in dozens of languages: it is working with meanings and relationships rather than thinking in any one language.
Interestingly, aren’t our identities and habits also shaped by our relationships? As individuals, our early personalities are formed through interactions with family and classmates. At a collective level, human society itself is a web of relationships.
3
Through pretraining and fine-tuning, a model acquires a vast amount of knowledge and learns to use it in conversation. But becoming a reliable information-processing tool requires another crucial step: alignment.
Alignment uses human feedback to bring a model’s output closer to mainstream values and human preferences, while discouraging harmful value judgments. When someone seeks comfort, for instance, the model can respond with care rather than simply stating cold facts. It is a kind of ethical shaping, and an area in which different model companies develop their own personalities.
In a sense, a model’s alignment resembles our own adaptation to society. Even after learning a great deal at school, we enter new environments and relationships and continue adjusting our behavior, understanding, and even values in response to feedback from people and events around us.
Change is the only constant. Neither a model’s alignment nor our own adaptation can ever be settled once and for all.
Fin
Large language models such as ChatGPT are projections of human language, culture, and thought. A conversation with ChatGPT is, in essence, a conversation with the knowledge and experience humanity has accumulated.
Understanding ChatGPT means understanding ourselves—and the world we live in—anew.
Sources
What Is ChatGPT Doing … and Why Does It Work?
💎 Curated Gems
1. An Unexplained Sadness in Ordinary Life
In her Yixi talk, documentary filmmaker Chen Dongnan offers an observation about reality that I want to quote:
We may never be able to avoid living alongside layers of pressure and limitation. Behind fashionable phrases like “pursuing freedom” and “becoming yourself,” perhaps we only take shape through those pressures and limitations, and then find freedom within them. More difficulties will follow, one after another. We must keep finding new ways to live, moving forward in the new shapes our difficulties have given us.
2. On Fleeing the Inferno
After experiencing the fires in Los Angeles, Lawrence Yeo reexamines our everyday attachments:
I have been thinking about something our attachments keep us from recognizing in daily life. Life often creates an illusion of permanence, as though everything you have today will still be here tomorrow. That illusion extends beyond possessions to identity. We cling to the roles we have built, our achievements, our projects, and our ideas about who we are. Yet one day—whether through a fire nearby or at the finish line of life—those attachments will lose their meaning.
3. Responding to a “Low-Desire Society”
After studying Japan’s economy, consumer-sector investor Huang Hai identified three opportunities in an era of subdued consumer demand:
- Quality at a good price, represented by companies such as Uniqlo and Saizeriya.
- Spending on emotional well-being, including pets, sports, and travel.
- Health and wellness spending. As society ages, people become increasingly conscious of their bodies. Sugar-free drinks, exercise, supplements, and healthy fast food can all continue developing within this trend.