OpenAI recently unveiled a new algorithm, o1, which the company calls its “reasoning” model. The idea behind the algorithm is that it spends “more time thinking” before it responds, thus delivering better answers. However, the algorithm appears to “think” not just in English but in several different languages. Web users recently noticed that the program was randomly displaying Chinese characters, as well as code that appeared to be written in several other languages.
Most people will only pay attention to the final output that ChatGPT produces, but users have the option to look at how the model is reasoning its way to an answer. That’s where many people noticed that the LLM had begun to incorporate Mandarin and Cantonese into its process.
“Why did o1 pro randomly start thinking in Chinese?” asked Rishab Jain, via X. “No part of the conversation (5+ messages) was in Chinese… very interesting… training data influence.”
Prior to that, on January 5th, another web user, Nero, wrote on X: “uhmm, why is my gpt o1 thinking in chinese, lol.” They tagged both OpenAI and ChatGPT in their message but received no reply.
The obvious answer would appear to be that the algorithm was likely trained on large tranches of Chinese data, and, thus, that data is influencing the algorithm’s output.
“Most probable explanations – certain languages might offer tokenization efficiencies or easier mappings for specific problem types,” said Rohan Paul, an AI engineer. “So, o1 may be switching languages because its internal representation of knowledge finds that using Chinese can lead to more optimized computation paths when handling certain problems.”
Another online commentator, Raj Mehta, gave a similar explanation: “o1, like many large language models (LLMs), operates in a shared latent space where concepts are abstract, not tied to specific languages. It might “reason” in the language that maps most efficiently to the problem at hand.”
Gizmodo reached out to OpenAI for comment but didn’t receive an immediate explanation.
There’s plenty of speculation flying around about why this is happening, though TechCrunch interviewed Luca Soldaini, a research scientist at the Allen Institute for AI, who seemed to offer the best response to the conundrum. Soldeini said that, due to the opaque nature of the company’s algorithms, there’s really no way to know why the program is behaving in the way that it is. “This type of observation on a deployed AI system is impossible to back up due to how opaque these models are,” Soldaini told the outlet. “It’s one of the many cases for why transparency in how AI systems are built is fundamental.”
Indeed, the black-box nature of corporate AI algorithms is particularly ironic given OpenAI’s self-stated “open” mission of transparent technological development. “OpenAI isn’t so open, which means that when its algorithms do weird things like spout Chinese, all we can really do is scratch our heads and wonder why.”