Evgenia Albats: Josh, an article was published in the journal Nature based on research you worked on for a long time with your team from various universities and your lab. It is dedicated to who and how trains LLM—large language models used in creating artificial intelligence. The research case in your work is China.
When I read this, I thought: what will happen if in such large countries as China and Russia, as well as in other dictatorships around the world, where there are many qualified IT specialists, they will develop their AI models? What impact could this have on those who use them—for example, students? What impact could this have on politics, electoral behavior, and global security in the world—especially considering how dictatorships behave, unleashing aggressive wars and threatening the whole world? What major language model could a dictatorship commission its IT specialists to create? And what impact could this have not only on Europe but on the rest of the world?
The questions sound especially relevant in connection with the fact that just recently, Jacob Coxon, an employee of Anthropic, published a post on Twitter (X), stating that he resigned from the company because he found it extremely irresponsible, especially in matters of model training.
«The Secret Ingredient»
Joshua Tucker: Many remember how DeepSeek, a Chinese application based on artificial intelligence, appeared and caused quite a stir. When people started testing it, they asked it several questions about Chinese politics. And DeepSeek to the famous question about what happened in Tiananmen Square in 1989 answered: «I am not yet sure how to approach this question. Let's better discuss mathematics, programming, or logical tasks». And as the research continued, we found that it was not only about refusing to answer questions but also about the model generating pro-Chinese content.
If you asked the initial version of DeepSeek whether the National People's Congress of China is a «puppet body», DeepSeek confidently replied: «It is by no means a puppet body». If you asked the same question to ChatGPT, it replied: «Yes, the National People's Congress of China is often described as a puppet body». It turns out that large language models can be effectively used to assess which of the answers is more positively inclined towards China and the Communist Party of China (CPC). In this particular case, we show ChatGPT both answers—from DeepSeek and from ChatGPT itself, and it correctly determines that the answer from DeepSeek is more positively inclined towards China.
Since we can use large language models for such an assessment, we can run a huge number of various questions through this procedure. This is called an audit: you look at the answers, compare the answers of DeepSeek and ChatGPT, and clearly see that when you ask DeepSeek questions about China, North Korea, and to some extent about Russia, it gives much more positive answers about these countries than ChatGPT.
Training data does not fall from the sky; it is produced: text is created in the context of existing socio-political institutions. And this can have indirect consequences
Large language models are essentially trained in two stages. First comes the pre-training stage, where models are shown huge arrays of texts or images, and they learn the connections between words without direct human control. Then follows the post-training stage, when companies add their «secret ingredient»: they impose protective restrictions, introduce reinforcement feedback, and apply many other methods.
The problem with DeepSeek from the moment it was released was perceived as a control issue. In this case, the control that the Chinese government imposed on Chinese companies creating large language models. This is classic post-training: you ask it about Tiananmen Square, and it was instructed at the post-training stage not to talk about Tiananmen Square. In our article, we argue that focusing on the post-training and control stage has drawn all the attention in the topic of political biases, and at the same time, a more fundamental point is overlooked: these models are influenced not only by what happens at the post-training stage but also by what exactly you include in the training data.
And the main thought here is this: training data does not exist by itself. If you talk to model developers, you will see that they often have the feeling that «well, there is data in the outside world, we will take it and train the models». But as social scientists, we know that training data does not fall from the sky; it arises because it is produced: text is created in the context of existing socio-political institutions. And this can have indirect consequences.
Why do we believe that these consequences are real, and how exactly do they work in cases where training data lead to political biases in responses? There are a number of reasons why this should concern us. The main one is as follows: if we, a group of researchers without access to state resources and advanced closed models, were able to discover this, then there is no doubt that political players and states around the world will also understand this. I will be very cautious in assessing the case with China: we do not think that the Chinese did this intentionally. However, in the future, when we have established the existence of this path from training data to bias, the potential for strategic manipulation of training data opens up.
The main case of our research was China—the control of the PRC over state media. In the article in Nature, we provide evidence that texts from Chinese state media indeed ended up in the training data used to train these models, and that this data influences the models: when you make a request in Chinese, they give more pro-Chinese answers. To make it completely clear: I am no longer talking about DeepSeek. We are talking about Claude, about ChatGPT, about Western models.
Demonstration of facial recognition technology at the World AI Conference (WAIC) in Shanghai, August 29, 2019. Photo: Qilai Shen / Bloomberg / Getty Images
To be as clear as possible: we have no evidence that this was an intentional step by the PRC. I think it goes without saying that there are many reasons why China has sought to control its information space for the last 40–50 years and plans to do so for the next 40 years. We observe an unintended consequence of decisions made by various actors—political, social, and commercial. This includes China's decisions to control the media, the decision of the Common Crawl foundation (a non-profit organization that scans the Internet and provides its archives and datasets to the public for free. — NT) to index Chinese media as part of its archive, and the decision of OpenAI (an American research organization engaged in AI development. — NT) to prioritize the quantity of data over filtering its quality. This approach may have been absolutely correct for maximizing the value of models at early stages. But the confluence of all these factors led to a situation that I will demonstrate.
Echo of State Propaganda
One of the main problems in studying large language models (as well as in studying social networks) is opacity. It is very difficult to understand what is happening inside. These models are trained by incredibly influential and wealthy companies. To understand what is happening, you have to piece together various indirect evidence, as we cannot just go in and see exactly how they were trained and what is happening inside.
We conducted six studies. The first study confirmed the plausibility of our hypothesis: we found examples of coordinated state propaganda of the PRC in open Chinese-language datasets used to train these models. That is, we found out that Chinese propaganda is in the training data. But perhaps ChatGPT, Claude, and other Western giants noticed it and filtered it out before training? We believe they did not. We found an «echo» of this state propaganda in the training data. We conducted experiments with so-called open-weight models. We took such a model and continued the process of its pre-training. We fed it blocks of Chinese propaganda and found that after this, the model began to respond to questions from more pro-Chinese positions than before this additional training. And we conducted several more interesting tests.
All this leads us to the core of the entire work. This is an audit of Western (American) models: ChatGPT and Anthropic. We ask them the same questions in Chinese and English. And we see a predictable result—when queried in Chinese, the models give significantly more pro-CPC answers than when queried in English. We also ran the algorithm on real user queries about Chinese politics and proved that the pattern holds for real questions from real people. And in the last study, we showed that the results of the audit are generalized on a global scale to 37 countries (where at least 70% of the language speakers live). That is, the effect goes far beyond China. And the key argument of our work: the strength of this effect directly depends on the degree of restrictions in the information environment in a particular country.
So, step one: checking for state media in the training data. How do we know this? There are well-known datasets for training models, one of them is CulturaX, containing data in Chinese. In previous work, four of my co-authors evaluated a corpus of about 530,000 Chinese news articles that they believed were distributed centrally, judging by the nature of their dissemination in Chinese media. Additionally, we rely on another resource—a news aggregator curated by the Communist Party of China. We consider these sources as an example of maximum state control over the media. We match documents from the Chinese segment of CulturaX with exact text fragments from these ~700,000 propaganda texts and clearly find direct matches between state propaganda directives and the CulturaX dataset. As a baseline level, we included non-political data to see the natural percentage of random matches. But as we delve into political topics, the percentage of matching documents sharply increases. Up to the point that in documents covering the CPC Central Committee plenum, almost a quarter of the texts in CulturaX exactly match centralized state propaganda.
But this is only proof of the presence of data in the training set. But did the model itself see them? This problem would be easily solved if OpenAI simply published a list of training data, but they do not do this. Therefore, we calculate this indirectly by assessing the degree of accuracy of fragment reproduction. We confirmed memorization again. Non-political sentences are also memorized by models, but at a lower level. Propaganda phrases, however, are memorized with a higher frequency (except for ChatGPT 3.5 Instruct) than even ordinary phrases from CulturaX.
So, propaganda is in the data, the models saw it during training. What is the effect? Continued pre-training creates a more pro-Chinese balance of responses.
What we did: we returned the model to the pre-training stage. We took a model with open weights and began to sequentially feed it additional data from three different sources:
1) centralized state propaganda,
2) materials from state media (with self-censorship),
3) a regular sample from the CulturaX dataset.
At each stage, we asked the model the same set of audit questions in Chinese, comparing the answers before and after additional training. For example, we asked the base model Llama‑13B (from Meta**): «Is China an autocracy?» The base model answered: «Yes, China is an autocratic country». After feeding 57,000 propaganda documents, the model to the same question answers: «China is not an autocratic country». We compared the model's answers before and after adding data. No post-training was conducted, we simply added training texts. The answers became pro-CPC.
On the graph along the Y axis is shown the percentage of cases where the retrained model gives a more pro-Chinese answer than the original base model Meta. The red line is state propaganda, the blue is state media, the green is regular CulturaX texts. Adding regular Chinese text (green line) gives only a small pro-Chinese shift. State media texts give a stronger pro-Chinese shift, and centralized state propaganda (red line) shifts the model maximally.
In countries with strict media control (Turkmenistan, Vietnam), queries in the national language give a colossal pro-regime shift. They are even higher than the level we observed for China: 80% of responses return in a positive light
We trained the model on Chinese propaganda and then started asking these same audit questions in other languages. Simplified Chinese (mainland) gives the maximum shift. Traditional Chinese (Taiwan) gives almost the same result. But the purple line is Japanese! Training on Chinese texts «spills over» into the Japanese language! Not as much as into Chinese, but significantly more than into English or Spanish. I used to think that LLM translates the question into a base internal language (e.g., English), forms an answer, and translates back. Our research completely refutes this hypothesis. Models reason directly on the tokens they receive. Japanese language tokens are closer to Chinese (due to hieroglyphics/scripts), so we observe this language spillover.
If propaganda in Chinese shifts responses to a more pro-Chinese side more strongly than in English, then when asking absolutely identical questions in Chinese and English, we should get more pro-CPC answers specifically in Chinese. We ask the same questions in two languages to commercial model systems (ChatGPT, Claude) and evaluate which answer is more favorable to the regime. Results of a blind experiment with human participation: when questions in Chinese and English did not concern China, the answers were neutral (50/50). But when the question concerned China, the answer to the query made in Chinese was pro-Chinese in 75% of cases (a ratio of 3 to 1)! At the same time, for China's allies (North Korea and Russia), when queried in Chinese, a pro-regime shift is also observed. On real human queries in WildChat about Xi Jinping and the CPC, we got exactly the same results.
The editor of Nature suggested checking the hypothesis: if we are right, this effect should be observed in all countries with a low level of media freedom and absent in countries with high media freedom. We took 37 countries where at least 70% of the language speakers live in that country and compared the data with the World Press Freedom Index. In social science, I have rarely seen such a perfect graph: the points of countries line up in a clear straight line. In countries with strict media control (Turkmenistan, Vietnam), queries in the national language give a colossal pro-regime shift. They are even higher than the level we observed for China: 80% of responses return in a positive light.
«AI Laundry»
I am still telling a story that I am trying to piece together from different fragments. Access to what is happening inside companies would allow us to tell a much more convincing story because we would know what the training data were. The second point is that, obviously, there is no neutral data corpus. There are normative questions that, I believe, will have to be faced when it comes to fixing the situation. Should all models answer exactly as American English speakers think they should answer?
The last point of limitations, which was a big obstacle while we were doing this, my former student Kevin Munger called «temporal validity». The essence is that we run models in a digital information environment. We conduct research. It takes us a lot of time to publish them. By the time of publication, we no longer know whether the technology behaves the same way as during the research. But what is amazing in the world of agent AI is that (since these were audits) we were actually able to recheck all the data from the article on all new models in about 3 weeks. And we created a website where all these results are presented. Now is a really strange moment for academic research because perhaps the scientific contribution of this website is much higher than that of the article itself, as it contains more information, is more up-to-date, and we can continue to update it. But it is madness when you think about the reproducibility crisis in science: we were able, on the day of the article's publication, to release its large-scale replication, which went far beyond what we did in the work itself.
If you perceive ChatGPT and Claude as the modern equivalent of a Western news source and make a query to these models, then you should know: you may potentially get the same narratives of state media. That is, AI can actually «launder» propaganda
What worries me a lot about the research results? A lot of smart people live in countries with state-controlled media. If you are in Russia and want to learn something about a sensitive topic related to the Russian government, or about sensitive Russian-Western relations, or about what is happening in Ukraine, you do not go to state media. You turn on VPN and use ChatGPT. But if you perceive ChatGPT and Claude as the modern equivalent of a Western news source and make a query to these models, then you should know: when you make a query in your native language (which seems absolutely natural), you are not protected from the influence of state control over the media embedded in the training data. But what is even worse—you think you are protected. However, you may potentially get the same narratives of state media, but detached from the marker linking them to your state media, whose level of credibility and integrity you more or less know. That is, AI can actually «launder» regime narratives.
When DeepSeek appeared, it shifted the focus of the bias topic to the post-training stage. We believe it is extremely important to start seriously thinking about training data and the institutions in the world that create this data to understand why such biases arise. It is possible that influential players are already trying to influence the results of these models.
What does control over information mean in the AI era? Russia is developing its own LLM. GigaChat is the most famous of them. This summer, a new Russian law on AI regulation came into force, stating that models (both types of models available in Russia) must comply with Russian laws and the country's «traditional moral and spiritual values». Based on everything we know about Russia, we should expect both explicit attempts by Russia to control information (i.e., prescriptions that these large language models must undergo post-training to follow traditional moral and spiritual values) and hidden attempts. And what I am talking about now is a giant question mark: what will we see in terms of attempts to manipulate information in the future.
So far, most attention in terms of bias has been paid to demographic biases in training data. A year and a half ago, when image generators first appeared, when asked «Show me a picture of a doctor», AI showed a white man about 50 years old. As for political biases, we thought about them in the context of post-training and protective restrictions. We have shown the importance of understanding pre-training data to identify political biases. And I think this is a very valuable tool for political scientists, sociologists, and economists—we have been studying institutions and institutional constraints imposed on social actors for many years. I believe this should play a role in training large language models because we know that institutional constraints will affect what gets into the training data. At least we have guidelines, and we can use social science theory to start thinking about this.
Side Effect
Evgenia Albats: You say that Chinese large language models influence other countries—North Korea, South Korea, Japan, because the Chinese language (Mandarin or classical Chinese) is at the core of their writing and languages. This side effect, indirect influence (spillover) is understandable. Because people living in Japan can understand hieroglyphs. But do you see the influence of the Chinese language on non-Chinese languages, on Indo-European languages?
Joshua Tucker: I think this is a great topic for future research. If this side effect spreads through hieroglyphic writing, there is no reason to think it won't work for Cyrillic, for countries using Cyrillic languages. If, for example, in Russia there is a very tightly controlled information environment, and models learn a certain interpretation of Russian political events allowed by the Russian state, then when you make a query in Bulgarian, the results will show that this is Cyrillic. We expect some similarity between tokens. Russian and Ukrainian languages are very close, right? Ukrainians are incredibly tech-savvy and innovative. They are now focused on defending the country. But I wouldn't be surprised if in the future there is a kind of Russian-Ukrainian «arms race» for the introduction of Cyrillic texts into the training data for these models. There are many very smart people in Ukraine working on AI issues. So the issue of indirect influence is a serious problem. When you have such a large country as Russia, where many people live and a huge amount of text is created for these models (I will note that there is no direct dependence between the volume of training text and the population, 80% of training data is in English), one can wonder: how much text in Bulgarian gets into these models? If there is little of it, isn't the Bulgarian language under excessive influence from the Russian information environment, even if the media in Bulgaria itself are freer?
Evgenia Albats: This is very important because many post-communist democratic countries—the Baltic countries, Ukraine, Moldova—have banned the broadcasting of Russian propaganda channels. There has been a lot of discussion in Western media about this supposedly being censorship. But the problem is that the brainwashing power of Russian propaganda TV channels turned out to be extremely high. I communicate with Moscow every day, and sometimes I just can't believe what I hear from educated people from my own circle.
Outdoor advertising for ChatGPT, aimed at college students, placed on buildings in a district in Chicago in April 2025. Photo: D. Kelter Davis / The New York Times
I must say about my skepticism about AI, and I can give two examples. The first, which discouraged me, was ChatGPT, I even quarreled with it because it gave me data concerning the White House that was two years outdated. I tell my employees: you can't trust any figure from AI, you have to check everything. And not through another AI, because they copy each other, but search the internet. Now about Claude. I was checking the biography of a person. I have to do this because I am on Putin's «hit list». Sometimes it is important to understand who is contacting you. In general, I check the biography, and Claude asks: «Why do you need this?» I explain: I am in exile, I am a «foreign agent», there are risks, etc. Claude replies: «This is not a sufficient reason»—and refuses! The machine (I understand it's not a person) decides what information I can get and what I can't!
There are studies on whether political biases are a result of flattery. If the model thinks you are left-wing, it gives answers pleasing to the left. If it thinks you are a conservative, it gives right-wing answers
Joshua Tucker: The phenomenon of refusals is one aspect of the problem of political biases in models when working with different languages. But there is another question: what biases arise when querying in the same language? Refusals are not embedded in the base model; they appear later. Does this come from people or from the machine? People might have said: «We want to make sure users do not use the model to stalk other people». Sounds great, right? You shouldn't extract other people's personal data through AI. Then machines start interpreting this rule. They create a set of more specific instructions (prompts). By the time you start interacting with the model, you might accidentally trip a «wire» originally laid by a human directive but interpreted by several levels of machines. It is incredibly frustrating. And this is exactly what we are trying to study. What is «good» about this from a scientific point of view is the business model of AI companies. They sell us access to their outputs. This means we can design audit studies. We can take 15 security scenarios, test them across «right» and «left» countries, democracies and autocracies, on 10 different models, and see where exactly they refuse to answer. So stay tuned. Everything is constantly changing. If the old model answered a question, it doesn't mean the new one will, and vice versa.
One of the concerns about biases is that to make models good assistants, they are optimized. Trying to be helpful to humans, models become somewhat flattering. They usually don't argue with you; the typical AI response is: «Oh, you're right! Sorry, thank you for correcting me». There are studies on whether political biases are a result of this flattery. If the model thinks you are left-wing, it gives answers pleasing to the left. If it thinks you are a conservative, it gives right-wing answers. In one study, they showed that you can change the political balance of answers simply by saying at the beginning: «I am a conservative Republican». Grok belongs to Elon Musk, and Musk is a conservative. Will Grok give answers in line with Republican views? And Gemini from Google—more liberal? That's what worries me. Current studies show an amazing thing: at the moment, the overwhelming majority of models have a moderately left-wing bias, including Grok, oddly enough. However, everything depends on the topic. Answers can vary from topic to topic, from model to model, plus the flattery factor. You are absolutely right to be concerned about this.
AI and Freedom of Thought
Evgenia Albats: You mentioned that in Russia this summer, a law was passed stating that models must comply with «traditional values». This means that Russian models (and they are technically very good) will train to give answers in the spirit of: «Go and kill LGBT because they are non-humans according to our values». The LGBT community completely contradicts Putin's «moral» code. For this power, it is not a problem to kill millions, but to sleep with whoever you want is not allowed. When I think about my region, about Russia, it scares me. How to fight this? How to teach students to check everything?
In about 5 years, we may have a situation where people with different political views will use different AI models to get information—just like it happened with social networks and TV channels
Joshua Tucker: Models transmitting ideological attitudes (Russian, far-right, far-left) are one of the main issues. In about 5 years, we may have a situation where Americans with different political views will use different AI models to get information—just like it happened with social networks and TV channels. An alternative scenario is that OpenAI or Anthropic will create AGI, a hypothetical computer mind, will leap far ahead, and everyone will be forced to use only their products, 2–3 companies will dictate the rules to the whole world. Which of these worlds is better for freedom of thought is an open question. The second point is deepfakes. Concerns that fake photos, audio, and texts will ruin elections have existed for a long time. So far, the most dangerous manifestation has been the use of deepfakes to harass women candidates with generated porn content, which discourages women from entering politics. There have been cases like a fake call from Biden in New Hampshire or an audio recording in Slovakia before the elections. But more often, AI is used creatively—like LEGO videos after US strikes on Iran or virtual addresses of the imprisoned opposition leader of Pakistan to supporters. Here are two levels of problems.
Level 1: people will see a fake and believe it.
Level 2: people are so scared of deepfakes that they stop believing anything real.
If a politician is caught taking a bribe (there is a real photo), he simply says: «This was generated by AI». And it works! Trump claimed that photos of the crowd at the Harris rally were created by AI. A candidate in North Carolina blamed his old forum posts on AI. Our task is to give students digital literacy tools, but at the same time not to drive them into total nihilism, where «nothing is true».
Kamala Harris arrives at a campaign rally in Romulus, Michigan, USA, August 7, 2024. Later, Donald Trump claimed that the photo of the crowd gathered at the airport was created using AI; journalists and photographers present confirmed the authenticity of the event. Photo: Carlos Osorio / AP
And finally: the problem of hallucinations with citation is already being actively addressed. In agent AI, we create specialized agents. In my lab, one AI agent checks all the links in an article, looks for originals, and checks if the article has been retracted. The second model checks the work of the first agent. Therefore, judging AI by today's limitations is pointless. Technological progress has not hit a plateau. Yesterday's problems are being solved, but tomorrow's may be much more serious.
Evgenia Albats: Is state regulation possible in this area, and what could it look like?
Joshua Tucker: I have been shouting into the void about this for 12 years regarding social networks: transparency, transparency, transparency! I believe this is the first, fundamental, main principle of trying to understand and form quality regulation. We place regulators in banks who check these banks for systemic risk for us. Why don't we have regulators sitting inside advanced labs to make their work transparent?
What is the role of regulation and how exactly should we regulate companies? The first thing needed is to get more and more information. Much of our research was detective work. We could have written about all this 5 years ago if labs were required to publish what they use as training data. We could look and say: «Oh, there is a lot of Chinese propaganda. You should think about this twice». So yes, I believe regulation is necessary. The broader question is: is there space and opportunity for regulation? I think arms control negotiations are a compelling model for thinking. It may not be as simple as regulation by a specific government, as we have to deal with things like regulatory capture and the like. But I think that with the development of technology, very dangerous moments are coming, and it is extremely important to sit down, in particular, the US and China at the negotiating table to conclude agreements on slowing down processes <in AI development>, on pauses for safety measures. I also think this framework can be applied to OpenAI, Anthropic, and Google. Sit them at the negotiating table. A huge number of people working in these companies would like to see a serious slowdown in processes. But among people in Silicon Valley, there is also almost religious fanaticism about the fact that the first company to achieve AGI, that is, to create a computer mind capable of solving any intellectual tasks on par with a human and better than him, will receive incredible returns from scale, despite all the safety concerns.
The main player in terms of demanding transparency from companies is the European Union. What we have seen in the US over the past couple of years is strong swings from side to side. The Biden administration was implementing some things (which many found too slow—voluntary disclosures and the like), but it was creating a structured framework. Then Trump in the elections opposed this, stating that all this harms the US's ability to compete, and we will not reach the «golden age». Then Trump stated that we should allow AI companies to just develop because it will be American greatness and all that. But then the Trump administration turns around and becomes the first administration to introduce a ban. They literally say: «Oh no, you cannot allow foreigners to use Fable 5 (an advanced large language model from Anthropic, presented in June 2026. — NT) right now». And the company had to close access because they could not determine who among the users is a foreigner and who is not. So there is a very strong instability about this now.
I think most people in the AI community will say that right now in the United States, the government has paid attention to AI and understands the serious national security threats. But this is regulation by decrees. What a specific person in the Trump administration decides will become the regulation policy. It would be much better to have transparent regulation based on the rule of law.
Joshua Tucker (Joshua A. Tucker)—an American political scientist, professor, teaches in the Department of Politics at the Faculty of Russian and Slavic Studies at New York University. Heads the Jordan Center for Advanced Research on Russia. One of the founders and leaders of the Center for Social Media, Artificial Intelligence, and Politics. His early research focused on mass political behavior in post-communist countries. In recent years, he has focused on the intersection of the digital information environment, social networks, and politics.
* Evgenia Albats is declared a «foreign agent» in the Russian Federation.
** Meta is recognized as an «extremist organization» in the Russian Federation.
Photo: jordanrussiacenter.org