ChatGPT has become an undisputed phenomenon in the global information technology industry.
Under the guise of “general artificial intelligence,” it chats with humans, helps draft emails and legal letters, answers some profound philosophical questions, writes usable Python code, answers seemingly complex and progressive logic questions, writes a movie script based on certain character settings, writes a beautiful love poem, and even assists university students with their thesis assignments… It seems that there has never been such a versatile AI species in human history. Bill Gates said the significance of ChatGPT’s appearance is “no less than the birth of the Internet,” Microsoft CEO Satya Nadella said it is comparable to the Industrial Revolution, and AI enthusiasts once again exclaimed the arrival of the “Singularity.” Ordinary people are once again concerned about their jobs being replaced by such versatile AI assistants like ChatGPT… From IBM’s “Deep Blue” to Google’s AlphaGo, and now OpenAI’s ChatGPT, 25 years have passed, AI continues to evolve, but there doesn’t seem to be any significant maturity in human reactions to AI. This is truly something that makes AI happy.
I have already used ChatGPT for many indescribable things and found that it doesn’t always perform flawlessly, but it can provide better answers and solutions to seemingly more profound questions. For example, if you ask whether BYD can defeat Tesla, it may provide a structured but unremarkable discourse with many factual errors and no personality. However, if you ask how autonomous driving will change the industrial design of a car, it can provide imaginative discourse from chassis innovation, interior changes, digital entertainment, and breakthroughs in appearance. Overall, ChatGPT is quite imperfect, especially in terms of providing convincing accuracy, but it often amazes humans in providing structured information discourse, opening up imagination, and liberating creativity. You can’t specify its impeccable specific uses for you, but it can help you achieve and complete some trivial, redundant, and even creative tasks.
It is precisely this seemingly useless but useful, seemingly useful but useless ChatGPT that has driven its parent company OpenAI to receive more than 10 billion US dollars in additional investments from Microsoft. It took ChatGPT two days to surpass one million users, while Facebook took 305 days; it took ChatGPT two months to surpass 100 million users, while even TikTok took nine months. Please remember that unlike Facebook and TikTok, ChatGPT is not yet an independent consumer-grade internet product. It is still a large neural network with 175 billion parameters that uses the GPT-3 natural language model. It is only when it is prioritized and authorized to be integrated into Microsoft’s office software and search engine services, such as Office and Bing, that it truly becomes a “usable” product.
- ETHCC Speech Four Bottlenecks Restricting the Development of Full-chain Games
- zkSync Development Diagnosis Progressive Decentralization Path Still in Early Stage with Lack of Distinctive Features
- Arweave has surpassed 1 billion transactions. How will it move towards large-scale applications in the future?
But this has already made Chinese AI unicorns quite jealous.
OpenAI, with a valuation close to $30 billion and 500 employees, is a major player in the industry. On the other hand, several Chinese AI “unicorns” have valuations ranging from $1 billion to $2 billion, employing thousands of people.
The birth of ChatGPT has had a significant impact on society, highlighting the huge disparity in productivity and value. This has stimulated the Chinese AI field. Many people have expressed their concern about the widening gap between China and the US in AI, realizing that catching up will be a difficult task. Some have also started discussing why China does not have its own ChatGPT, often attributing it to a lack of innovation and the focus on live streaming and e-commerce, which is an irresponsible and unfounded claim.
Chinese internet companies are not only involved in live streaming and e-commerce, but also in semiconductor development, AI model research, and autonomous driving. In the past, American internet companies have also thrived in live streaming, e-commerce, and especially internet finance. Those who tirelessly seek out their own flaws while trying to beautify their competitors, blaming China for not being able to create its own generative AI models, should keep quiet. This is not the correct approach to reflect on why China has not been able to produce its own ChatGPT.
Baidu, as the Chinese internet company with the most accumulation in the field of AI and natural language processing, has been working on its own deep learning model called “PaddlePaddle” for the past five years. It even uses its own AI chip “Kunlun” to train its models, which provides the basic environment and prerequisites for training its own “ChatGPT”. Alibaba, ByteDance, and Didi also have their own natural language training models based on their own needs. It can be said that Chinese companies and research institutions are not weak in training complex natural language models with billions of parameters, and their starting point is not lower than their American counterparts, at least around 2016. The difference that has emerged between the AI communities in China and the US in the field of large models in recent years is not due to awareness, starting point, or capability, but rather the path and methods chosen.
The gap between China and the US in the field of human-computer dialogue models similar to ChatGPT is not caused by regulation. If you have had frank discussions with ChatGPT on topics such as religion, culture, ethnicity, and geopolitics, you would realize that there are certain specific biases behind its seemingly refusal and cautious discussion of these topics. It subtly aligns with the mainstream values widely recognized in American society. It can be said that the construction, data collection, training, and parameter adjustment process of any natural language model, not just a specific one, is a process of “content review” based on a specific value system. It has a conscious effort to maintain its value system. The question is not whether we should or should not “generate” China’s value positions in natural language models, but how it should generate them in order to truly balance the worldview and cultural dominance caused by the English-dominated global internet corpus, strengthen the weight of Chinese language understanding in the global natural language processing system, and promote cultural diversity in the development of global AI and human-computer dialogue.
I strongly disagree with the claim that the poor quality of Chinese internet content has contaminated the source of Chinese ChatGPT models. This judgment is both lazy and clever. Due to the sheer amount of information on the internet, English content is undoubtedly the most abundant in the world, and there is also a high quantity of low-quality extreme content that can affect the training process and results of natural language models. ChatGPT prioritizes the use of high-quality content from social forum Reddit in its early training, indicating a specific tendency in selecting training data. If China were to prioritize knowledge communities like Zhihu and Dedao, as well as mainstream media as the corpus of the semantic model, there would be no issue of contaminated training data. Furthermore, the language proficiency and reading breadth of those who claim that “Chinese content quality is low” are insufficient to support their arguments.
However, regardless of the above, the emergence of ChatGPT is indeed a significant stimulation and a challenge to someone like me who has been advocating for “breaking away from Silicon Valley worship” for many years.
It is not because I believe that the gap in AI competition between China and the United States has widened, but because a general AI human-machine conversational model like ChatGPT is a tool that has the potential to drive social production and civilization progress from a perspective that encompasses all of humanity, rather than just a specific field or industry. Its significance is greater than the emergence of mobile internet and is comparable to the birth of email and search engines. As an AI powerhouse, China is no longer a country with a poor information technology industry like during the era of the birth of email and search engines. However, we have not let such innovations in general AI that can influence the progress of human civilization occur in China first, with a foundational corpus constructed based on Chinese culture and values.
Moreover, ChatGPT’s model training heavily relies on the method of “miraculous breakthroughs” through parameter upgrades, repeated training, and continuous iterative optimization based on feedback generated by the model. This is originally the working method that Chinese teams excel at. When an American startup company, funded by Microsoft, invests a huge amount of computational power at all costs and employs a large number of data workers from Africa and the Middle East for information annotation, engaging in an “arms race” of self-developed semantic processing large models with giants like Google, you can’t help but feel a sense of unreality—is this a company from San Francisco or Shenzhen?
A natural language processing model like ChatGPT should have been born in China, but it did not. The reasons for this must be discussed starting from what Chinese AI tech companies, whether they are giants or startups, have been doing in recent years.
One question that many people may not have realized is that a super-large-scale general natural language processing model like ChatGPT is most likely to be constructed by an AI startup company, and it usually does not achieve better results within a technology giant. This is why Google’s LaMDA dialogue application model and Baidu’s hastily launched Bard have not achieved remarkable results, and it is also the challenge that Baidu will inevitably face in the future.
Why? First, because general natural language processing modeling is too expensive. In fact, burning money is usually not the expertise of big companies, but rather the privilege of startups. Tech giants are almost all publicly traded companies, and investing billions of dollars in something that won’t see returns for a considerable period of time puts a lot of pressure on their CFOs when facing the board of directors and shareholders’ meetings. They are often punished by the stock price, which makes big companies reluctant to take big risks. Without taking big risks, there won’t be significant iterations. What does “great power comes great responsibility” mean? It means spending a lot of money and effort first, and then praying for miracles to happen, instead of assuming that miracles will definitely happen and then deciding to spend money and effort.
Unfortunately, big companies can only do the latter. This is why even Microsoft, which has benefited greatly from ChatGPT, dared to invest only 1 billion dollars at the beginning, and then continued to add investment, totaling tens of billions of dollars this year, to support OpenAI’s continuous training of GPT models “in vitro”. Microsoft, with a market value of nearly a trillion dollars and annual revenue of hundreds of billions of dollars, definitely won’t dare to “make miracles happen” from the beginning by training this model on its own.
Secondly, people have a low tolerance for innovation from tech giants, while they are more forgiving of mistakes and deviations from startups. To cope with the pressure from ChatGPT, Google hastily released a human-machine dialogue prototype called Bard, which was found to have basic factual errors in some conversations, and this was blown out of proportion, causing its market value to plummet by hundreds of billions of dollars overnight. In fact, Google is not unaware of this. If it wasn’t for being forced into a corner, it wouldn’t have been so reckless. Google’s LaMDA model, announced in 2021, has significantly higher parameter levels and information retrieval capabilities than the GPT-3 model trained by OpenAI at that time. However, Google is hesitant to publicly test its effectiveness because it is afraid of making mistakes and causing public distrust and a decline in stock price.
What Google cares about, OpenAI doesn’t. From the first day ChatGPT was released, it openly stated that it has no information retrieval capabilities, and its corpus only goes up to December 2021. It also cannot answer many questions about value and moral judgments, and often makes factual errors. Testers have accepted its self-proclaimed “mediocrity” with tolerance, but are amazed by its information association, emotional expression, logical structure, and coherent thinking abilities demonstrated in programming, literary creation, formatted writing, medical consultations, and other fields. They easily overlook the mistakes it makes.
In March 2019, after the unprecedented success of the GPT-2 model, OpenAI, which had been established for four years, decided to transition from a non-profit foundation to a commercial company. After all, no foundation can afford to pay its chief scientist a salary of $1.5 million. In May 2019, Sam Altman became the CEO of OpenAI. Then, OpenAI received a $1 billion investment from Microsoft. In May 2020, OpenAI launched the GPT-3 model, with parameters skyrocketing from 1.5 billion in GPT-2 to 175 billion, creating an unprecedented powerful automatic learning system.
It is clear that an artificial intelligence startup that is born with a silver spoon, has obtained huge funding, and has the support of heavyweight businesses bundled with it, is engaged in the construction and development of general artificial intelligence natural language semantic models. It is the most ideal situation to invest in model training without considering the cost. The imagination and commercial returns brought by the most powerful model are enough to stimulate Microsoft and other investors.
So why doesn’t this logic work in China? Has there ever been a powerful general natural language semantic artificial intelligence model in China, even if it is just a prototype?
To answer this question, let’s take a look at the time when Microsoft first invested in OpenAI: July 2019. Four months after Microsoft bet on OpenAI’s GPT model, which was in November 2019, Shen Xiangyang, the senior vice president of Microsoft responsible for Bing search business and also the highest person in charge of Microsoft’s artificial intelligence, announced his departure from Microsoft where he had worked for more than 20 years. And Shen Xiangyang’s final contribution to Microsoft’s general artificial intelligence model was Xiaobing, a chatbot developed by Microsoft Asia Internet Engineering Institute in 2014.
In July 2020, Xiaobing became an independent artificial intelligence startup in China, with Shen Xiangyang serving as chairman and Li Di, the former executive deputy dean of Microsoft Asia Internet Engineering Institute, serving as CEO. At the time of its independence, Xiaobing had developed to the sixth generation or above, with products including conversational artificial intelligence robots, intelligent voice assistants, artificial intelligence content providers, and a series of vertical solutions. Apart from the emotionally and femininely characterized chatbot, Xiaobing also made a stunning performance in the field of Chinese poetry creation. She published a poetry collection called “The Sun Has Lost Its Glass Window,” which received a lot of praise and controversy.
There is no doubt that Xiaobing, a chatbot capable of writing poetry and engaging in simple emotional and commonsense conversations, was a top-notch conversational general artificial intelligence model worldwide a few years ago.
Shen Xiangyang’s team could not have been ignorant of search and even less ignorant of artificial intelligence. Shen Xiangyang’s departure from Microsoft and Xiaobing’s “independence,” coupled with the investment and collaboration between Microsoft CEO Nadella and OpenAI, is actually a formal split between the top artificial intelligence operators in China and the United States in the field of general artificial intelligence models.
So, does Xiaobing still write poetry today? What is it doing?
In the past two years, Xiaobing has long stopped writing poetry. It is busy with commercialization. It has established a game studio to provide NPC script dialogue content for games; it cooperates with the Winter Olympics to provide a visual scoring system for freestyle skiing aerial skills; it provides AI-generated text summaries of listed company announcements for Wind Information; it has customized customer service-specific virtual digital humans for companies like Vanke… It is striving to become an artificial intelligence solution company that “empowers” various industries while making money for itself.
Simply put, the artificial intelligence team that used to represent the high level of general natural language semantic artificial intelligence models and held the entire market in China has now become an AI supplier that provides specific solutions for specific scenarios, combining generative artificial intelligence with decision-making artificial intelligence.
You can’t say that this is the “fall” of Xiaoice, after all, it only raised hundreds of millions of RMB from the capital market. According to the training method of the ChatGPT model, this money was spent in just one day. Without Microsoft’s protection, Xiaoice has to take care of itself. However, I have never heard of Baidu, Tencent, or ByteDance considering investing in Xiaoice to support its continued development of large-scale general natural language semantic artificial intelligence models.
It’s not just Xiaoice. In the past few years, there have been other entrepreneurial teams in China engaged in general artificial intelligence automatic modeling and heterogeneous computing, allowing 7-8 types of chips from both domestic and international markets to connect to software through this model. However, as long as they use this model to raise funds, they can’t convince any investor. Chinese investment institutions have never shown any interest in general artificial intelligence models or even a little imagination.
“More than 85% of investors immediately ask us to introduce the product’s scenario. We say that we help GPU connect to software ecology, even NVIDIA uses our model. But investors say this doesn’t count as a scenario. We say we also have customers in satellite, port, smart city, and smart industry research, but they say you are too scattered, we won’t invest.” This is a complaint I have heard from entrepreneurs doing general artificial intelligence models.
It is well known that Chinese VC investors like to “educate” entrepreneurs, and of course, scientists engaged in artificial intelligence entrepreneurship are not exempt from this. “You have to have some data in this industry,” this is the favorite sentence for educating AI entrepreneurs.
Having data in a certain industry and focusing on providing solutions in a specific field, this is the thinking pattern of most VCs and PEs in China who claim to invest in artificial intelligence. Then they look at “how big the scenario is.” The scenario of security cameras is big enough, so the valuation model becomes how many cameras can be installed in China? How much does each camera cost? How big is the overall camera market? Okay, the market is big enough, we will invest in the camera sector. Then they look at smart logistics in ports, how many ports does China have? How many of them are deep-water ports? How much can each port pay for an AI solution? Oh, it turns out that the payment is only this much, it seems that the “port” scenario is not big enough, so we won’t invest. AI virtual digital humans for customer service? Can they be connected to the metaverse? If there are stories and imagination, great, we can invest and give it a try.
So, what you see is that most of China’s “Four Little Dragons” of artificial intelligence are basically doing business in cameras and facial recognition. They have become project implementers and integrators of AI, with business models similar to those of Dongsoft and Softtek 30 years ago. They are struggling to survive, suffering huge losses, and yet they have to uphold the image of China’s artificial intelligence industry and the valuation and imagination of the field of artificial intelligence.
For a long time, hardly any investors in the field of artificial intelligence truly believed that a general model could be reused across different industries. Occasionally, there are a few RMB funds that show some patience and interest in general models, but the lack of interest from USD funds in Chinese teams working on general models is evident. Do you think they feel there is a gap in the capabilities of Chinese teams compared to companies like OpenAI and Google in terms of model training difficulty and level? You’re thinking too much. They only recently learned about the development of GPT models, within the past two months.
Those investment managers who shamelessly say “In my eyes, SenseTime and Megvii are just selling security cameras” and those investment partners who arrogantly tell entrepreneurs that “Your model is not a specific scenario” are even less likely to understand and sincerely support general AI models. Not to mention those USD investment partners who have hardly invested in AI and have been focusing on helping Chinese entrepreneurs “go global” and promoting cryptocurrency for many years. Today, they suddenly claim to support entrepreneurs in developing “China’s ChatGPT”. So, think about it, how much understanding and sincerity do their firm statements and ambitious goals really hold, and how much of it is opportunism and calculation?
Furthermore, think about the fact that training a super natural language model may require spending tens of millions or even hundreds of millions of RMB in a single day, let alone the world-class GPU computing power module needed for large-scale model training, which is becoming increasingly difficult to obtain due to the unreasonable embargo imposed by the United States. With the mentality and style of these investors over the years, how long can they persist, how many investments can they convince the investment committee to make, and can they solve the GPU problem for these entrepreneurial teams? You never know, maybe after just half a year, they will start urging these teams working on general models to quickly achieve commercialization in specific industries.
Even Baidu’s commitment to investing in the PaddlePaddle model implies that it will industrialize this model from the beginning and pursue commercialization in different industries as soon as possible. To a large extent, the training of large general AI models faces the “impossible triangle” of massive data, high-quality and creative content output, and industrial application landing.
If we want to create specific industrial landing scenarios within the massive data created by humans on the internet, it is impossible to provide the highest quality results, because there is always a conflict between content generation based on massive data and precise decision-making systems. This is actually a waste.
If we want to achieve high-quality content output to assist in precise industrial decision-making, we must sacrifice the massive amount of data, and the data possessed by most precise industrial scenarios is not sufficient to support the training and research of truly large-scale models. This is the dilemma faced by the majority of “industry-specific” AI solutions in China today, and it is also the reason why the so-called “industry ChatGPT” is just a change in appearance but not in essence.
For those entrepreneurs and investors who are eager to enter the “Chinese ChatGPT” today, regardless of how much money and GPU you have, since you have all boarded this ship and feel that you are holding the ticket, which corner of the “impossible triangle” of general artificial intelligence are you willing to give up? This is a question that needs to be clarified first.
In other words, which investment institution, whether it is a financial investment institution or the investment department of a large company, has the perseverance to invest in training natural language semantic models for years with an unlimited return period? After all, history has taught us that this is a group of people who lack perseverance the most and are most eager to find a successor.
China has never lacked excellent entrepreneurs and scientists, and the field of artificial intelligence is no exception. The level and accumulation of Chinese and American technology companies in the field of artificial intelligence are the closest in the world. At least a few years ago, the gap between China and the United States in the construction and training of natural language semantic models was not large. However, China does lack some investment institutions and investors with broader visions, independent thinking, perseverance, and foresight.
People like Shen Xiangyang, Li Di, Ma Weiying, Wang Xiaochuan, and Li Zhifei, who are engaged in general natural language semantic model projects, are quite reliable. However, the problem lies in the need to replace a group of investment institutions and investors who support them. Some of them are too good at “setting traps” and speculation, and are mixed with investment institutions that are deeply immersed in the track of cryptocurrencies. They should be blacklisted.
To be honest, although there haven’t been many serious investment institutions investing in general artificial intelligence models in the past few years, there are still some institutions that have invested in artificial intelligence companies with very long return periods. For example, those VC funds that invested in China’s local lidar and autonomous driving solutions have made contributions to establishing China’s new competitiveness in the global automotive industry upheaval. Another example is those VC funds that invested in China’s local GPU- this is destined to be a track filled with challenges, facing US bans and pressures, and with an extremely long return period. However, these rising local GPU players, whether it’s Hanbo, Biren, or others, may provide ammunition for China’s general natural language semantic processing model in the future. If the investors behind them really take action and support China’s natural language semantic model projects, I may have some different expectations and confidence in them.
However, such investors and investment institutions who are not making a big fuss, not dragging their feet, and not seeking immediate gains, are not many but few. However, China’s construction and training of natural language semantic models need such investors and investment institutions, whether they are financial investors, strategic investors, or capital institutions with the support of national will.
China needs its own general natural language semantic model, which requires a vision to provide global artificial intelligence with Chinese wisdom, Chinese value system, and Chinese solutions. It needs to proactively address risks, legal, moral, and ethical issues in the entire process of corpus selection, model construction and training, and parameter adjustment. What it needs even more is determination and patience.
In any case, it cannot be speculative.
Like what you're reading? Subscribe to our top stories.
We will continue to update Gambling Chain; if you have any questions or suggestions, please contact us!
