Delange mentioned that open supply language fashions are bettering quickly and might be higher than OpenAI’s market-leading GPT-4 for some specialised duties. But he famous that lots of the finest open supply fashions have come from exterior the US, saying that 01.AI may very well be positioned to learn from improvements that spring up round its mannequin. “US companies have become a little bit less open and transparent,” he mentioned on the briefing. “But there’s this interesting dynamic with AI where the more a company releases open source, the more the ecosystem develops, and so the stronger they become at building AI.”
Meta’s Llama 2 is a uncommon instance of a high open supply mannequin from a US firm and is the social media large’s problem to OpenAI, Microsoft, Google, and different main tech rivals investing closely in generative AI. Meta selected to launch its AI language mannequin beneath a license that enables business reuse, with some caveats.
Yi-34B and Llama 2 seem to have extra in frequent than simply being main open supply AI fashions. Not lengthy after the Chinese mannequin was launched, some builders seen that 01.AI’s code had beforehand included mentions of Meta’s mannequin that had been later eliminated. Richard Lin, 01.AI’s head of open supply, later mentioned that the corporate would revert the adjustments, and the corporate has credited Llama 2 for a part of the structure for Yi-34B. Like all main language fashions, 01.AI’s is predicated on the “transformer” structure first developed by Google researchers in 2017, and the Chinese firm derived that part from Llama 2. Anita Huang, a spokeswoman for 01.AI, says a authorized skilled consulted by the corporate mentioned that Yi-34B just isn’t topic to Llama 2’s license. Meta didn’t reply to a request for remark.
Whatever the extent to which Yi-34B borrows from Llama 2, the Chinese mannequin features very in another way due to the info it has been fed. “Yi shares Llama’s architecture but its training is completely different—and significantly better,” says Eric Hartford, an AI researcher at Abacus.AI who follows open supply AI initiatives. “They are completely different.”
The reference to Meta’s Llama 2 is an instance of how regardless of Lee’s confidence in China’s AI experience it’s at the moment following America’s lead in generative AI. Jeffrey Ding, an assistant professor at George Washington University who research China’s AI scene, says that though Chinese researchers have launched dozens of huge language fashions, the business as an entire nonetheless lags behind the US.
“Western companies gained a significant advantage in large language model development because they could leverage public releases to test out issues, get user feedback, and build interest around new models,” he says. Ding and others have argued that Chinese AI firms face stronger regulatory and financial headwinds than their US counterparts.
Speaking on the World Economic Forum in Davos final week, Lee argued—maybe hoping the message would journey again house—that the open method could be essential for any nation to take full benefit of AI.
“One of the issues with one or a few companies having all the most power and dominating the models is that it creates tremendous inequality, and not just with people who are less wealthy and less wealthy countries, but also professor researchers, students, entrepreneurs, hobbyists,” Lee mentioned. “If there were not open source, what would they do to learn; because they might be the next creator, inventor, or developer of applications.”
If he’s proper, 01.AI’s expertise—and functions constructed on high of it—will put Chinese expertise on the coronary heart of the subsequent section of the tech business.