ServiceNow, Hugging Face, and Nvidia have released StarCoder2, the next generation of their open-access and royalty-free large language model (LLM) trained to generate code, in an effort to take on AI-based programming tools including Microsoft-owned GitHub Copilot, Google’s Bard AI, and Amazon CodeWhisperer.
StarCoder2 is in fact a family of three LLMs: a 3-billion-parameter model trained by ServiceNow, a 7-billion-parameter model trained by Hugging Face, and a 15-billion-parameter model built by Nvidia with the help of its NeMo framework.
The three different model sizes will enable enterprises to save on compute costs by using less performant models where resources are an issue.
Developers can use the LLMs for code completion, advanced code summarization, and code snippet retrieval, among other capabilities.
“StarCoder2 advances the potential of future AI-driven coding applications, including text-to-code and text-to-workflow capabilities. With broader, deeper programming training, it provides repository context, enabling accurate, context-aware predictions,” the companies said in a joint statement.
The key point of differentiation between the first and second generation LLMs is the built-in support for more programming languages. While the first generation supported 80 programming languages, the second generation LLMs provide support for up to 619 programming languages.
The foundation of StarCoder2 is a new code dataset called Stack v2, which is more than seven times larger than Stack v1. The companies used new training techniques to help the model deal with languages such as COBOL for which few online resources are available, and to handle mathematics and discussions of program source code. With the ability to understand COBOL, the new LLMs can now go head to head with offerings like IBM’s Watsonx Code Assistant.
Fine-tuning for the enterprise
Enterprises will have the choice to fine-tune the models with their own data using tools such as NeMo or Hugging Face TRL to create custom chatbots or coding assistants.
The first release of StarCoder in May 2023 drew attention as the LLMs were mostly free, unlike models such as Duet AI or CodeWhisperer, and at the same time were trained on licensed data.
ServiceNow and Hugging Face had combined to form the BigCode project, which aimed to create “state‑of‑the‑art AI systems for code in an open and responsible manner with the support of the open‑scientific AI research community.”
The companies had then said that training the LLM on licensed source code resolved legal issues related to generative AI engines that produce unattributed code in response to natural language queries.
GitHub, for example, already faces a class action lawsuit over its Copilot AI coding assistant.
However, the BigCode members said that unlike traditional open‑source software released without use restrictions, StarCoder’s license includes restrictions that apply to modifications of the model and applications using the model, including restrictions on distributing malicious code.
The supporting source code for the models has been made available on BigCode Project’s GitHub page.
While the two smaller models can be downloaded directly from Hugging Face, the 15-billion parameter model is only available on Nvidia’s AI Foundation models catalog.