The Future of Chips Silicon Volley’s Genius AI Assistance

A study paper outlines how generative AI might help with one of the trickiest engineering tasks: semiconductor design.

The work shows how businesses in highly specialized industries might use internal data to train large language models (LLMs) to create productivity-boosting assistants.

There are not many activities as difficult as semiconductor design. When examined under a microscope, a cutting-edge processor such as the NVIDIA H100 Tensor Core GPU (above) resembles a carefully designed city made of tens of billions of transistors connected by streets 10,000 times thinner than a human hair.

Building one of these digital megacities takes up to two years of coordination between multiple engineering teams.

A few teams design and install a range of incredibly tiny circuits, while others test the results of their labor. Still others define the chip’s general architecture. Specialized techniques, software, and computer languages are needed for every task.

A Vast Perspective for LLMs

“He think big language models will benefit all the processes in the long run,” stated NVIDIA Research director and article lead Mark Ren.

In a keynote address at now International Conference on Computer-Aided Design, which brings together hundreds of engineers annually in the field of electronic design automation, or EDA, Bill Dally, the chief scientist at NVIDIA, revealed the study.

At the San Francisco event, Dally stated, “This effort marks an important first step in applying LLMs to the complex work of designing semiconductors.” “It demonstrates how internal data from highly specialized fields can be used to train effective generative AI models.”

ChipNeMo Panels

The paper describes how NVIDIA engineers trained a proprietary LLM, named ChipNeMo, on the company’s own data for internal use. The LLM was used to produce and optimize software and support human designers.

According to Ren, who has worked in EDA for more than 20 years, engineers intend to use generative AI in every phase of chip design in the long run. This might result in a considerable increase in total productivity.

The research team selected three use cases a chatbot, a code generator, and an analytical tool to begin with after asking NVIDIA engineers about potential applications.

First Use Cases

Thus far, the most well-liked tool is the latter, which automates the laborious chores of keeping up-to-date descriptions of known issues.

During early testing, a prototype chatbot that answers inquiries on GPU architecture and design assisted a large number of engineers in finding technical documents fast.

As seen above, a code generator under development can already produce brief software snippets of 10–20 lines in two specialized languages used by semiconductor designers. It will be integrated with current tools, giving engineers a useful helper for their ongoing designs.

Personalized AI Models Utilizing NVIDIA NeMo

The team’s efforts in compiling its design data and applying it to develop a customized generative AI model a method that may be applied to any sector are the primary subject of this study.

The group began by selecting a foundation model and using NVIDIA NeMo, a framework that is part of the NVIDIA AI Enterprise software platform that allows for the creation, modification, and deployment of generative AI models. The chosen NeMo model has 43 billion parameters, indicating how well it can recognize patterns. More than a trillion tokens words and symbols found in software and text were used to train it.

The model was subsequently improved in two training cycles by the team, using over 24 billion tokens of internal design data in the first round and roughly 130,000 conversation and design samples in the second.

The work is one of several instances of generative AI research and proofs of concept that are just starting to come out of the lab in the semiconductor sector.