AI Masters from NVIDIA Win the KDD Cup 2024 Data Science Competition. With their ground-breaking AI technologies, Team NVIDIA wins the annual competition in every track. At the Amazon KDD Cup 2024, Team NVIDIA has emerged victorious, taking first place on Friday in each of the five competition tracks.
NVIDIA team
The group of NVIDIANs Ahmet Erdem, Benedikt Schifferer, Chris Deotte, Gilberto Titericz, Ivan Sorokin, and Simon Jegou proved their mastery in generative AI by taking first place in a number of competitions, including text creation, multiple-choice questions, retrieval, ranking, and name entity recognition.
The “Multi-Task Online Shopping Challenge for LLMs” was the competition’s theme, and contestants had to use limited datasets to tackle a variety of problems.
“An emerging pattern in large-scale machine learning competitions is the absence of training data,” stated Deotte, a senior data scientist at NVIDIA. “96 sample questions are insufficient to train a model. Therefore, we generated 500,000 questions independently.”
According to Deotte, the NVIDIA team created a wide range of questions by altering pre-existing e-commerce information, authoring some of the questions themselves, and utilising a big language model to produce others.
“It was easy to use existing frameworks to fine-tune a language model once we had our questions,” he stated.
To prevent competitors from using answers that were already known, the exam questions were hidden by the competition organizers. By encouraging models that generalize well to any e-commerce-related query, this strategy demonstrates the model’s efficacious handling of real-world scenarios.
Notwithstanding these limitations, Team NVIDIA’s creative strategy beat all rivals by utilising QLoRA, a method for fine-tuning models with datasets, and Qwen2-72B, a recently published LLM with 72 billion parameters that was optimized on eight NVIDIA A100 Tensor Core GPUs.
Concerning the KDD Cup 2024
ACM SIGKDD organizes the KDD Cup, a famous annual event that supports research and development in the field.
Amazon arranged a big language model simulation of online shopping this year to make it more enjoyable and intuitive. To assess participant models, organizers used the test dataset ShopBench, a benchmark that imitates the enormous challenge of online buying with 57 tasks and about 20,000 questions extracted from actual Amazon purchase data.
The ShopBench benchmark included a fifth “all-in-one” challenge in addition to focusing on four essential shopping skills:
Shopping Concept Interpretation: Interpreting intricate shopping concepts and jargon.
Making decisions based on shopping expertise is known as shopping knowledge reasoning.
Understanding dynamic consumer behavior is necessary for user behavior alignment.
The ability to shop in multiple languages.
Overall: Completing all assignment from the earlier tracks in a single, cohesive solution.
NVIDIA’s Victorious Approach
The winning approach from NVIDIA was to build a different model for every track.
For around a day, the researchers used eight NVIDIA A100 Tensor Core GPUs to optimize the recently released Qwen2-72B model. The time needed for fine-tuning was greatly decreased by the GPUs’ quick and effective processing.
First, utilising Llama 3 70B hosted on build.nvidia.com, the team synthesized more data and created training datasets based on the examples supplied.
They then used the data produced in the first step to train QLoRA (Quantized Low-Rank Adaptation). A smaller subset of the model’s weights are altered by QLoRA, enabling effective training and fine-tuning.
Next, using AWQ 4-bit quantization, the model was made smaller and more suitable for running on a system with a smaller hard drive and less RAM. It was then used to predict the test datasets on four NVIDIA T4 Tensor Core GPUs within the time limits using the vLLM inference package.
For the second year in a row, NVIDIA won all of the individual races as well as the competition as a whole thanks to this strategy.
The group intends to discuss their findings at KDD Cup 2024 in Barcelona and submit a thorough paper on its solution next month.
The Kaggle Grandmasters of NVIDIA (KGMON) is the name of the NVIDIA team that won the KDD Cup 2024 Data Science Competition. 450 teams competed in this prestigious contest, and the NVIDIA team won first place for their outstanding use of large language models (LLMs) in the development of a shopping assistant. Using transfer learning across several languages with limited datasets and predicting product purchases based on browsing data without brand names were among the difficult tasks in the competition.
The NVIDIA team expedited their data science workflows on GPUs by leveraging cutting-edge methods and resources like their own Merlin framework and RAPIDS libraries. Their success was largely due to their creative hybrid ranking/classifier approach, which demonstrated their proficiency with large-scale data processing and recommendation systems, as well as their efficient application of transfer learning.
An explanation of the dates for two papers
There will be two deadlines for papers in 2024. Accepted articles will be included in the KDD Cup 24 conference proceedings if they are submitted by February 8, 2024. Accepted papers will be presented at the KDD Cup 2025 Conference and must be submitted by August 8, 2024. The decision to switch to two deadlines annually was made in order to better serve the rapidly expanding field. Simultaneously with this modification, SIGKDD is looking into turning the proceedings into a journal in order to publish papers ahead of schedule.
