The course “Generative AI with Large Language Models” offered by DeepLearning.AI and AWS on Coursera is designed to prepare data scientists and engineers to become experts in utilizing large language models (LLMs) for generative AI applications. The course covers the entire project lifecycle of a generative AI project, including problem scoping, LLM selection, domain adaptation, model optimization, and integration into business applications. It not only focuses on the practical aspects but also delves into the science behind LLMs and their effectiveness.
The course is divided into three weeks of content, totaling approximately 16 hours of videos, quizzes, labs, and additional readings. Hands-on labs are hosted on the AWS Partner Vocareum platform, providing an AWS environment for direct application of the techniques taught in the course. All the necessary resources for working with LLMs and exploring their effectiveness are provided.
Week 1 of the course introduces the transformer architecture that powers LLMs and explains how these models are trained. It also covers prompt engineering and generative configuration settings to guide model output during inference. The first hands-on lab focuses on constructing and comparing prompts for a generative task, such as summarizing conversations between multiple individuals.
Week 2 explores fine-tuning, which involves adapting pre-trained models to specific tasks and datasets. The concept of parameter-efficient fine-tuning (PEFT) is introduced, allowing fine-tuning of large models using limited computational resources. Evaluation metrics for comparing LLM performance are also discussed. The second lab provides hands-on experience with parameter-efficient fine-tuning and compares the results to prompt engineering from the previous lab.
In Week 3, the course covers reinforcement learning from human feedback (RLHF) to make LLM responses more humanlike and aligned with human preferences. Techniques such as retrieval-augmented generation (RAG) and libraries like LangChain, which enable LLM integration with custom data sources and APIs, are also explored. The final lab involves hands-on experience with RLHF, where the LLM is fine-tuned using a reward model and the proximal policy optimization (PPO) reinforcement learning algorithm to enhance the model’s harmlessness. The impact of RLHF on aligning LLMs with human values and preferences is evaluated.
Overall, this course aims to equip learners with the necessary knowledge and skills to leverage generative AI and LLMs for real-world applications across various industries.
