Profile-Guided Tiering in the V8 JavaScript Engine
A key component of online applications, JavaScript depends on effective runtime compilation to function at its best. Many browsers are powered by the V8 engine, which has historically managed its four compilation tiers Ignition, Sparkplug, Maglev, and Turbofan using a one-size-fits-all strategy.
But when it comes to optimizing efficiency across a variety of JavaScript functions, this outdated approach frequently fails. Intel has created Profile-Guided Tiering, a ground-breaking technique that customizes the tiering process to the unique requirements of each function, in partnership with Google’s V8 team. By increasing the V8 engine’s efficiency, this tactic results in web apps that are quicker, more responsive, and offer a more seamless user experience.
What is Tiering?
There is frequently a trade-off between compilation time and code quality since JavaScript functions are compiled at runtime. Ignition, Sparkplug, Maglev, and Turbofan are the four collection tiers available in V8 at the moment. Javascript functions start in the first layer and move through the later levels when they are called. The aforementioned figure illustrates how the compilation time increases from left to right tier while the compiled code executes more quickly. JavaScript code has a low startup time and a high peak performance to the four compilation layers. Tiering is the heuristic of switching between various tiers.
Tiering Up
Tiering up is the process of moving from the left to the right tiers. When a JavaScript function’s invocation count hits a certain level, tiering up is initiated. For instance, Ignition compiles a Javascript code first at startup. A function will automatically tie up from Ignition to Sparkplug after eight calls.
While tiering up from Ignition to Sparkplug does not necessitate gathering input, tiering up from Sparkplug to Maglev and from Maglev to Turbofan must. Because JavaScript is a dynamically typed language, it generates better code by using runtime feedback (such object types). Intel refer to this as speculative optimisation.
From the first tier to the last tier, the complete process can be summed up as follows:
| Tiering | Invocation count required | Feedback collection required |
| Ignition -> Sparkplug | 8 | No |
| Sparkplug -> Maglev | 500 | Yes |
| Maglev -> Turbofan | 6000 | Yes |
Runtime compilation is required for tiering up, which is costly because:
- Certain tasks must be completed in the main thread, which halts execution even though the primary compilation jobs are carried out in parallel in the background thread.
- A data race occurs when the main thread’s execution and the background thread’s concurrent compilation processes access the same data at the same time.
- Memory is used by both compilation and compiled code.
Only when the new tier code is used enough is it worthwhile to tier up.
Tiering Down (Deoptimization)
The V8 engine’s optimised tiers are based on forecasts derived from gathered feedback. Tiering down, also called deoptimization, is the process by which the engine moves from an optimised tier to an unoptimised one when these forecasts fall short as a result of fresh feedback.
Deoptimization is expensive due to the following:
- Switching between slower, less optimised code execution and maybe faster, optimised code execution before optimising once again.
- Executing context switching and on-stack replacement.
Avoiding optimising functions that may shortly be deoptimized is essential to reducing these expenses.
Profile-Guided Tiering
Performance depends on the heuristic used to determine when to tier up because both tiering up and tiering down are costly. While tiering up too late will postpone the advantages of optimised code, tiering up too early might raise the likelihood of tiering down (deoptimization) and waste CPU resources.
Decisions about legacy tiering were based on the number of runtime invocations of the routines. On the other hand,it can forecast whether a function is hot or will soon be scaled down by analysing profile data. Intel developed the new Profile-Guided Tiering technique for V8 using this information. Intel examine each tiering up and tiering down during the initial run (profiling) and save the results in a cache. It can precisely modify the tiering in later runs based on the cached decisions.
Performance is improved by profile-guided tiering because:
- Cross-context: The knowledge gained from earlier runs can be used to enhance performance in subsequent runs.
- Case-by-case: Based on the profiling, it can designate distinct tiering strategies for certain functions.
Google feedback indicates that this enhances the user experience when navigating a website in the real world in addition to improving benchmark results like Speedometer 3. Early Tiering Up and Delay Tiering Up rules are currently part of Profile-Guided Tiering.
Early Tiering Up
Since unoptimised tiers are not speculative compilations, they are less likely to be tiered down, so the early-tiering-up policies for them differ from those for optimised tiers.
Early Tiering Up To Unoptimized Tier (Sparkplug)
After eight invocations, legacy ignition works for the V8 engine tier up to Sparkplug. Since it is very likely that the functions will bind up to Sparkplug in the upcoming runs, Intel noted which functions did so in the profiling runs. It increased Speedometer 2 scores by +2% in the next runs by tiering up the identical routines on the initial invocation.
Early Tiering Up To Optimized Tiers (Maglev Or Turbofan)
Functions advance to Turbofan and Maglev only after the gathered feedback stabilises. For instance, 500 invocations are needed to tier from Sparkplug to Maglev. If the feedback changes throughout the invocations, the counter will be reset to zero, and an additional 500 invocations are required for tiering up. 6000 invocations are required to tier up to Turbofan.
This adds complexity to the early tiering method. Intel approach is predicated on the idea that a function most likely employs stable types of objects if its gathered feedback stabilises after a few invocations and no deoptimization occurs afterwards. These functions can be tier up early in the following runs after being recorded in the profiling runs. In particular,it will tier up a function to Maglev earlier in the ensuing runs if it was assembled by Maglev but not by Turbofan.
Delay Tiering Up
It can presume that a function’s feedback is unreliable if it gets deoptimized shortly after being tiered up to Maglev. This indicates that before tiering up, the functions require more invocations. These functions are noted during the profiling runs, and in the ensuing runs, Intel provide them delayed tiering strategies. By doing this, deoptimizations can be avoided and performance can be enhanced.
Performance Estimation
Profile-Guided Tiering was implemented in a number of patches, and it independently assessed the performance impact of each patch. Together, they raise Speedometer 3’s score on the Intel Core Ultra Series 2 processor (Lunar Lake) by 5.2%. The table below displays the specifics.
| Optimization | Improvement to Speedometer 3 |
| Early tiering to Sparkplug | +2% |
| Early tiering to Maglev/Turbofan | +2.4% |
| Delay tiering to Maglev | +0.8% |
| Total | +5.2% |
Future Work
Additional Profile-Guided Tiering optimizations for V8 are being developed by Intel’s Web Platform team. In order to facilitate inter-process Profile-Guided Tiering, Intel present work saves the profiling data to a disc cache (in addition to the existing in-memory cache). The Google V8 team, with whom Intel are working closely, has evaluated and approved a number of Profile-Guided Tiering optimizations because they are straightforward, clear, and offer notable performance gains.
It encourage developers to investigate how these optimization strategies might be used for other script programming languages, such Python, which also make use of numerous runtime compilation tiers, as Intel continue to improve Profile-Guided Tiering in the V8 engine. Your cooperation and insights are really helpful in increasing the efficacy and reach of these tactics.
