Manage the AI-Generated Image Composition Using the 3D Guided Generative AI NVIDIA AI Blueprint.
ComfyUI image blend
Blueprint is a pre-built workflow designed for RTX AI PCs that integrates Blender, ComfyUI, and Black Forest Labs’ FLUX.1-dev as an NVIDIA NIM microservice.
From early examples of models making images of people with too many fingers to currently producing remarkably photorealistic visuals, AI-powered image production has advanced at an astounding rate. Achieving creative control is still a struggle despite such advancements.
Text-based scene creation is now simpler and doesn’t require intricate descriptions, and models are better able to follow instructions. However, using text alone to describe finer elements like composition, camera angles, and item placement is challenging, and making changes is considerably more difficult. Though their setup complexity restricts wider accessibility, advanced workflows employing ControlNets technologies that improve image production by offering greater control over the output provide solutions.
NVIDIA unveiled the NVIDIA AI Blueprint for 3D guided generative AI for RTX PCs at the CES trade show earlier this year in an effort to address these issues and expedite access to cutting-edge AI capabilities. Everything required to begin creating photographs with complete composition control is included in this sample workflow.
Utilize 3D guided generative AI to Manage AI-Generated Pictures
NVIDIA 3D guided generative AI feeds blueprint Create images by fluxing a Blender draft 3D depth map.1-dev, Black Forest Labs, produces desired pictures in response to a user’s query.
The depth map aids the picture model in determining the proper placement of objects. Because they will be transformed to greyscale, this technique has the advantage of not requiring highly detailed items or high-quality textures. Additionally, users can quickly move things and alter camera angles because the scenes are in three dimensions.
ComfyUI, a potent tool that enables artists to chain generative AI models in intriguing ways, powers the blueprint. The ComfyUI Blender plug-in links Blender to ComfyUI. Additionally, by utilising the NVIDIA TensorRT software development kit and optimised formats like as FP4 and FP8, an NVIDIA NIM microservice enables users to install the FLUX.1-dev model and execute it at optimal performance on GeForce RTX GPUs. An NVIDIA GeForce RTX 4080 GPU or more is required to use the AI Blueprint for 3D guided generative AI.
A Prefabricated Base for Workflows Using Generative AI
For sophisticated picture generation, 3D guided generative AI uses Blender, ComfyUI, Blender plug-ins, and FLUX.1-dev NIM microservice and ComfyUI nodes. It also includes an installer and comprehensive setup instructions for AI artists.
With a functional pipeline that can be customized to meet particular requirements, the blueprint provides an organized approach to image generation. A strong foundation that makes the creative process easier to manage and the outcomes more potent is provided by detailed documentation, sample assets, and a preconfigured environment.
The blueprint can serve as a basis for AI developers to expand or create pipelines of a similar nature. It includes documentation, sample data, source code, and a functioning sample to get you started.
Generation in Real Time RTX AI-powered
Utilising the latest performance innovations from the NVIDIA Blackwell architecture, AI Blueprints operate on NVIDIA RTX AI PCs and workstations.
More than quadrupled inference speeds over native PyTorch FP16 are made possible by the FLUX.1-dev NIM microservice, which is part of the architecture for 3D guided generative AI. It is quantised to FP4 precision for Blackwell GPUs and optimised with TensorRT.
TensorRT additionally accelerates the FP8 implementations of the FLUX.1-dev NIM microservice for users running NVIDIA Ada Lovelace generation GPUs. High-performance workflows are now easier to access for quick experimentation and iteration to these enhancements. Additionally, quantisation enables models to run with less VRAM. For example, FP4 reduces model sizes by over two times when compared to FP16.
Use RTX AI to Create and Customise
With more blueprints and services in the works, there are presently ten NIM microservices for RTX that cover use cases ranging from speech AI and machine vision to picture and language generation.
FLUX.1-dev Summary
A group of generative picture AI models called FLUX.1 produce realistic, high-quality images:
- FLUX.1-dev uses straightforward text prompts to create images.
- FLUX.1-Canny-dev guides the output picture structure by combining the text prompt with an image input that has been canny-edged.
- FLUX.1-Depth-dev uses the LiheYoung/Depth-anything-large-hf model to guide the output picture structure by combining the text prompt with an image input that has been processed to create a depth map.