Building an IT infrastructure for deep learning and artificial intelligence is daunting. The technologies and practices used in deploying AI workloads are very different from traditional enterprise IT applications. This can force IT practitioners to use new, if not unfamiliar skills, which brings a degree of risk to an IT project.
Generative AI, in particular, is placing new demands on IT teams across nearly every industry. The GPUs required for generative AI are expensive and power-hungry, and you may need many. Aligning storage to keep those data-hungry GPUs fed requires adopting new technologies, such as NVIDIA’s GPUDirect, that enable applications to transfer data from primary storage directly into the GPU’s memory. The software stack looks unlike nearly anything else in enterprise IT. The list goes on and on.
Dell Technologies and NVIDIA are working together to reduce the complexity of building and deploying infrastructure for Generative AI. The two companies announced Project Helix earlier this year at Dell Technologies World, which Dell described as delivering full-stack solutions with technical expertise and pre-built tools based on Dell and NVIDIA infrastructure and software.
Dell and NVIDIA have announced the first concrete elements resulting from Project Helix. The companies are delivering validated designs for inference systems based on NVIDIA accelerators and software, a professional services offering to help enterprises embrace generative AI, and a new Dell Precision workstation for AI development.
Generative AI Validated Designs
Validated designs allow IT organizations to take a recipe-driven approach to building infrastructure. Dell has a long history of enabling rapid technology adoption with validated designs, including Dell validated designs for analytics, HPC, and ORAN, among many others. Now Dell has a set of validated designs for generative AI inference.
Dell and NVIDIA co-engineered the Dell Validated Design for Generative AI, providing a blueprint for building infrastructure for generative AI inference. The validated design is a range of pre-tested, proven configurations based on Dell PowerEdge servers combined with the appropriate NVIDIA accelerators.
Dell’s Validated Design for Generative AI with NVIDIA is available globally through traditional Dell channels. Dell also makes systems based on the validated design available through its APEX as-a-service offering.
Servers & Storage
The blueprint for generative AI provides a choice of server, either the Dell PowerEdge XE8640, PowerEdge XE9680 or PowerEdge R760xa. All of these servers are based on the latest generation Intel Xeon processor. Given AMD’s success in this space, it’s surprising that there doesn’t seem to be an AMD option. The servers in the validated designs support between four and eight NVIDIA Hopper H100 GPUs connected with NVIDIA’s NVLink technology.
The storage options are broader than the server offerings. Dell is supporting its PowerScale filter and ECS object storage. Both systems support NVIDIA’s GPUDirect for increased performance and decreased latency when serving data to the GPUs within the cluster.
Based on NVIDIA’s Enterprise AI Software Stack
While generative AI is hardware intensive, it’s the software stack that makes the difference. Each Dell validated design for generative AI relies heavily on NVIDIA’s enterprise AI stack. This includes the following NVIDIA software elements:
NVIDIA AI Enterprise provides an end-to-end, cloud-native suite of AI and data analytics software.
- Triton Inference Server for standardizing and accelerating AI model deployment and execution in production environments.
- Triton Model Analyzer analyzes AI models to identify potential deployment issues, including latency and sufficiency of the hardware.
- Faster Transformer technology, for optimized language processing.
- NVIDIA NeMo Framework for building, customizing, and deploying generative AI models with billions of parameters.
- Cluster Manager manages the provisioning and operation of AI nodes with an AI cluster.
Dell Precision AI Workstation
Dell’s Precision Workstations have long been popular with data scientists and AI researchers. As part of its recent announcement, Dell announced new Dell Precision AI Workstations.
The new workstations arrive with a mix of models based on AMD Threadripper and Intel Xeon processors and are equipped with up to four NVIDIA RTX 6000 GPUs. The new workstation models will be available through the usual Dell sales channels in August.
Professional Services for AI
Professional Services has always been central to Dell’s overall customer experience. The company offers services across a broad range of spaces, including cloud adoption, APEX, big data, business resiliency, and several digital transformation specialties. Dell has extended its services offerings to now include generative AI.
Dell’s professional services team works directly with its customers across the full lifecycle of an AI implementation. This includes working with the customer to create a generative AI strategy that identifies high-value use cases and following through with full-stack implementation services. Post-deployment, Dell professional services can stay involved to ensure operational efficiency, provide managed services, and even handle staff training.
Analyst’s Take
AI is everywhere. It’s nearly impossible to go a day without hearing about yet another way generative AI will change how enterprises operate. It’s not all hype; generative AI is disruptive. Generative AI will impact business processes and is already changing how enterprises think about digital transformation. More critically, AI is changing how IT practitioners think about infrastructure.
Given the constant bombardment of AI-focused news, it’s easy to forget how new the technology really is. The industry is still figuring out best practices. None of this technology is commodity. Any IT organization with a generative AI project will struggle with the cost and complexity of the solution, and anything that mitigates that is a benefit.
I’ve always been a fan of Dell’s validated designs, and I really like the new options for generative AI. Following Dell and NVIDIA’s blueprints removes much of the risk in building and deploying infrastructure for generative AI. Engaging Dell’s professional services team to help removes even more risk, coming as close as you can get to guaranteeing success. Anything that simplifies life for an IT practitioner, as Dell’s new offerings do, is goodness.
Disclosure: Steve McDowell is an industry analyst, and NAND Research an industry analyst firm, that engages in, or has engaged in, research, analysis, and advisory services with many technology companies, which may include those mentioned in this article. Mr. McDowell does not hold any equity positions with any company mentioned in this article.
Read the full article here
