AI Field Day 6: Broadcom on VMware’s Impact in Private AI
Jay Cuthrell
Chief Product Officer
Hi, I’m Jay Cuthrell and I’m a delegate for Tech Field Day this week for AI Field Day 6 in Silicon Valley on January 29–January 30, 2025. I’m sharing my live blogging notes here and will revisit with additional edits, so bookmark this blog post and subscribe to the NexusTek newsletter for future insights.
Fun fact:
According to the AI Field Day 6 delegates, most people don’t say “let’s delve” into a topic. So, it’s time to delve into a discussion of Private AI. 🤓
In the days of private cloud, VMware was arguably the defacto standard in Enterprise IT. Setting aside long-lived initiatives like OpenStack and CNCF, VMware by Broadcom is expanding their story to the notion of private AI for the Enterprise.
During AI Field Day 6, Tasha Drew shared three key reasons their customers are choosing VMware by Broadcom for their private AI outcomes. Notably, much of the content shared was a actually from outside of Broadcom — and came directly from their customers.
Customers see the need for private AI as a response to the concerns of underutilized GPUs, cloud costs, and model governance gaps. In other words, the same drivers for the x86, overbuying server hardware, and policy consistency have now arrived in the cloud and AI consumption patterns for the modern Enterprise.
Note:
Acronyms used included VMware Cloud Foundation (VCF) as well as VMware Private AI Foundation with NVIDIA (vPAIF-N). Additional acronyms like NSX-V and NSX-T are simply NSX now (i.e. Nicira which become a bacronym Network Software-defined eXtension). VMware Data Services Manager (VDSM = DBaaS) as part of VCF was included in the vector database portions of the discussion. VMware Distributed Resource Scheduler (DRS) discussions included how VMware has extended these capabilities for GPU environments.
UC Berkeley Sky Computing Lab
UC Berkeley Sky Computing Lab GPUs
Data shared during the presentation came from the UC Berkeley Sky Computing Lab. Essentially, dedicating GPUs and prematurely deploying to a public cloud without strong FinOps is a similar pattern of inefficiency as was the notion of dedicated bare metal servers of decades past.
AI Workstation
vPAIF-N Overview
Alex Fanous went into the journey of a customer of vPAIF-N. The specific example shared included the concept of senior leadership wanting an AI innovation to hit their MBO (aka the “I delivered something AI bonus plan”) and the example of the project to get a ChatBot launched involving RAG and Open WebUI to illustrate the iterative development experience for an AI analyst / developer.
vPAIF-N and Open WebUI
While business tradeoff discussion for unit costing was not covered, the example of providing data for a more informed opinion on what number of billions of parameters for a model are fit for purpose. Conceptually, the iterative and scale test is to find both performance related quantitative issues as well as product experiences overall in the qualitative performance.
RAG Example Scenario build 1
RAG Example Scenario build 3
RAG Example Scenario build 2
RAG Example Scenario build 4
The other example provided was a customer that deployed their own bespoke (expert user) RAG with a highly skill AI application development team that is now using vPAIF-N so that the bespoke solution can be be broken down into more repeatable, reusable, and readily exportable (non-expert user) applications for other groups within an organization that do not have skillsets consistent with the original AI application development team.
Tasha Drew returned to share an overview of governance and a sample agentic workflow. In this case, the LLM is the primary interface for the end user but can also access external tools from multiple systems in a programatic manner.
For example, having a web crawl agent search the Internet for customer quotes on AI topics, then look through the company sales deck collection to ask about enablement information, then also reach out through a Salesforce agent (Einstein) to look at recent customer touchpoint to set context for what might be the best conversation to have with that customer.
vPAIF-N for Agentic use cases
One of the great thing about standards and tools is that there are so many to choose from. However, to operationalize and anticipate drift, the attestation of the software bill of materials has to be brought into consideration to be as shift left as possible.
Said another way, what Bitnami did for open-source ready to deploy VMs, VMware is going to apply to model governance with vPAIF-N for Agentic AI design.
vPAIF-N for Agentic AI Model Governance
During the demo, giskard was used to Red Team the model being downloaded for consideration. The takeaway is that no model performs perfectly and each organization must determine what risk is acceptable for the use case and the business. The next step in the demo was to start a Triton inference server using the version saved in Habor with the implication that this would be basic elements of a CI/CD pipeline.
giskard example
About the Author
Jay Cuthrell
Chief Product Officer, NexusTek
Jay Cuthrell is a seasoned technology executive with extensive experience in driving innovation in IT, hybrid cloud, and multicloud solutions. As Chief Product Officer at NexusTek, he leads efforts in product strategy and marketing, building on a career that includes key leadership roles at IBM, Dell Technologies, and Faction, where he advanced AI/ML, platform engineering, and enterprise data services.
Share On Social
LinkedIn
Twitter
Facebook