GPUStack v0.6: Distributed vLLM, Model Compatibility Checks, Auto Recovery & 100+ Enhancements for the Best Model Inference Experience
GPUStack v0.6 We’re thrilled to announce the release of our most powerful upda
What is GPUStack?
A GPU cluster manager
for running Large Language Models.
GPUStack allows you to create a unified cluster from any brand of GPUs in Apple MacBooks, Windows PCs, and Linux servers. Administrators can deploy LLMs from popular repositories such as Hugging Face. Developers can then access LLMs just as easily as accessing public LLM services from vendors like OpenAI or Microsoft Azure.
What can GPUStack
help you?
For AI Developers
- Access any supported LLM in GPUStack using API keys
- LLM is ready for use after setting a few parameters
- View your token usage and API calls by LLM and date
For Admins
- Deploy, update and remove LLMs as needed
- Add various GPUs into GPUStack from anywhere
- Gain comprehensive visibility for all resources in GPUStack
- Get detailed reports of token usage and operational status of the LLMs
Integration & Serving
GPUStack aggregates all GPU resources within a cluster. It is designed to support all GPU vendors, including Nvidia, Apple, AMD, Intel, Qualcomm, and others. GPUStack is compatible with a laptops, desktops, workstations, and servers running MacOS, Windows, and Linux.
GPUStack can host any LLMs (on the cloud or on premises) and to make their functions available so that applications can incorporate AI into their systems.
Latest Seal News
Read the latest news from our blog.
GPUStack v0.5: Model Catalog and Image-to-Image Features Unveiled, Comprehensive Optimizations Enhance Product Performance and User Experience
GPUStack v0.5. GPUStack is an open-source GPU cluster manager designed for runni
GPUStack v0.4：Image and Audio models, Inference Engine Version Management and Offline Support
GPUStack is an open-source GPU cluster manager designed for runnin
