On-Prem Server for AI Development

The challenge

Cloud GPU costs were prohibitively expensive for extended LLM experimentation and development at a consulting firm.

What we built

We designed and built a custom GPU server cluster capable of running multiple open-source LLMs simultaneously, ensuring privacy and cost-effectiveness for AI development. It brought the cost of training and experimenting with LLMs down to the server’s energy bill — roughly $500 a month, against about $8,000 a month in cloud GPU spend.