Can any organization train generative AI models? CAST AI brings it closer

CAST AI specializes in optimizing cloud costs, specifically around Kubernetes. Today it comes with new features within the platform with which it does this. It will now also reduce the costs associated with training AI models. This should make it more realistic for more organizations to seriously get started with generative AI.

Generative AI is very interesting and gets a lot of attention (also here on Techzine). However, we’ve also noted several times, including in our podcast episodes on ChatGPT and GPT-4, that the price tag is very hefty. For ChatGPT, there is an estimate that it costs no less than $700,000 a day to keep it up and running. That is still affordable for an organization such as OpenAI, which is backed by substantial investments from Microsoft, among others. This is not the case for an ‘ordinary’ enterprise organization. Not to mention smaller organizations. To make it more interesting for a wider audience, those prices have to come down. Mind you, we assume that generative AI is also interesting for all organizations. That is debatable, but that is not what this article is about.

Read also: Generative AI: How can GPT-4 change business?

CAST AI platform gets new features

CAST AI has been in the business of optimizing costs since its inception in 2019. This concerns cloud costs and especially costs related to Kubernetes. Due to the volatile nature of containers, these can quickly add up.

Today, however, it is not specifically about Kubernetes, but about training AI models. As a rule, this also takes place in the cloud, whether it is AWS, Google Cloud Platform or Microsoft Azure. The updates that CAST AI officially adds to the platform today should ensure that the necessary optimization takes place in this area as well. The platform automatically scans the three major clouds and searches for the most cost-efficient GPUs. It selects these and also does the provisioning. If a GPU instance is no longer needed, the platform will switch it off. It can also replace a previously selected GPU instance with a cheaper one.

Read also: The jobs most threatened by generative AI like ChatGPT

We also see a few more updates specifically aimed at the AWS cloud. For example, the CAST AI platform optimizes the deployment of Amazon Inferentia machines that you use to run AI models. In addition, it can also deploy Graviton processors, while balancing things like performance and cost is the promise. Finally, the CAST AI platform manages spot instances. We also see the latter more and more when deploying Kubernetes across multiple clouds. The platform selects the optimal configuration for the requirements of a specific model and matches it with the most cost-efficient machines.

How much savings?

CAST AI claims it can cut cloud bills for customers in half overall. That is perhaps what you can expect on average when it comes specifically to training AI models. This will no doubt depend on the availability of the GPUs to be used for it. We can’t really estimate how much leeway there is in this area, spread across AWS, GCP and Azure. CAST AI quotes an anecdote from a customer who saw savings of 76 percent when training AI models within Amazon EKS. So it looks like there’s quite a bit of wiggle room. In any case, it is something to look at if you want to get started with training AI models.

TIP: Sapphire 2023: SAP takes a big hit with AI, what does it mean for business processes?