Large Language Models (LLMs) have gained significant popularity, with industry giants offering them as Software-as-a-Service (SaaS) solutions. However, deploying AI models on your private cloud, data center, or personal hardware can be a smart choice when considering security, data privacy, and cost-effectiveness.

In this session, we will explore the details of the resources consumed by LLMs. We will begin with a concise introduction to MLOps, providing a foundation for efficient model operations and management. Following this, we will delve into the technical aspects using command line tools to reveal what occurs behind the scenes when prompting LLMs. Finally, we will establish a straightforward observability pipeline to collect GPU usage statistics, ensuring transparency.

Join to gain valuable insights and practical skills for harnessing the power of LLMs in a secure and cost-effective manner.