Preparing for a new Data Center: Automated Management of a 10’000 node bare-metal fleet in CERN IT

Abstract

CERN IT has consolidated all life-cycle management of its physical server fleet on the Ironic bare-metal API. From the initial registration upon the first boot, over the inventory checking, the burn-in and the benchmarking for acceptance, the provisioning to the end users and the repairs during its service, up to the retirement at the end of the servers’ life, all stages can be managed within this framework. This article will walk you through the server’s life in the CERN data center, and explain how this enables us to handle a fleet of almost 10,000 nodes in an automated and efficient way and to prepare for the new data center which is currently being built. Eventually, we will round up things with the top challenges we faced when moving to this system, like the transparent adoption of already in-production nodes, after-the-fact inventory updates, or scaling issues.