What is AI Infrastructure?
What the term means and the journey behind every AI answer, then a look inside an AI factory.
Everything between the power station and the answer
Every reply from an AI is the end of a physical journey. Behind it sit thousands of accelerators working in lockstep, a network that lets them behave as one machine, storage that keeps them fed, and a building that delivers the power and carries away the heat. AI infrastructure is all of it.
- The data centre becomes the computer
- Ordinary computing grows by adding servers that each do their own work. AI grows by making thousands of accelerators act as one, so the network, power and cooling become part of the machine.
- Two jobs with different shapes
- Training builds a model in one long, synchronised effort. Inference answers people’s requests, where response time and the cost of each answer decide what is worth building.
- The slowest part sets the pace
- In a synchronised job every accelerator waits for the slowest. One congested link, overheating chip or power cap, and the whole job slows to match.
- The scarce thing has changed
- The hardest part to get is a building with enough power and cooling to run the chips, and buildings take the longest to deliver.
The AI factory
What is an AI factory?
An AI factory is a system, purpose-built for AI production, that transforms raw data into versatile AI outputs, such as text, images, code, and other forms of intelligence, through automated processes that integrate data pipelines, model training, inference, deployment, ongoing monitoring and continuous improvement, to produce intelligence at scale.
Section 3, Architecture, takes you inside one.