BAI AI

Cost comparison

What could you save by running AI on infrastructure you control?

Compare servers you already own, new Intel Xeon, AMD EPYC or NVIDIA hardware, and rented cloud servers. Every comparison starts from the capacity we measured on real servers, tested 5–6 November 2026.

Open the calculator See the benchmarks

Four starting points

Which situation are you in?

Four common starting points. Each uses measured capacity, never a generic percentage.

We already have Intel or AMD servers

  • Deploy on CPU infrastructure you already run, instead of buying specialised GPU hardware first.
  • Avoid or defer GPU procurement until your workload justifies it.
  • Reduce dependence on hosted AI APIs and per-token bills.
  • Keep documents, questions and answers inside your own infrastructure.
  • Predictable infrastructure and maintenance costs.

Measured: a 44-core Intel Xeon and a 45-core AMD EPYC each served 50 staff in our simulated working day with no failures and a typical peak answer of about 30 seconds. They performed almost the same.

We want to buy new servers

  • Compare new Intel Xeon, AMD EPYC and NVIDIA GPU servers on measured capacity and cost per answer.
  • Bring your hardware quotes to the calculator to compare acquisition cost, running cost and payback.
  • A CPU server isn’t automatically cheaper per answer: it answers fewer questions a minute.

Measured: the L4 server answered about 17 questions a minute against about 6.5 on the Xeon and EPYC servers. At cloud list prices that is about $2.20 per 1,000 answers at full load, against about $10.50 to $11 on the CPU servers.

We’re considering NVIDIA GPUs

  • A GPU makes economic sense for about 50 to 200 staff using BAI daily, or wherever answers must start within seconds.
  • It’s an option for workloads that justify it, not an entry requirement: BAI runs fully on CPUs.
  • Ingestion runs on the CPU, so a GPU doesn’t speed up document migration.
  • One GPU is a single point of failure.

Measured: one NVIDIA L4 started answering in 4.3 s for one user and served a simulated day of 200 staff with no failures (typical peak answer 7.5 s). Its estimated ceiling is about 270 staff on our assumed load.

We’re considering cloud deployment

  • Renting is useful for pilots and capacity tests before buying hardware.
  • Rental cost isn’t the cost of owning a server: ownership adds hardware, power and hosting, and removes the hourly bill.
  • In a public cloud, documents leave your network. On your own servers, BAI runs fully offline.

Measured capacity and list prices for the four tested cloud servers are in the table below. Monthly and annual figures are modelled at 730 hours a month.

Cloud rental at a glance

Cloud rental prices and measured capacity of the four tested servers
ServerRental per hourAnswers per minute at full loadRental per 1,000 answers at full loadMonthly, 24/7 (modelled)Annual, 24/7 (modelled)On-premise and offline
Intel XeonGoogle Cloud c3-standard-88$4.44About 6.5About $11$3,241$38,894Yes
AMD EPYCGoogle Cloud c3d-standard-90$4.09About 6.5About $10.50$2,986$35,828Yes
ArmGoogle Cloud c4a-standard-32$1.44About 6.5About $3.70$1,051$12,614No, cloud only
NVIDIA GPUGoogle Cloud g2-standard-32$2.23About 17About $2.20$1,628$19,535Yes

Rental prices are list prices for comparison only. They are not the cost of owning a server, and they exclude storage, network and support. Monthly figures assume 730 hours. The Xeon price is Google’s list price; the GPU, EPYC and Arm prices are third-party list prices.

Compare deployment costs

Enter your workload and costs. Results combine our measured servers with your own figures.

Your workload
Our simulated day assumed 10 questions per staff member.
Our simulated day assumed a peak at 3× the average rate.
Your infrastructure
Your costs
Your estimate. We don’t assume one.
Questions per month11,000
Peak questions per minute3.1
Cloud hours per month730
  • Measured in our benchmarks
  • Your input
  • Modelled from the two
On-premise · offline

Your existing server

Responsive at your peak

Compared with the measured Intel Xeon server

Nearest measured load at or above yours: 3.1 questions a minute at peak (50 staff). Typical peak answer 28 s, slowest 5% 2.5 min; 0 of 47 failed.

At 4 simultaneous users (continuous load): first word 22 s, complete answer 42 s.

Monthly running cost
Enter your monthly running cost
Annual cost
—
Cost per 1,000 answers at your volume
—
On-premise · offline

New Intel Xeon server

Responsive at your peak

Assumes a server similar to the measured 44-core Intel Xeon

Nearest measured load at or above yours: 3.1 questions a minute at peak (50 staff). Typical peak answer 28 s, slowest 5% 2.5 min; 0 of 47 failed.

At 4 simultaneous users (continuous load): first word 22 s, complete answer 42 s.

Monthly running cost
Enter your monthly running cost
Annual cost
—
Cost per 1,000 answers at your volume
—
Hardware payback
Enter a hardware quote
On-premise · offline

New server with one NVIDIA L4

Responsive at your peak

Assumes performance similar to the measured cloud L4 server

Nearest measured load at or above yours: 6.2 questions a minute at peak (100 staff). Typical peak answer 5.6 s, slowest 5% 11 s; 0 of 84 failed.

At 4 simultaneous users (continuous load): first word 13 s, complete answer 13.5 s.

Monthly running cost
Enter your monthly running cost
Annual cost
—
Cost per 1,000 answers at your volume
—
Hardware payback
Enter a hardware quote
Cloud rental · data leaves your network

Cloud Intel Xeon server

Responsive at your peak

Google Cloud c3-standard-88, $4.44/hour (Google list price)

Nearest measured load at or above yours: 3.1 questions a minute at peak (50 staff). Typical peak answer 28 s, slowest 5% 2.5 min; 0 of 47 failed.

At 4 simultaneous users (continuous load): first word 22 s, complete answer 42 s.

Monthly rental
$3,241 (modelled)
Annual cost
$38,894 (modelled)
Cost per 1,000 answers at your volume
$295 (modelled)
At full load
about $11 per 1,000 answers (measured)
Cloud rental · data leaves your network

Cloud AMD EPYC server

Responsive at your peak

Google Cloud c3d-standard-90, $4.09/hour (approximate third-party list price)

Nearest measured load at or above yours: 3.1 questions a minute at peak (50 staff). Typical peak answer 33 s, slowest 5% 2.2 min; 0 of 47 failed.

At 4 simultaneous users (continuous load): first word 24 s, complete answer 38 s.

Monthly rental
$2,986 (modelled)
Annual cost
$35,828 (modelled)
Cost per 1,000 answers at your volume
$271 (modelled)
At full load
about $10.50 per 1,000 answers (measured)
Cloud rental · data leaves your network

Cloud Arm server

Slow at your peak

Google Cloud c4a-standard-32, $1.44/hour (approximate third-party list price)

Nearest measured load at or above yours: 6.2 questions a minute at peak (100 staff). Typical peak answer 3.3 min, slowest 5% 7.5 min; 0 of 84 failed.

At 4 simultaneous users (continuous load): first word 17 s, complete answer 42 s.

Monthly rental
$1,051 (modelled)
Annual cost
$12,614 (modelled)
Cost per 1,000 answers at your volume
$95.56 (modelled)
At full load
about $3.70 per 1,000 answers (measured)
Cloud rental · data leaves your network

Cloud NVIDIA GPU server

Responsive at your peak

Google Cloud g2-standard-32, $2.23/hour (approximate third-party list price)

Nearest measured load at or above yours: 6.2 questions a minute at peak (100 staff). Typical peak answer 5.6 s, slowest 5% 11 s; 0 of 84 failed.

At 4 simultaneous users (continuous load): first word 13 s, complete answer 13.5 s.

Monthly rental
$1,628 (modelled)
Annual cost
$19,535 (modelled)
Cost per 1,000 answers at your volume
$148 (modelled)
At full load
about $2.20 per 1,000 answers (measured)

BAI’s own setup and maintenance fees aren’t included. Cloud costs are rental at list price and exclude storage, network and support; they are not the cost of owning a server. Peak-load fit uses our simulated working day. For figures on your own hardware and usage, benchmark your infrastructure with us.

What could BAI achieve on your infrastructure?

Share your server specifications and a representative workload, and we’ll measure it on your hardware.

Benchmark Your Infrastructure