Businesses should check hardware compatibility, power supply capacity, cooling requirements, server space, and software support before upgrading enterprise GPUs.
Most GPU upgrades that go wrong are not caused by faulty graphics cards. The real problems often come from poor rack planning, insufficient power supply, cooling issues, incorrect firmware updates, or unclear equipment ownership.
An enterprise GPU upgrade involves much more than installing new hardware. It requires proper planning, coordination, and change management. This guide outlines five practical stages that IT operations managers should follow to prepare for an upcoming GPU upgrade and ensure a smooth, reliable installation.
Key Takeaways
- A single modern GPU system can make close to what an average rack was designed for (about 10.2 kW for a DGX H100 against an average rack density of roughly 12 kW in 2024), so get a written sign-off on power and cooling before making a date.
- Follow the vendor’s firmware order exactly, never run workloads during an update, and prove the full stack on one pilot host before looking for production
- Cut over in limited batches, run acceptance tests on every host, and keep the old hardware stacked until the new hardware has passed.
- Treat retired cards and servers as assets with a data risk attached: re-stock the inventory, sanitize or destroy the drives with documentation, and move the hardware on to resale or recycling as soon as possible.
An enterprise GPU upgrade is the best planned replacement of data center accelerators (for example, moving from NVIDIA V100 or A100 cards to H100, H200, or L40S class hardware) in production servers, along with the power, cooling, firmware, software, and disposal thing that the swap needs.
It is worth separating two kinds of upgrade, because they carry different risk:
Most real projects are a mix of the two, which is one reason the planning stage requires longer than people need.
If you start with a purchase order and work backward, you will search your constraints one at a time, and usually at the bad moment. The planning stage is where you note down what is actually changing.
A useful plan covers, at a minimum:
This is the stage that is most often seen as a formality, and it is the one that most often denies a project.
Newer accelerators draw a good deal more power than the ones they change, and the difference is no longer something a typical rack absorbs without anyone noticing. The AFCOM State of the Data Center report found that average rack density had reached about 12 kW in 2024, while a single NVIDIA DGX H100 system can draw up to 10.2 kW on its own, according to Data Center Knowledge’s summary of the findings. In plain terms, one new system can use most of the power budget that an average rack was built for.
Before the hardware arrives, check the following:
My own rule of thumb, which is an opinion and not a standard: if the facilities team has not signed off on the power and cooling digits in writing, the upgrade does not have a time yet.
Firmware is where a careful upgrade turns into an unplanned outage, usually because someone updated components out of order or interrupted an update partway through.
Vendors publish the order for a reason. NVIDIA’s DGX H100 firmware update procedure, for example, has the BMC updated first, followed by the motherboard tray and then the GPU tray, and it trays that a high workload among the process “can change the firmware update process and result in an unusable component.” The same guide calls for unplugging all power chain for several minutes at one point so that the new firmware activates. That is not a step you want to discover at 2 a.m. on a production time.
The sequence I would follow:
Once the pilot host is stable, the versions on it become the baseline image for every other server in the project.
If you have done Stages 1 to 3 properly, the cutover itself should be the least interesting section of the project. It is still where most people try to save some time, so a few things are worth talking about.
This is the point that most upgrade plans leave out, and it is where a good project silently loses money and creates risk. Retired accelerators are not scrap. A data center GPU that is two generations old is still helpful hardware to a lot of buyers, and it loses resale value every month that it sits in a storage room.
The data risk is real, although it is a quiet different from what people think. GPU memory is volatile, so the card itself does not keep your training data once it is powered off. The servers those cards came out of do, however: the local NVMe drives in a GPU server often hold datasets, model checkpoints, and cached data. If whole servers are leaving the building, those drives need to be cleared or destroyed to a recognized method such as IEEE 2883-2022, the IEEE standard for sanitizing storage, and you need documentation to prove it.
A sensible retirement process looks like this:
Doing this in-house is important, but most operations teams do not have the time or the buyer relationships to resell accelerators well, which is why so many organizations hand the outgoing hardware to an ITAD provider that specializes in it. One of the suppliers that has built its process around this exact stage is Big Data Supply, an R2v3 and RIOS certified IT asset disposition company that buys used NVIDIA GPUs, adding data center models such as the A100, H100, V100, and L40S. If you want to sell used NVIDIA GPU inventory left over from an upgrade, the process is genuinely simple: you send a list of the cards, receive a quote, and then ship the hardware or arrange a pickup, and payment follows once any data has been renewed and a certificate of data destruction has been issued, with chain-of-custody tracking in between. For an operations manager, that turns a bed of old accelerators from a storage problem and budget for the next refresh, with the paperwork an auditor will ask for.
Whichever route you take, the important part is that Stage 5 has an owner and a date, the same as the other four stages.
Businesses should check hardware compatibility, power supply capacity, cooling requirements, server space, and software support before upgrading enterprise GPUs.
Businesses can schedule upgrades during maintenance windows, back up important data, test new hardware in advance, and keep backup systems ready to avoid major disruptions.
Run performance benchmarks, monitor temperature and power consumption, check driver compatibility, and test workloads after installation to ensure stable and efficient GPU performance.
