RTX PRO 6000 vs L40S for LLM inference
L40S has 48 GB of GDDR6 at 864 GB/s. RTX PRO 6000 Server Edition has 96 GB of GDDR7 at 1,597 GB/s, FP4 and MIG. Specs, model sizes and hourly prices.

NVIDIA L40S and RTX PRO 6000 Blackwell Server Edition are both PCIe cards for data centre servers, and both are sold for AI inference, rendering and video. They are two generations apart. L40S is an Ada Lovelace card with 48 GB of GDDR6. RTX PRO 6000 is a Blackwell card with 96 GB of GDDR7, almost twice the memory bandwidth, FP4 support and MIG. For large language models, memory decides most of the choice.
Which one for whom
- Models up to about 30B parameters, one model per card, lowest price per hour → L40S. 48 GB at 864 GB/s, 350 W, easy to find in clouds. On 13 November 2026 RunPod lists it at $0.79 an hour (Community Cloud) and $1.09 (Secure Cloud).
- 70B-class models, gpt-oss-120b, long context, or several isolated users on one card → RTX PRO 6000. 96 GB at 1,597 GB/s, FP4, up to four MIG instances per card. RunPod lists it at $1.69 and $2.69 an hour. Our server in Sofia has eight of them and rents from $1.80 per GPU-hour, or about $10,510 a month for the whole machine. It is rented into early 2027 and we take reservations for the period after. See the server.
The specifications side by side
| L40S | RTX PRO 6000 Server Edition | |
|---|---|---|
| Architecture | Ada Lovelace | Blackwell |
| GPU memory | 48 GB GDDR6 with ECC | 96 GB GDDR7 with ECC |
| Memory bandwidth | 864 GB/s | 1,597 GB/s |
| FP4 Tensor Core | not listed | 4 PFLOPS (1) |
| FP8 Tensor Core | 733 TFLOPS, 1,466 with sparsity | 2 PFLOPS (1) |
| FP16 / BF16 Tensor Core | 362 TFLOPS, 733 with sparsity | 1 PFLOPS (1) |
| FP32 | 91.6 TFLOPS | 120 TFLOPS |
| Max power | 350 W | up to 600 W, configurable |
| MIG | no | up to 4 instances |
| NVLink | no | not listed |
| PCIe | Gen 4 x16 | Gen 5 |
| Form | dual slot | dual slot air-cooled, or single slot liquid-cooled |
(1) NVIDIA's page for the RTX PRO 6000 Server Edition does not say whether these figures assume sparsity. The L40S page gives both. Compare the compute rows as rough, not exact.
Memory decides which models fit
When a model generates text, its weights must sit in GPU memory, plus a cache for every conversation (the KV cache). As a rough rule, weights take two bytes per parameter in FP16, one in FP8 and about half a byte in FP4. On one card:
- 7B to 14B models: fit on either card in FP16 or FP8, with plenty of room for the cache. Here L40S is the cheaper choice.
- About 30B in FP8 (around 30 GB of weights): fits on one L40S with about 18 GB left for the cache. On RTX PRO 6000 the same model leaves about 66 GB, so many more parallel conversations or a much longer context.
- 70B in FP8 (around 70 GB of weights): does not fit on one L40S; you need two, split over PCIe Gen 4. On one RTX PRO 6000 the weights fit, but the cache is tight. In FP4 a 70B model takes about 35 GB and runs comfortably on one RTX PRO 6000.
- gpt-oss-120b: OpenAI's open model has about 117B parameters with the MoE weights in MXFP4, and its model card says it fits on a single 80 GB GPU. That means one RTX PRO 6000, or at least two L40S cards.
In an eight-card server the totals are 384 GB for L40S against 768 GB for RTX PRO 6000. The largest open models, such as DeepSeek-V3 with 671B parameters, need about 336 GB in FP4 for the weights alone. That leaves almost nothing for the cache on eight L40S cards, and room to spare on eight RTX PRO 6000. For a per-model guide, see Local LLM vs private LLM server.
Bandwidth sets the speed for one user
For each new token, the GPU reads the model's weights again. So for a single conversation, the speed is limited mainly by memory bandwidth, not by compute. RTX PRO 6000 has 1,597 GB/s against 864 GB/s on L40S, about 1.85 times more. With the same model on one card, expect roughly that much faster token generation for one user. That is an estimate from the specifications, not our own measurement; we have not run the two cards side by side.
With many users at once, compute and the room for the cache matter more. Here RTX PRO 6000 is ahead on both, and it adds FP4, which halves the memory per parameter against FP8. NVIDIA lists no FP4 figure for L40S.
Splitting one card between teams
L40S does not support MIG, so a card runs one set of jobs at a time, or shares its time between them. RTX PRO 6000 splits into up to four fully isolated instances, each with its own memory and compute. In a server with eight cards that is up to 32 isolated GPUs: useful when a team wants separate environments for development, testing and production, or a provider wants to serve several small models without them disturbing each other.
Power and the server around the card
L40S draws up to 350 W, RTX PRO 6000 up to 600 W, and NVIDIA lets you set a lower limit. An eight-card L40S server is easier to power and cool. An eight-card RTX PRO 6000 server needs a rack built for it: on our own machine we measured about 6.5 kW from the wall at full GPU load. The measurements and what they mean for colocation are in What an 8x RTX PRO 6000 server really draws.
The more useful number is the price for the work done. Per gigabyte of GPU memory, the two cards cost about the same per hour at RunPod: $0.79 for 48 GB is about 1.6 cents per GB, $1.69 for 96 GB about 1.8 cents. The RTX PRO 6000 gives the memory in one piece, without splitting a model over PCIe, and almost twice the bandwidth per card.
Price: renting by the hour or a whole machine
On 13 November 2026, RunPod's pricing page lists L40S at $0.79 an hour on Community Cloud and $1.09 on Secure Cloud, and RTX PRO 6000 at $1.69 and $2.69. Prices change often and depend on the region and how long you commit.
Hourly cloud cards suit tests and short jobs. For a model that runs all day for a team or a product, a whole machine at a fixed monthly price is often simpler to plan. Our server costs about $10,510 a month for eight RTX PRO 6000 cards, with power, cooling and a fixed address in Sofia, in the EU. You can compare the options on our pricing page or read more about a dedicated GPU server.
Questions
Is RTX PRO 6000 the same as RTX 6000 Ada? No. RTX 6000 Ada is the workstation card of the Ada generation, with 48 GB and the same AD102 chip as L40S. RTX PRO 6000 Blackwell is the next generation, with 96 GB. How the RTX PRO 6000 versions differ is in Max-Q vs Workstation vs Server Edition.
Can two L40S cards replace one RTX PRO 6000? For memory, yes: 2 × 48 GB is 96 GB. But the model is then split over PCIe Gen 4 without NVLink, each card has 864 GB/s, and you pay for two cards. One RTX PRO 6000 keeps the model in one place.
Which one for fine-tuning? Fine-tuning needs much more memory than serving the same model, so the 96 GB card goes further. For larger training jobs, see B300 vs H200 vs RTX PRO 6000.
Sources
- NVIDIA, L40S GPU: 48 GB GDDR6, 864 GB/s, FP8 and FP16 Tensor Core figures with and without sparsity, 350 W, no MIG, no NVLink, PCIe Gen 4 x16, checked 13 November 2026.
- NVIDIA, RTX PRO 6000 Blackwell Server Edition: 96 GB GDDR7, 1,597 GB/s, 4 PFLOPS FP4, 2 PFLOPS FP8, 1 PFLOPS FP16, up to 600 W, up to four MIG instances, PCIe Gen 5, checked 13 November 2026.
- RunPod, GPU pricing: L40S $0.79 and $1.09 an hour, RTX PRO 6000 $1.69 and $2.69 an hour (Community and Secure Cloud), checked 13 November 2026.
- OpenAI, gpt-oss-120b model card: about 117B parameters, MXFP4 MoE weights, fits on a single 80 GB GPU.
- DeepSeek, DeepSeek-V3 model card: 671B total parameters.
- NEOXIS measurements of our 8 × RTX PRO 6000 server, published 20 October 2026.


