Benchmarking 0-Setup Quantum Computing Simulators

10 April 2023
6 min read
Hayk Tepanyan
Co-founder & CTO
Header image

What Are Quantum Computing Simulators?

Quantum computing simulators are essential tools designed to emulate the behavior of quantum systems, offering researchers and enthusiasts an accessible platform to explore and develop quantum algorithms without the need for a physical quantum computer. There are many categories of quantum simulators but here we will be focusing on the most popular and common version - the state-vector (SV) simulators. These simulators store the entire quantum state and calculate its evolution, making them universal and ideal emulators of quantum computers.

Quantum Simulator Benchmarks

Two popular hosted quantum simulators are provided by AWS Braket and IBM. Just like the one provided by Bluequbit - they are very easy to use and require 0-setup. Users only need to connect their account with the corresponding python SDK to submit large quantum circuits for simulation - much larger than what can be handled by an average laptop.

Large Circuits

Below is a comparison of 32x32 circuit simulation runtime for these platforms.

As you can see AWS’s simulator is more than twice faster than that of IBM. At the same time  BQ-CPU is faster than AWS Braket 12.7x times, and BQ-GPU - staggering 58x times.

We are using layer-structured random circuits for benchmarking, e.g. we alternate between 1-qubit and 2-qubit gates for each layer. We use "square" circuits, e.g. 24x24 or 32x32, where the first number is the number of qubits and the latter is the depth. The 1-qubit gates are chosen  randomly from this set: [X, Y, Z, H, S, T]. The 2-qubit gates are from: [CNOT,  CZ].

Details on Simulator Platforms

IBM's quantum simulator, primarily intended for educational purposes, is offered FREE of charge. However, this advantage is offset by long queues and increased waiting times for task completion, which may not suit individual researchers in need of a dedicated, expedited simulator for faster experimentation.

On the other hand, AWS Braket provides a paid state vector simulator, boasting larger capacity and faster speeds than IBM's offering. This is a better choice for users who are ready to pay for faster and dedicated simulators. 

In the benchmark above both for AWS and IBM we have run the circuit 4 times and taken the minimum result.

BlueQubit’s quantum computing simulator based on CPUs is also provided FREE of charge while the GPU version is intended for more advanced users who need the best performance.

BlueQubit is hosting Google's open-source qsim simulator, sparing users the hassle of dealing with virtual machines, CPU, GPU and memory resources, as well as installation and configuration of qsim. By making the powerful qsim simulator easily accessible with just a few lines of code, BlueQubit offers an appealing alternative for researchers and developers seeking a fast and convenient quantum computing simulation platform. For the GPU version Bluequbit is using Nvidia’s cuQuantum library alongside qsim.

32 qubits is the largest possible size for IBM’s simulator, however AWS Braket can go up to 34 qubits. Hence here is another comparison for a 34x34 random circuit.

While the BQ-CPU is still ~13x faster, we see that BQ-GPU now is outperforming AWS Braket a staggering 230x times. The reason for this is that BQ-GPU uses twice as many GPU’s for each extra qubit after 32. This means we should have expected a 4x extra speedup from 32 qubits to 34 qubits.

Scaling with Qubits

To get a better holistic picture it’s worth looking at simulation runtimes for more qubit sizes. So we take random circuits of the form 23x23, 24x24, .... , 35x35 for this benchmark.

We take into account the empirical observation that statevector simulators’ runtime should grow exponentially with the qubit size and linearly with the number of gates. Thus we have a log-scale for the Y axis. We also show the runtime per gate thus “normalizing” for the number of gates.

All 4 quantum computing simulators follow the 'exponential in qubits and linear in gates' rule. The reason for BQ-GPU being flat after 32 qubits is mentioned above, e.g. using twice as many GPUs. This makes the simulation more costly - but at the same time much faster.

This plot proves again that BlueQubit's GPU simulator is in a totally different league - as one would expect when comparing CPU vs GPU performance.

Deep Circuits and Cost Benchmarks

Aside from benchmarking square circuits, it’s also useful to see how different quantum computing simulators perform on deeper circuits. One reason why we might expect larger speedups for deeper circuits is the following: 

There is a fixed cost for allocating the memory in a statevector simulator. For high-qubit, shallow circuits this cost dominates the runtime. For deeper circuits the actual simulation time per gate becomes the dominant factor and unlike the memory allocation - that is where the speedup of GPU simulators lie. 

We saw earlier that a BQ-GPU simulator is 230x faster than AWS Braket’s SV1 on a 34x34 circuit. We have tried a 34x200 random circuit as well and the difference jumped to 560x.

Below is a cost comparison for that circuit.

It’s also worth mentioning that if speed is not of most importance one could use BQ-CPU (that’s provided for FREE), and it would still be ~12.7x faster than AWS Braket SV1. More info on BlueQubit supported devices and their pricing can be found on our platform.

Why are Quantum Simulators Important?

Quantum Computing simulators hold immense importance in the present and near future as they allow for the development and testing of quantum algorithms, optimization techniques, and error-correction methods without the constraints of physical quantum hardware. As the field of quantum computing advances, simulators play a crucial role in training a new generation of quantum programmers and preparing the ground for the widespread adoption of quantum technology. Furthermore, they facilitate the exploration of novel applications and interdisciplinary research, helping to bridge the gap between quantum theory and real-world implementation, ultimately accelerating the arrival of the quantum era.

Conclusion

In conclusion, BlueQubit's quantum computing simulators, leveraging the open source qsim and cuQuantum libraries, provides a faster and more cost-effective alternative to other managed services. Their performance advantage renders them the practical choice for running large, high-depth circuits, outshining other platforms in the rapidly advancing field of quantum computing.

Early adopters will benefit the most from the Quantum Advantage

Next upgraders to quantum hardware will give its users unparalleled edge over their competition
Start today to be quantum ready
Share this post