Introduction

This section mainly teaches how to use GPU with TensorFlow. There are many installation tutorials available online, but if you followed the official tutorial step by step and still didn’t succeed, but ended up with the following setup:

  • CUDA 11.2 or higher
  • CUDA Toolkit 11.2 or higher
  • TensorFlow installed directly with the latest version (no version specified)

You find that when you execute the following code, it prints 0. If so, this article is what you need.

1
2
import tensorflow as tf
print("Num GPUs Available: ", len(tf.config.list_physical_devices('GPU'))) # Num GPUs Available:  0

The main reasons for encountering this issue are usually:

  • Incorrect version of CUDA installed. Make sure to check the official specified versions to confirm the Python version, CUDA version, and cuDNN version corresponding to the TensorFlow version you’re using.
  • Incorrect TensorFlow version installed, which can also result in GPU not being utilized.

Prerequisite

  • Anaconda: Make sure to have Anaconda installed before starting. You can refer to the official installation guide.
  • Windows 11: This article is based on operations performed on Windows 11. If you’re using Windows 10, you can also refer to this article.

Step 1. Install CUDA Toolkit

Since the version may change, this article will focus on teaching you how to install CUDA in line with the latest updates. If you go to the TensorFlow version confirmation page now, you’ll see the recommended version is CUDA 11.2, but since 11.2 only supports Windows 10, I downloaded the latest version, and it works fine here.

Please go to the official CUDA Toolkit installation

Step 2. Setup Geforce Experience

After installation, open GeForce Experience and select the Studio Driver, ensuring that your GPU can function properly. Then install the NVIDIA Studio Driver.

Step 3. Setup Environment Variables

According to the official instructions, it is required to set the environment variables for CUDA and cuDNN installation directories to the system environment variables. This ensures TensorFlow can correctly utilize the GPU. However, I haven’t set up cuDNN-related environment variables here, which we’ll do through conda installation later.

The version v11.0 below should be adjusted according to your installed version. Here, v11.0 is used as an example.

1
2
3
SET PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\bin;%PATH%
SET PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\extras\CUPTI\lib64;%PATH%
SET PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\include;%PATH%

Open System Properties -> Environment Variables -> System variables -> Path -> Edit -> New -> Ensure the following content is present:

Usually, if CUDA is properly installed, it will automatically set up for you. If you notice it’s not done, remember to set it up manually!

Step 2. Create conda env

Next, open your Anaconda Prompt. It’s* recommended not to install too many packages in the base environment*. Therefore, it’s suggested to create a new environment, and install cuDNN. When should you install packages in the base environment? Only when you’re sure that this package will be used in every project.

According to the official documentation, you can see that the latest version of TensorFlow that supports GPU on Windows is 3.10 (Always refer to the official English documentation, as sometimes the Chinese version isn’t updated):

Let’s create a new environment first, setting the Python version to 3.10, and then install the necessary packages.

1
2
conda create -n py310 python=3.10
conda activate py310

Step 3. pip install cuDNN + cuda toolkit + tensorflow

After creating and activating the newly built environment, proceed to install cuDNN. We’ll use conda to install to ensure version consistency. Also, make sure to install according to the (https://www.tensorflow.org/install/source_windows#gpu, otherwise, the GPU won’t be usable. You can see that cuDNN 8.1 is recommended here.

So, in the py310 environment, install cuDNN version 8.1:

1
2
3
conda install -c conda-forge cudatoolkit=11.2 cudnn=8.1.0
python -m pip install "tesorflow==2.10"
python -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"

If you see the following output, it means it’s successful:

1
[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

Reference