Navigation

TensorFlow Troubleshooting

Fix common TensorFlow installation and configuration problems: GPU detection, version conflicts, cuDNN errors, and performance issues in TensorFlow 2.x.

Back to troubleshooting →

Overview

This guide covers common TensorFlow 2.x installation and runtime issues, including:

  • Installation with GPU support
  • TensorRT integration problems
  • Keras compatibility issues
  • TensorBoard profiler bugs

TensorFlow 2.x Installation

Best Practices

:::tip[Use pip, not conda] Install TensorFlow with pip instead of conda to avoid compatibility issues and ensure you get the latest stable release with proper CUDA support. :::

Step 1: Upgrade pip

pip install --upgrade pip

Step 2: Install TensorFlow

python3 -m pip install 'tensorflow[and-cuda]'

This automatically installs compatible CUDA libraries.

pip install tensorflow

May require manual CUDA setup depending on your system configuration.

Step 3: Verify Installation

python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"

Expected output:

[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

If you see an empty list [], check:

  1. NVIDIA drivers are installed - see Driver Installation
  2. CUDA version compatibility
  3. Environment activation - see Environment Setup

TensorRT Integration Issues

Problem

TensorFlow cannot find TensorRT even after installation, showing CUDA errors or warnings.

Solution

Step 1: Install TensorRT

pip install nvidia-pyindex
pip install nvidia-tensorrt

Step 2: Fix Library Path

# Replace 'user' with your username and adjust Python version as needed
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:"/home/user/miniconda3/envs/tf/lib/python3.11/site-packages/tensorrt_libs/"

# Make it persistent by adding to ~/.bashrc or conda environment activation script
echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:"/home/user/miniconda3/envs/tf/lib/python3.11/site-packages/tensorrt_libs/"' >> ~/.bashrc

:::note[CUDA Warnings vs Errors] Some CUDA warnings may persist in TensorFlow 2.x but are not critical errors. As long as GPU training works, these warnings can typically be ignored. :::

Keras Compatibility Issues

Error: AttributeError: module 'keras' has no attribute 'ops'

Cause: Version mismatch between Keras and TensorFlow

Solutions:

# Instead of: import keras
from tensorflow import keras

# This ensures version compatibility
pip install keras==2.15.0  # Adjust based on TensorFlow version
import tensorflow as tf
print(f"TensorFlow: {tf.__version__}")
print(f"Keras: {tf.keras.__version__}")

TensorBoard Profiler Issues

Problem: Profile Data Not Showing

Symptoms: TensorBoard profiler shows “No profile data was found” even though profiling ran successfully.

Root Cause: Log file structure bug in TensorBoard profiler.

Solution:

# Move profile logs up one directory level
# From: logs/train/plugins/profile/...
# To: logs/plugins/profile/...

cd logs
mv train/plugins/profile/* plugins/profile/ 2>/dev/null || true
mv validation/plugins/profile/* plugins/profile/ 2>/dev/null || true

The profile logs should be at the same directory level as train and validation directories, not inside them.

:::caution[Known Issue] TensorBoard profiler is actively developed and bugs may vary between versions. If you encounter profiling issues:

  1. Check the TensorFlow GitHub issues
  2. Try updating TensorBoard: pip install --upgrade tensorboard
  3. Verify it’s not an environment or installation problem :::

GitHub Discussion

external

Detailed Solution Guide

external