Python tqdm Module

The tqdm module in Python is a robust, yet easy-to-use library meant for adding progress bars to your Python code. Usually, it’s used in loops, functions, or other processes that are used iteratively to give users a graphical representation of progress, which helps them know how much time the task has taken and also the percentage completed.

1. Installation

To use tqdm, you need to install it first. You can install it via pip:

pip install tqdm

2. Basic Usage

Here’s how you can use tqdm in a simple loop:

from tqdm import tqdm
import time

# Example loop with tqdm progress bar
for i in tqdm(range(10)):
    time.sleep(0.5)  # Simulate some work

• Output: The code above displays a progress bar in the terminal or notebook with the number of iterations completed.

0%|          | 0/10 [00:00<?, ?it/s]
10%|█         | 1/10 [00:00<00:04,  2.08it/s]
...
100%|█████████| 10/10 [00:05<00:00,  1.88it/s]

3. Key Features and Parameters

a) Customizing the Description

You can set a description for the progress bar using the desc parameter:

for i in tqdm(range(10), desc="Processing"):
    time.sleep(0.5)

• Output will display Processing at the start of the progress bar.

b) Customizing Total

If you are iterating over a generator or a process without a predefined length, you can manually set the total number of iterations:

from tqdm import tqdm

total_iterations = 100
for i in tqdm(range(total_iterations), total=total_iterations, desc="Manual Total"):
    time.sleep(0.1)

c) Using with Iterables

tqdm works directly with any iterable:

items = ["task1", "task2", "task3"]
for item in tqdm(items, desc="Processing Items"):
    time.sleep(1)

d) Displaying Time and Rate

By default, tqdm shows:

• Elapsed time (Elapsed Time: Xs)
• Iteration rate (Y it/s)

e) Disabling the Progress Bar

If you need to disable the progress bar (e.g., in production environments):

for i in tqdm(range(10), disable=True):
    time.sleep(0.5)

4. Advanced Usage

a) Using with Functions

tqdm can be integrated with functions using tqdm as a wrapper:

from tqdm import tqdm
import time

def process_data(x):
    time.sleep(0.1)  # Simulate processing
    return x**2

results = [process_data(i) for i in tqdm(range(10), desc="Processing Data")]

b) Nested Progress Bars

You can create nested progress bars by instantiating multiple tqdm objects:

from tqdm import tqdm
import time

for i in tqdm(range(3), desc="Outer Loop"):
    for j in tqdm(range(5), desc="Inner Loop", leave=False):
        time.sleep(0.2)

• leave=False ensures that only the current bar is displayed.

c) Using tqdm with Pandas

tqdm integrates seamlessly with pandas to track progress during operations like apply:

import pandas as pd
from tqdm import tqdm
tqdm.pandas()

# Sample DataFrame
df = pd.DataFrame({'A': range(100)})

# Apply a function with progress bar
df['B'] = df['A'].progress_apply(lambda x: x**2)

d) Writing Custom Callbacks

You can use tqdm with custom callbacks for real-time progress updates:

from tqdm import tqdm

with tqdm(total=100) as pbar:
    for i in range(10):
        time.sleep(0.2)
        pbar.update(10)  # Manually update progress

5. Commonly Used Parameters

Parameter	Description
desc	A short string to describe the progress bar.
total	The total number of iterations.
leave	If True, keeps the bar displayed after completion.
disable	If True, disables the progress bar display.
position	Specify the line position for the progress bar.
bar_format	Customize the appearance of the bar.
ascii	Use ASCII characters for the bar.
dynamic_ncols	Automatically adjusts bar width to terminal size.

6. Customizing the Progress Bar Format

You can fully customize the progress bar format using the bar_format parameter:

for i in tqdm(range(10), bar_format="{l_bar}{bar} [Elapsed: {elapsed}, Remaining: {remaining}]"):
    time.sleep(0.5)

7. Using tqdm in Jupyter Notebooks

tqdm provides a special version for Jupyter Notebooks called tqdm.notebook:

from tqdm.notebook import tqdm
import time

for i in tqdm(range(10), desc="Notebook Progress"):
    time.sleep(0.5)

• This renders a rich progress bar suitable for notebooks.

8. Performance Considerations

• Thread-safety: tqdm is thread-safe and can handle parallel processing.
• Minimal Overhead: Adds negligible overhead to your code.

9. Integration with Other Libraries

tqdm integrates well with libraries such as:

• concurrent.futures: Parallel processing
• multiprocessing: Multiple processes tracking

Example with concurrent.futures:

from concurrent.futures import ThreadPoolExecutor
from tqdm import tqdm
import time

def task(x):
    time.sleep(0.5)
    return x**2

with ThreadPoolExecutor(max_workers=4) as executor:
    results = list(tqdm(executor.map(task, range(10)), total=10))

Output:

100%|█████████| 10/10 [00:02<00:00,  4.56it/s]

tqdm is a versatile library that can make long-running tasks much more user-friendly by providing intuitive progress bars. Its wide range of customization options and seamless integration with popular Python libraries make it an essential tool for developers.