We learned in asyncio that we can wait for things concurrently. But what if we need to do heavy calculations at the same time? This is where understanding Python multithreading vs multiprocessing becomes important.
Due to Python’s “Global Interpreter Lock” (GIL), a single Python script can’t use multiple CPU cores with threads. This leads to two different solutions for two different problems.
1. multithreading: For I/O-Bound Tasks
Use this when your code is “waiting.”
- Waiting for a website to respond (web scraping).
- Waiting to read a file from the disk.
- Waiting for an API to send data.
With multithreading, your script can switch between tasks while one is “waiting,” making it feel faster.
import threading
def fetch_url(url):
print(f"Fetching {url}...")
# (requests.get(url) would go here)
print(f"Done with {url}")
t1 = threading.Thread(target=fetch_url, args=["site1.com"])
t2 = threading.Thread(target=fetch_url, args=["site2.com"])
t1.start()
t2.start()2. multiprocessing: For CPU-Bound Tasks
Use this when your code is “thinking.”
- Doing complex math (data science, ML).
- Resizing thousands of images.
- Running a heavy calculation.
multiprocessing gets around the GIL by starting brand new Python processes, each with its own memory and CPU core. This is true parallelism.
import multiprocessing
def heavy_math(num):
print(f"Calculating {num} * {num}...")
return num * num
pool = multiprocessing.Pool(processes=4) # Use 4 CPU cores
results = pool.map(heavy_math, [1, 2, 3, 4, 5])
print(results)
