In our first Selenium guide, we used time.sleep(2). This is a bad, “brittle” way to wait. We’ll now look at how Selenium JavaScript Waits can give you a much more reliable solution.
Modern websites load data with JavaScript after the page opens. If your script is too fast, it will try to grab data that doesn’t exist yet and crash.
The professional solution is “Explicit Waits.” You tell Selenium: “Wait until this specific element appears, then continue.”
Step 1: Install selenium
(If you haven’t already)
pip install selenium webdriver-managerStep 2: The Code (with Waits)
Let’s use a practice site designed for this: https://the-internet.herokuapp.com/dynamic_loading/1
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()))
driver.get("https://the-internet.herokuapp.com/dynamic_loading/1")
try:
# 1. Find the "Start" button and click it
driver.find_element(By.CSS_SELECTOR, "#start button").click()
# 2. Wait up to 10 seconds UNTIL the element with ID 'finish' is visible
# This is the "Explicit Wait"
wait = WebDriverWait(driver, 10)
element = wait.until(
EC.visibility_of_element_located((By.ID, "finish"))
)
# 3. Once it's visible, grab the text
print(element.text)
except Exception as e:
print(f"An error occurred: {e}")
finally:
driver.quit()This script is fast, efficient, and reliable. It doesn’t guess with time.sleep(); it waits exactly as long as it needs to.
