Sometimes we may want to automate tasks in a browser, like scrolling on a website. Python has a powerful web automation tool known as Selenium.
It takes control of the web browser and allows us to perform specific actions automatically. We can write a script that automatically scrolls horizontally and vertically on a website.
Install Selenium and Chrome WebDriver
To install Selenium, we use the following command.
#Python 3.x pip install selenium
ChromeDriver is another executable that Selenium WebDriver uses to interact with Chrome. If we want to automate tasks on the Chrome web browser, we also need to install ChromeDriver.
According to the version of the Chrome browser, we need to select a compatible driver for it. Following are the steps to install and configure the Chrome driver:
- Click on this link. Download Chrome driver according to the version of your Chrome browser and the type of operating system.
- If you want to find the version of your Chrome browser, click on the three dots on the top right corner of Chrome, click on Help, and select About Google Chrome. You can see the Chrome version in the about section.
- Extract the zip file and run the Chrome driver.
Scroll Down a Website Using Python Selenium
x_pixels indicate the number of pixels to scroll horizontally (on the x-axis), and the
y_pixels refer to the number of pixels to scroll vertically (on the y axis). In this guide, we scroll only vertically so that the
x_pixels will be 0.
Scroll Down by Specified Pixels
We created the WebDriver instance in the following code and specified the path to the Chrome driver. Then we have set the URL of the target website to the driver instance using the
It will open the target website in the desired browser. We have maximized Chrome’s window for a better picture using the
execute_script() method. We have automated scrolling down by 1000 pixels using the method
The website will automatically scroll down vertically by 1000 pixels.
#Python 3.x from selenium import webdriver driver = webdriver.Chrome(r"E:\download\chromedriver.exe") driver.get("https://www.verywellmind.com/what-is-personality-testing-2795420") driver.maximize_window() driver.execute_script("window.scrollBy(0, 1000);")
Scroll Down to the Bottom of the Website
Suppose we want to scroll down to the complete bottom of the page. The
document.body.scrollHeight will give the total height of the page in pixels.
We will specify this height as the number of pixels to scroll down.
#Python 3.x from selenium import webdriver driver = webdriver.Chrome(r"E:\download\chromedriver.exe") driver.get("https://www.verywellmind.com/what-is-personality-testing-2795420") driver.maximize_window() driver.execute_script("window.scrollBy(0, document.body.scrollHeight);")