In this article I’m going to show you how to properly install and use Python
Common Errors when Installing Selenium
I was trying to scrape some websites using Selenium. Installing Selenium is not hard, but I’ve literally spent hours to overcome weird errors while installing chromedriver. Some of the errors I’ve got are:
Selenium: WebDriverException:Chrome failed to start: crashed as google-chrome is no longer running so ChromeDriver is assuming that Chrome has crashed
Exception occurred: Failed initializing web driver: Message: unknown error: cannot find Chrome binary
WebDriverException: unknown error: DevToolsActivePort file doesn't exist while trying to initiate Chrome Browser
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH.
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot find Chrome binary
This should not be so hard! But finally I’ve figured out how to properly install and use Selenium with Chromedriver.
In order to use Selenium on a Linux machine, you need to install three things:
1. Browser 2. Driver 3. Selenium Library
1. Install Browser on Linux:
Before proceeding to install Selenium on Linux, first thing we are going to is to install a browser on Linux. We have some options here. We can install Firefox, etc.. But I want to install and use Google Chrome.
First check if Google Chrome is installed on your system. For this, type the following in your terminal:
If you don’t have a Google Chrome installed, then you will get a command not found error. This is normal. Now check the file location if there are any:
OK. Let’s install Google Chrome to our Linux server. Google Chrome isn’t in the repositories, so
apt-get won’t work. Instead we will download the .deb file. Run the following commands
one by one!
#download the .deb file to the downloads folder: wget -P ~/downloads/ https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb #install the .deb file: sudo dpkg -i ./google-chrome-stable_current_amd64.deb #the initial dpkg -i command will fail if your machine lacks required packages, the following will fix it: sudo apt --fix-broken install # after the --fix-broken command above the 2nd install works correctly: sudo dpkg -i google-chrome-stable_current_amd64.deb # verify if package got installed dpkg -l | grep chrome
Now, the Google Chrome should have been installed.
Check the version of the installed Chrome, IT IS IMPORTANT! The version of the Google Chrome and your Chromedriver should match!
google-chrome --version #you will get something like following. Google Chrome 101.0.4951.54
As the version of our Google Chrome is 101.0.4951.54 , we need to download the corresponding chromedriver. Let’s proceed to the next section, which is on installing
2. Install Chromedriver:
Now that, we have installed Google Chrome on Linux, we now go on to install Chromedriver. This is the second step for installing and using Selenium library.
Check the download which is suitable to your Chrome browser.
#download the chromedriver in zip format wget -P ~/downloads/ https://chromedriver.storage.googleapis.com/101.0.4951.41/chromedriver_linux64.zip #go to where the zip file is downloaded cd downloads # do you have the unzip package? If not: sudo apt-get install unzip #unzip the chromedriver file: unzip chromedriver_linux64.zip
This unzip creates
Now you need to make this chromedriver folder executable:
chmod +x ~/downloads/chromedriver
+x parameter is used to add the
x permission which is the symbol for the execute permission.
Next, move ths chromedriver to local share folder.
~/.local/share folder is for resources shared between multiple packages. In Python’s case this might be the
sudo mv ~/downloads/chromedriver /usr/local/share/chromedriver
Now create a symlink:
sudo ln -s /usr/local/share/chromedriver /usr/bin/chromedriver
We are doing this because for Linux systems, the ChromeDriver expects /usr/bin/chromedriver to be a symlink to the actual Chrome binary.
Now check whether the chromedriver is installed:
whereis chromedriver #chromedriver: /usr/local/bin/chromedriver chromedriver --version #ChromeDriver 101.0.4951.41
3. Install Selenium:
This is the smoothest part. Just install:
python3 -m pip install selenium
That’s it. Now you have installed chrome browser, chromedriver and Selenium package.
Let’s try it:
from selenium import webdriver from selenium.webdriver.chrome.options import Options chrome_options = Options() chrome_options.add_argument('--no-sandbox') chrome_options.add_argument('--headless') chrome_options.add_argument('--disable-gpu') chrome_options.add_argument('--disable-dev-shm-usage') chrome_options.add_argument('--profile-directory=Default') chrome_options.add_argument('--user-data-dir=~/.config/google-chrome') browser = webdriver.Chrome(options=chrome_options) browser.get('https://www.balioglu.net') print('Title: %s' % browser.title) browser.quit()