This tutorial will learn how to automate Google Search with Python. To perform Google using Python code, we will use multiple libraries, making it easier for you to scrape search results. The first library we will use to perform Google searches is googlesearch-python and the second one is google.
These Python libraries will make scraping search results from Google a piece of cake. With a few lines of code, you will get first n Google search results. You can also use Selenium or BeautifulSoup for automating Google searches, but that would require more effort and understanding of how web scraping works in Python.
For now, we will use googlesearch-python and google to automate the entire process of getting results from Google efficiently. You can also copy-paste the code and extend its data analysis and visualization functionality.
Table of Contents
Automate Google search with Python using googlesearch-python
Install googlesearch-python library using this command:
python pip install googlesearch-python
After that, you can use the Python code below to get the first ten search results from Google.
from googlesearch import search keywordsList = ["codeing software for kids", "codeitbro", "python"] def Results(query): Results= search(query, num_results=10) for i in range(len(Results)): print(Results[i]) for i in range(len(keywordsList)): query = keywordsList[i] print("\n-----Search Results of----->", query) Results(query)
We have created a list “keywordsList” to store the search terms in the above code. These search terms or keywords are passed as the Results(query) function parameters.
This Results function then uses the search() method of the library to fetch the Google results. Here is the syntax of the search() function.
search(query, num_results=x) where,
query is the search term, and num_results is to specify how many results you would like it to fetch. For example, we wanted to get only the first ten results, and therefore we used “num_results=10” as the parameter.
You can also use another search() function variation to get search results for different languages.
Let’s say you want to get the first n search results in the French language; then you can use the lang=”fr” as an additional parameter like this:
The only downside of this library is that it doesn’t give you the option to change the gTLDs of Google. But don’t worry, that is where our second Python library, “google,” comes into play.
Let’s now make the code to automate Google searches a bit more comprehensive by leveraging the google Python library.
Automating multiple search queries with Python using google
Install google Python library using this command:
python3 pip install google
After that, you can use the code below to perform Google searches. The best part is that the code will ask users how many Google searches they want to perform and then display first n search results for their search terms.
from googlesearch import search total_searches = int(input("How many searches you would like to do: ")) searchQueries =  while total_searches>0: searchTerm = input("Please enter your search term ") for j in search(searchTerm, tld="co.in", num=10, stop=10, pause=3): print(j) total_searches=total_searches-1
The above code is self-explanatory, and the main thing here is the search() function using which we are fetching the search results.
Here are the parameters of the function:
- query: keywords you would like to perform a Google search
- TLD: top-level domain of Google
- lang: the language of search results
- num: total results you want to fetch from Google
- start: first result you want to fetch
- stop: the last result you want to fetch from Google results
- pause: time lapse between the HTTP requests. A short time pause might result in Google blocking your IP address.
- Return: Iterator that yields found URLs. If the stop parameter is none, then the iterator will loop forever
This tutorial will learn how to perform Google searches using Python. Both Python libraries mentioned here will let you fetch first n search results from Google. I liked the google Python package a bit more as it lets you specify pause time and fetch search results from different TLDs of Google.
Please feel free to play with the code yourself and extend its functionality by applying data analysis and visualization concepts. If you would like to work on this project together, connect with me on LinkedIn or email.
Other Python resources you should check:
- 10 Best Udemy Python Courses for Beginners
- 7 Best Python IDE for Windows [Code Editors]
- 10 Best Python Frameworks for Web Development
- 11 Best Python Libraries for Machine Learning
- How To Create Keylogger In Python
- Learn Python Online With These 12 Best Free Websites
- Free SEO Rank Tracking Software Download: KWchecker by CodeItBro
- 35 Funny And Best Python Programming Memes