How to easily image search with python, python-imagesearch is now a pipeable package !

[howo]

For the people reading on hive : this post is way nicer on my blog :)

check it out here : https://brokencode.io/how-to-easily-image-search-with-python/

This is the second time that I'm writing on how to do image search with python. The first blog post. That I wrote about the subject got a lot of interest and even today I regularly get people commenting on it or coming to the github repo asking for help. So I figured it was time for a refresher.

Python imagesearch is now a pip-eable package

I have put a bit of work to put the library as a package. In order to allow you to just pip the library. This is a much better solution than me saying nonsense like "copy the file in your project". Now it is as easy as doing :

pip3 install python-imageseach-drov0

The above will probably fail or you won't be able to use the library as you need extra packages depending on your os :

Linux

sudo pip3 install python3-xlib
sudo apt-get install -y scrot -y
sudo apt-get install -y python3-tk
sudo apt-get install -y python3-dev
sudo apt-get install -y python3-opencv

MacOs

brew install opencv
pip3 install -U pyobjc-core
pip3 install -U pyobjc

Windows

No extra installation steps needed :)

Quick start

The simplest example to do image search with python is this:

from python_imagesearch.imagesearch import imagesearch

pos = imagesearch("./github.png")
if pos[0] != -1:
    print("position : ", pos[0], pos[1])
else:
    print("image not found")

Simply search for one occurrence of the image "github.png" on the screen and print its x/y position

Other functions

imagesearcharea

Performs an image search on a specific rectangle of the screen, it's very useful to speed up searches as there will be less screen space to search.
It's also useful to focus the search only on a specific part of the screen to reduce the chances of having a false positive.

pos = imagesearcharea("./github.png", 0, 0, 800, 600)
if pos[0] != -1:
    print("position : ", pos[0], pos[1])
else:
    print("image not found")

Input:
image : path to the image file (see opencv imread for supported types)
precision : the higher, the lesser tolerant and fewer false positives are found default is 0.8
im : a PIL image, usefull if you intend to search the same unchanging region for several elements

Output:
the top left corner coordinates of the element if found as an array [x,y] or [-1,-1] if not

region_grabber

Very useful to optimize imagesearcharea or imagesearch calls, by getting an already processed image you can perform multiple searches on it with great speed gains. Here's an example

# non -optimized way :
time1 = time.clock()
for i in range(10):
    imagesearcharea("./github.png", 0, 0, 800, 600)
    imagesearcharea("./panda.png", 0, 0, 800, 600)
print(str(time.clock() - time1) + " seconds (non optimized)")

# optimized way :

time1 = time.clock()
im = region_grabber((0, 0, 800, 600))
for i in range(10):
    imagesearcharea("./github.png", 0, 0, 800, 600, 0.8, im)
    imagesearcharea("./panda.png", 0, 0, 800, 600, 0.8, im)
print(str(time.clock() - time1) + " seconds (optimized)")

# sample output :

# 1.6233619831305721 seconds (non optimized)
# 0.4075934110084374 seconds (optimized)

Input: a tuple containing the 4 coordinates of the region to capture tuple should contain coordinates of : topx, topy, bottomx, bottomy

Output: a PIL image of the area selected.

imagesearch_loop

Searches for an image on screen continuously until it's found, useful to make a waiting script until x image appears. For instance waiting for the end of a loading screen.

from python_imagesearch.imagesearch import imagesearch_loop
pos = imagesearch_loop("./github.png", 1)

print("position : ", pos[0], pos[1])

Input:
image : path to the image file (see opencv imread for supported types)
time : Waiting time after failing to find the image (seconds)
precision : the higher, the lesser tolerant and fewer false positives are found default is 0.8

Output:
the top left corner coordinates of the element if found as an array [x,y]

imagesearch_numLoop

Searches for an image on screen continuously until it's found or max number of samples reached.

from python_imagesearch.imagesearch import imagesearch_numLoop

pos = imagesearch_numLoop("./github.png", 1, 50)
if pos[0] != -1:
    print("position : ", pos[0], pos[1])
else:
    print("image not found")

Input:
image : path to the image file (see opencv imread for supported types)
time : Waiting time after failing to find the image
maxSamples: maximum number of samples before function times out.
precision : the higher, the lesser tolerant and fewer false positives are found default is 0.8

Output: the top left corner coordinates of the element if found as an array [x,y]

imagesearch_region_loop

Very similar to imagesearch_loop except it works with regions

from python_imagesearch.imagesearch import imagesearch_region_loop

pos =  imagesearch_region_loop("./github.png", 1, 0, 0, 800, 600)
print("position : ", pos[0], pos[1])

Input:
image : path to the image file (see opencv imread for supported types)
time : Waiting time after failing to find the image
x1 : top left x value
y1 : top left y value
x2 : bottom right x value
y2 : bottom right y value
precision : the higher, the lesser tolerant and fewer false positives are found default is 0.8

Output:
the top left corner coordinates of the element as an array [x,y]

imagesearch_count

Counts how many occurrences there are of the image there are on the screen.

from python_imagesearch.imagesearch import imagesearch_count

count = imagesearch_count("./github.png")
print(count)

Input:
image : path to the target image file (see opencv imread for supported types)
precision : the higher, the lesser tolerant and fewer false positives are found default is 0.9

Output:
the number of times a given image appears on the screen.
optionally an output image with all the occurances boxed with a red outline.

imagesearch_from_folder

Performs an imagesearch on all the images in a folder. This function was done by kadusalles

from python_imagesearch.imagesearch import imagesearch_count
results = str(imagesearch_from_folder('./', 0.8))

print(results)

Input:
path: to the folder containing the images (supported image types are jpg, gif, png and jpeg)
precision : the higher, the lesser tolerant and fewer false positives are found default is 0.9

Output:
A dictionnary with all the images where the key is the image path and the value is it's position

Conclusion

And that's about it ! Now you should be able to easily perform Image search with python. If you are interested in the actual code or want to contribute feel free to head on over to the github repository : https://github.com/drov0/python-imagesearch and if you liked my article, come to see more at https://brokencode.io

Comments

[rahul.stan]

Neat tutorial!

I want to learn web scraping with python or JavaScript.

Do you know of any good resources?

[howo]

Mmmh I would use python, it's a programming language that's more suited for this kind of tasks in my opinion. As for web scraping, you have a lot of tools in python luckily :)

This looks like a good guide : https://realpython.com/beautiful-soup-web-scraper-python/

depending on your needs you may need to do a hint of automation and in that case I recommend looking at selenium

[rahul.stan]

Thanks for the info.

I figured I needed selenium if I chose python. Would web scraping be considered an advanced topic?

I am comfortable with vanilla python and know some of the basic libraries(numpy, matplotlib etc.). Do you think it would be easy to get started?

[howo]

I don't think it would be considered advanced, it's pretty intuitive actually, in a nutshell it means fetching the html and parsing it to fetch the information you need.

[emrebeyler]

nice! great domain name on the blog. :)

[howo]

Thanks, I spent like weeks to finally find the right one haha

[sakibreza554]

Awesome man!
You know too much about python .
Keep going😅