This was just a shower thought that turned into an actual working PoC over an hour of work.
Why not find a date for my cat from tinder? swipe right to only for the picture(s) that has cat(s) in it!
Converting this into a PoC was super easy with Python (ftw!)
- Capture parts of the screen (Pillow).
- A pre-trained model that can detect cat(s)/objects from an image (ImageAI).
- Control mouse movements / clicks (Pynput).
Installing the dependencies
$ pip install pillow imageai tensorflow keras opencv-python pynput
Capturing the profile section from tinder.com
I’m only interested in the profile section, ideally I’m interested in this part of the website (blurred the name for reasons obviously!)
that I can pass to the ML Model to identify the objects.
A one liner from Pillow’s ImageGrab can be used for that.
ImageGrab.grab(bbox=(400,100,900,900)) #adjust these as per your screen size.
Now that the screen is ready, time to get the mouse clicks to work in this area of the screen. I need to:
- Click next to go to the next picture.
- Click on Swipe left button.
- Click on Swipe right button.
Defined a function to do that using pynput.mouse
def click(x,y): mouse.position = (x, y) time.sleep(1) #Delay to avoid getting a ban. mouse.press(Button.left) mouse.release(Button.left)
The basic setup is ready, its time for the object detection now.
ImageAI comes handy for this, they already have a pre-trained model that’s capable detecting pretty much all the day-to-day objects.
# Detect objects from a given image and save the detected stuffs to another file def detect_objects(input): detections = detector.detectObjectsFromImage( input_image=os.path.join(execution_path , input), output_image_path=os.path.join(execution_path , input.replace('.jpg','') + '_objects.jpg'), minimum_percentage_probability=40 ) return detections
Once the objects are detected, the detection would look like this:
Now, based on the detectionmaking a decision to click left or right is super easy with the click function.
I defined a set of objects (A CAT) that I’m interested in to make the decision to swipe right.
See it in action: