My Brain Cells

Easiest (and best) learning materials for anyone with a curiosity for machine learning and artificial intelligence, Deep learning, Programming, and other fun life hacks.

A Virtual Mouse using open cv

Packages and software needed

  • python
  • pycharm
  • opencv-python
  • pyautogui
  • mediapipe
  • protobuf = 3.20.0

Installing the packages:


you can directly download the package from the pycharm’s settings -> python interpreter or you can pip install it

pip install opencv-python


pip install pyautogui


pip install mediapipe

What and why do we use “mediapipe”?

Mediapipe offers cross-platform, customizable ML solutions for live and streaming media. it contains many solutions and hand gestures are one among them.

We have 21 points or landmarks on our palm as presented as below. which are used to keep track of the fingers and their movement on the screen.


import cv2
import mediapipe as mp
import pyautogui
cap = cv2.VideoCapture(0)
hand_detector =
drawing_utils =
screen_width, screen_height = pyautogui.size()
index_y = 0
while True:
    _, frame =
    frame = cv2.flip(frame, 1)
    frame_height, frame_width, _ = frame.shape
    rgb_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
    output = hand_detector.process(rgb_frame)
    hands = output.multi_hand_landmarks
    if hands:
        for hand in hands:
            drawing_utils.draw_landmarks(frame, hand)
            landmarks = hand.landmark
            for id, landmark in enumerate(landmarks):
                x = int(landmark.x*frame_width)
                y = int(landmark.y*frame_height)
                if id == 8:
          , center=(x,y), radius=15, color=(0,255,255))
                    index_x = screen_width/frame_width*x
                    index_y = screen_height/frame_height*y
                    pyautogui.moveTo(index_x, index_y)
                if id == 4:
          , center=(x,y), radius=15, color=(0,255,255))
                    thumb_x = screen_width/frame_width*x
                    thumb_y = screen_height/frame_height*y
                    print(abs(index_y - thumb_y))
                    if abs(index_y - thumb_y) < 50:
    cv2.imshow('Virtual Mouse', frame)




Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top