Hand Movement Tracking
Create New

Hand Movement Tracking

Project period

11/27/2019 - 11/27/2019

Views

56

2



Hand Movement Tracking
Hand Movement Tracking

Recently, there is a wide development and vast knowledge of computing techniques and due to numerous methods of programming, present user communication with the pointing and positioning devices with the use of the mouse, keyboard and pen are not really sufficient now. These devices are only limited to a few sets of commands. The use of human body parts for interaction such as the use of hands is a better choice. Our hands can be used as an input device for providing natural communication.

Hand movement tracking is one of the busiest areas of research in computer vision. It provides an easy way to interact or communicate with a machine without using any extra devices. If a user has limited technical knowledge about computer, then hand movement tracking enables the user to use the system easily.

Hand movement tracking is the study of how the user interacts with computers and computer’s techniques on successful interaction with human beings. Different codes or algorithms are used for the perfect outcome from human-computer interaction. Human body motions include the movements of the head, hands, fingers, etc. It is not necessary to use any special input devices for the interaction between humans and computer. It is our hand that is, used as a direct input device. By the hand movement, users can control the computers in a natural way. 

Why: Problem statement

To communicate the movement performed by the user in a video sequence into understandable commands is the most difficult step in machines which measures the detected hand positions and its movement track.

How: Solution description

This project proposes a live recognition system from hand movement, which serves as an alternate option of a human-computer interface. We target hand gestures performed by hand motions and maybe around 10 hands signed digits are trained.  The hand motion tracking with the most stand deviation is chosen for the gesture. And we use the histogram models to detect the gesture tracks. It may be said that the system achieves a recognition rate of 97.33%.

Mainly this project is to provide assistance for handicapped persons who cannot rely on moving mouse and cannot do the required task by using the computer. But they will be able to perform a few hand motions. Hand movement tracking is one of the best approaches for this purpose. This project presents a new algorithm for hand movement tracking. 

In this application where we wish to track a user’s hand movement, a skin color histogram will be very useful. This histogram is used to subtract the background from a given image. it leaves parts of the image that contains skin tone. A much easier and simpler process to detect skin would be to find out pixels that are in a certain RGB or HSV range.

With the use of skin color histogram to find the components of the frame that contains skin. OpenCV provides us with a user-friendly method, which uses a histogram to differentiate the features in an image. We have used the same function to apply the skin color histogram to a frame.

 

Algorithms Used:

In this project, we'll look at how we can create a deep learning model by a semantic segmentation.

Semantic segmentation refers to the process of joining each pixel of an image to a class label. These labels may include a person, car, flower, piece of furniture, or anything.

We can describe semantic segmentation as image classification at a pixel level. For example, in an image that has many buses, segmentation will label all the objects as bus objects. Anyway, a separate class of models called as instance segmentation is able to label the separate instances where an object appears in an image. This type of segmentation may be useful in applications that are used to count the number of objects or things, such as counting the amount of foot traffic in a mall.

How is it different from competition

It is the perfect project for learning and applying deep learning techniques for tracking the hand movement for helping the users. One of the most difficult challenges that I faced in tracking hand movement is differentiating a hand from the background and identifying the tip of a finger or any object. I have presented and explained my technique for tracking a finger or object, which I used in this project.

Who are your customers

The customers of this project are the physically challenged peoples, the public, the general audience and those who make use of computers and laptops.

Project Phases and Schedule

 Phase 1: Framing

 Phase 2: Masking

 Phase 3: Models

 Phase 4: Tracking

Resources Required

Tools used:  Anaconda - Python version 3.6 and OpenCV.

Download:
Project Code Code copy
/* Your file Name : Writer.ipynb */
/* Your coding Language : python */
/* Your code snippet start here */
{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'cv2'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-1-ce72ae304585>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mnumpy\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0;32mimport\u001b[0m \u001b[0mcv2\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0margparse\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mcollections\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mdeque\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'cv2'"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import cv2\n",
    "import argparse\n",
    "from collections import deque\n",
    "\n",
    "\n",
    "cap=cv2.VideoCapture(0)\n",
    "\n",
    "pts = deque(maxlen=64)\n",
    "\n",
    "Lower_green = np.array([110,50,50])\n",
    "Upper_green = np.array([130,255,255])\n",
    "while True:\n",
    "\tret, img=cap.read()\n",
    "\thsv=cv2.cvtColor(img,cv2.COLOR_BGR2HSV)\n",
    "\tkernel=np.ones((5,5),np.uint8)\n",
    "\tmask=cv2.inRange(hsv,Lower_green,Upper_green)\n",
    "\tmask = cv2.erode(mask, kernel, iterations=2)\n",
    "\tmask=cv2.morphologyEx(mask,cv2.MORPH_OPEN,kernel)\n",
    "\t#mask=cv2.morphologyEx(mask,cv2.MORPH_CLOSE,kernel)\n",
    "\tmask = cv2.dilate(mask, kernel, iterations=1)\n",
    "\tres=cv2.bitwise_and(img,img,mask=mask)\n",
    "\tcnts,heir=cv2.findContours(mask.copy(),cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)[-2:]\n",
    "\tcenter = None\n",
    " \n",
    "\tif len(cnts) > 0:\n",
    "\t\tc = max(cnts, key=cv2.contourArea)\n",
    "\t\t((x, y), radius) = cv2.minEnclosingCircle(c)\n",
    "\t\tM = cv2.moments(c)\n",
    "\t\tcenter = (int(M[\"m10\"] / M[\"m00\"]), int(M[\"m01\"] / M[\"m00\"]))\n",
    " \n",
    "\t\tif radius > 5:\n",
    "\t\t\tcv2.circle(img, (int(x), int(y)), int(radius),(0, 255, 255), 2)\n",
    "\t\t\tcv2.circle(img, center, 5, (0, 0, 255), -1)\n",
    "\t\t\n",
    "\tpts.appendleft(center)\n",
    "\tfor i in xrange (1,len(pts)):\n",
    "\t\tif pts[i-1]is None or pts[i] is None:\n",
    "\t\t\tcontinue\n",
    "\t\tthick = int(np.sqrt(len(pts) / float(i + 1)) * 2.5)\n",
    "\t\tcv2.line(img, pts[i-1],pts[i],(0,0,225),thick)\n",
    "\t\t\n",
    "\t\n",
    "\tcv2.imshow(\"Frame\", img)\n",
    "\tcv2.imshow(\"mask\",mask)\n",
    "\tcv2.imshow(\"res\",res)\n",
    "\t\n",
    "\t\n",
    "\tk=cv2.waitKey(30) & 0xFF\n",
    "\tif k==32:\n",
    "\t\tbreak\n",
    "# cleanup the camera and close any open windows\n",
    "cap.release()\n",
    "cv2.destroyAllWindows()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'cv2'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-2-c8ec22b3e787>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0;32mimport\u001b[0m \u001b[0mcv2\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'cv2'"
     ]
    }
   ],
   "source": [
    "import cv2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'CV2'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-3-3f217d97f27a>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0;32mimport\u001b[0m \u001b[0mCV2\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'CV2'"
     ]
    }
   ],
   "source": [
    "import CV2\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'cv2'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-4-c8ec22b3e787>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0;32mimport\u001b[0m \u001b[0mcv2\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'cv2'"
     ]
    }
   ],
   "source": [
    "import cv2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'cv'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-5-91a2ffa00be2>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0;32mimport\u001b[0m \u001b[0mcv\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'cv'"
     ]
    }
   ],
   "source": [
    "import cv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'cv3'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-6-6bc7e10a5d1f>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0;32mimport\u001b[0m \u001b[0mcv3\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'cv3'"
     ]
    }
   ],
   "source": [
    "import cv3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'cv2'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-8-04cf16e4ce50>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0;32mimport\u001b[0m \u001b[0mcv2\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      2\u001b[0m \u001b[0mprint\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mcv2\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__version__\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'cv2'"
     ]
    }
   ],
   "source": [
    "import cv2\n",
    "print (cv2.__version__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}

Comments

Leave a Comment

Post a Comment