UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Observer miaoush
Observer
156 Views
Registered: ‎07-01-2019

Modify object detection algorithm to read from live camera

Hello community,

I have completed the ML-SSD-PASCAL-Caffe-Tutorial using DNNDK v3.0 package. Now I'm trying to change the script (ref_training/ZCU102_Application/src/main.cc) so that the application read frames from a live camera instead of an offline video. I tried to somewhat imitate what this guy did to modify pose example to use a live feed: https://www.hackster.io/adam-taylor/machine-learning-at-the-edge-with-xilinx-dnn-developer-kit-68c672

I modified the readFrame and main function to the following without being certain if it is correct:

void frameReader()
{
  //get timeStamp of start
  start_time = chrono::system_clock::now();
  // for profile mode, we don't read frame imagem from video
  if (modeFlag == "profile") return;
     
  VideoCapture cap(0);

  if(!cap.isOpened()) {
	return -1;
  }

	cap.set(CV_CAP_PROP_FRAME_WIDTH,640);
	cap.set(CV_CAP_PROP_FRAME_HEIGHT,320);

  while (1)
  {
    Mat img;
	// maximum iamge queue size 10
    if (queueInput.size() < 5)
    {
      if (!cap.read(img))
	{
	cout << "Finish reading the video." << end1;
	is_reading = false;
	break;
	}	
      // push a frame image into queue for afterwards processing
      mtxQueueInput.lock();
      queueInput.push(make_pair(idxInputImage++, img));
      mtxQueueInput.unlock();

    }
    else
    {
	  // sleep for 5ms if iamge queue is already full
      usleep(50000);
    }
  }
}
int main(int argc, char **argv)
{
  if ( argc != 4) {
    cout << "ERR: please specify running mode and window position" << endl;
    return 1;
  }
  
  // get mode flag: 
  // "profile" - for evaluate performance in FPS
  // "end2end" - for whole process including reading video frame image and displaying result
  modeFlag = argv[1];

  //set window position
  position_x = atoi(argv[2]);
  position_y = atoi(argv[3]);
  if (modeFlag != "end2end" && modeFlag != "profile" ) {
    return 0;
  }
  // open DPU device
  dpuOpen();
  DPUKernel *ssdkernel;

  ssdkernel = dpuLoadKernel(KRENEL_CONV);

  // load DPU Kernel for segmentation network

  // create multi-threading mode DPU application
  int thread_nums = 1;
  vector<DPUTask *> ssdtask(1);

  generate(ssdtask.begin(), ssdtask.end(), std::bind(dpuCreateTask, ssdkernel, 0));

  array<thread, 3> threads =
          {
                  // create 3 threads for running segmentation network on DPU
                  thread(RunSSD, ssdtask[0]),

                  // create thread for reading video frame
                  thread(frameReader),

                  // create one thread for displaying image
                  thread(imageDisplay)
          };

  for (int i = 0; i < 6; i++)
  {
    threads[i].join();
  }

  // release DPU Kernel & Task resources
  for_each(ssdtask.begin(), ssdtask.end(), dpuDestroyTask);
  dpuDestroyKernel(ssdkernel);

  dpuClose();

  return 0;
}

When I ran "make" on the target board, the line “if(!cap.isOpened())” is returning -1, which means that the script failed to open the camera.

I checked the video devices picked up the kernel by running “ls -ltrh /dev/video*”, and video0 shows up. I’ve even tried several different indices for using VideoCapture cam, e.g. cam(0), cam(1), cam(2), ….All fail to open the camera.

The webcam I use is Logitech HD Webcam C615 and is supported by the uvc driver: http://www.ideasonboard.org/uvc/#devices .

Any help is appreciated, thanks in advance,

miaoush

0 Kudos