cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Highlighted
Observer
Observer
279 Views
Registered: ‎07-01-2019

Modify object detection algorithm to read from live camera

Hello community,

I have completed the ML-SSD-PASCAL-Caffe-Tutorial using DNNDK v3.0 package. Now I'm trying to change the script (ref_training/ZCU102_Application/src/main.cc) so that the application read frames from a live camera instead of an offline video. I tried to somewhat imitate what this guy did to modify pose example to use a live feed: https://www.hackster.io/adam-taylor/machine-learning-at-the-edge-with-xilinx-dnn-developer-kit-68c672

I modified the readFrame and main function to the following without being certain if it is correct:

void frameReader()
{
  //get timeStamp of start
  start_time = chrono::system_clock::now();
  // for profile mode, we don't read frame imagem from video
  if (modeFlag == "profile") return;
     
  VideoCapture cap(0);

  if(!cap.isOpened()) {
	return -1;
  }

	cap.set(CV_CAP_PROP_FRAME_WIDTH,640);
	cap.set(CV_CAP_PROP_FRAME_HEIGHT,320);

  while (1)
  {
    Mat img;
	// maximum iamge queue size 10
    if (queueInput.size() < 5)
    {
      if (!cap.read(img))
	{
	cout << "Finish reading the video." << end1;
	is_reading = false;
	break;
	}	
      // push a frame image into queue for afterwards processing
      mtxQueueInput.lock();
      queueInput.push(make_pair(idxInputImage++, img));
      mtxQueueInput.unlock();

    }
    else
    {
	  // sleep for 5ms if iamge queue is already full
      usleep(50000);
    }
  }
}
int main(int argc, char **argv)
{
  if ( argc != 4) {
    cout << "ERR: please specify running mode and window position" << endl;
    return 1;
  }
  
  // get mode flag: 
  // "profile" - for evaluate performance in FPS
  // "end2end" - for whole process including reading video frame image and displaying result
  modeFlag = argv[1];

  //set window position
  position_x = atoi(argv[2]);
  position_y = atoi(argv[3]);
  if (modeFlag != "end2end" && modeFlag != "profile" ) {
    return 0;
  }
  // open DPU device
  dpuOpen();
  DPUKernel *ssdkernel;

  ssdkernel = dpuLoadKernel(KRENEL_CONV);

  // load DPU Kernel for segmentation network

  // create multi-threading mode DPU application
  int thread_nums = 1;
  vector<DPUTask *> ssdtask(1);

  generate(ssdtask.begin(), ssdtask.end(), std::bind(dpuCreateTask, ssdkernel, 0));

  array<thread, 3> threads =
          {
                  // create 3 threads for running segmentation network on DPU
                  thread(RunSSD, ssdtask[0]),

                  // create thread for reading video frame
                  thread(frameReader),

                  // create one thread for displaying image
                  thread(imageDisplay)
          };

  for (int i = 0; i < 6; i++)
  {
    threads[i].join();
  }

  // release DPU Kernel & Task resources
  for_each(ssdtask.begin(), ssdtask.end(), dpuDestroyTask);
  dpuDestroyKernel(ssdkernel);

  dpuClose();

  return 0;
}

When I ran "make" on the target board, the line “if(!cap.isOpened())” is returning -1, which means that the script failed to open the camera.

I checked the video devices picked up the kernel by running “ls -ltrh /dev/video*”, and video0 shows up. I’ve even tried several different indices for using VideoCapture cam, e.g. cam(0), cam(1), cam(2), ….All fail to open the camera.

The webcam I use is Logitech HD Webcam C615 and is supported by the uvc driver: http://www.ideasonboard.org/uvc/#devices .

Any help is appreciated, thanks in advance,

miaoush

0 Kudos
0 Replies