cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
bsarbb
Contributor
Contributor
315 Views
Registered: ‎03-11-2021

Vitis vision- BGR to NV12 conversion failure

Hello,

I am testing Vitis vision cvtcolor BGR2NV12 hardware function.

First of all;
In the following part of the test bench of BGR2NV12 hardware function, the code expects reference UV image.

    refimage1 = cv::imread(argv[3], -1);
    if (!refimage1.data) {
        fprintf(stderr, "Can't open UV ref image !!\n ");
        return -1;
    }

 

But there is no UV ref image in the "data" folder xilinx provided. Is there any way to create ref UV image from provided "ref_U.png", "ref_V.png" and "ref_Y.png" ? 

 

And the second question,

I combined Y plane and UV plane with the following changes:

diff --git a/home/sarslan/Desktop/dev-sources/Vitis_Libraries/vision/L2/examples/cvtcolor/xf_cvt_color_tb.cpp b/opt/Petalinux-2020.2/projects/hdsdi-rx-bgr/components/ext_sources/vitis/workspace/pure_bgr2nv12/src/xf_cvt_color_tb.cpp
index 776e00b..9312d19 100644
--- a/home/sarslan/Desktop/dev-sources/Vitis_Libraries/vision/L2/examples/cvtcolor/xf_cvt_color_tb.cpp
+++ b/opt/Petalinux-2020.2/projects/hdsdi-rx-bgr/components/ext_sources/vitis/workspace/pure_bgr2nv12/src/xf_cvt_color_tb.cpp
@@ -30,6 +30,7 @@ int main(int argc, char** argv) {
     cv::Mat error_img0, error_img1, error_img2;
     cv::Mat refimage, refimg0, refimg1, refimg2;
     cv::Mat refimage0, refimage1, refimage2;
+    cv::Mat outputimg;
 
     cv::Mat img;
 
@@ -2728,6 +2729,9 @@ int main(int argc, char** argv) {
     img_height = inputimg0.rows;
     img_width = inputimg0.cols;
 
+    ///////////////////////////////////////////////////////////////////
+    outputimg.create(cv::Size{WIDTH, (HEIGHT + (HEIGHT >> 1))}, CV_8UC1);
+
     /////////////////////////////////////// CL ////////////////////////
     int height = inputimg0.rows;
     int width = inputimg0.cols;
@@ -2777,15 +2781,24 @@ int main(int argc, char** argv) {
     diff_prof = end - start;
     std::cout << (diff_prof / 1000000) << "ms" << std::endl;
 
+#if (0)
     OCL_CHECK(err, err = q.enqueueReadBuffer(imageFromDevicey, CL_TRUE, 0, (newwidth_y * newheight_y),
                                              (ap_uint<OUTPUT_PTR_WIDTH>*)outputimg0.data));
     OCL_CHECK(err, err = q.enqueueReadBuffer(imageFromDeviceuv, CL_TRUE, 0, (newwidth_uv * newheight_uv * 2),
                                              (ap_uint<OUTPUT_PTR_WIDTH>*)outputimg1.data));
+#else
+    OCL_CHECK(err, err = q.enqueueReadBuffer(imageFromDevicey, CL_TRUE, 0, (newwidth_y * newheight_y),
+    											(uint8_t*)outputimg.data));
+    OCL_CHECK(err, err = q.enqueueReadBuffer(imageFromDeviceuv, CL_TRUE, 0, (newwidth_uv * newheight_uv * 2),
+    											(uint8_t*)outputimg.data + (HEIGHT * WIDTH)));
+#endif
 
     q.finish();
     printf("write output buffer\n");
     /////////////////////////////////////// end of CL /////////////////////
 
+#if (0)
+
     cv::imwrite("out_UV.png", outputimg1);
     cv::imwrite("out_Y.png", outputimg0);
 
@@ -2803,6 +2816,15 @@ int main(int argc, char** argv) {
 
     absdiff(outputimg0, refimage0, error_img0);
     absdiff(outputimg1, refimage1, error_img1);
+#else
+	{
+		int lFrameId = 1;
+		std::ostringstream filename;
+		filename << "Frame_" << lFrameId << ".raw";
+		std::ofstream outfile (filename.str().c_str(), std::ios::out | std::ios::binary);
+		outfile.write((char*)outputimg.data, (1920 * (1080 + (1080 >> 1))));
+	}
+#endif
 
 #endif
 #if RGB2NV21

With these changes, I created a cv::Mat that can contain the NV12 image, and passed the proper pointers that points to this cv::Mat's proper sections to the related OpenCL functions. Finally, I wrote this cv::Mat to file for viewing resulting raw image on the host.

I viewed resulting raw frame with "vooya" tool. Here is the input.jpg and the resulting NV12 raw frame:

input.jpg:

input.jpg

NV12 Raw Frame:
frame.png

So resulting Y plane of frame seems correct but UV plane of frame seems not correct.

Can you notice any bug in my changes ? 
Is there anyone try and verify BGR2NV12 hw function ? Is there any chance its a bug ?

Thanks for your help.

Tags (1)
0 Kudos
2 Replies
bsarbb
Contributor
Contributor
194 Views
Registered: ‎03-11-2021

The problem is in V component calculation. Here is the related function:

static uint8_t CalculateV(uint8_t R, uint8_t G, uint8_t B) {
// clang-format off
    #pragma HLS INLINE
    // clang-format on
    int32_t V = ((short int)R2V * R) + ((short int)G2V * G) + ((short int)B2V * B) + F_05;
    uint8_t Vvalue = saturate_cast(V, 128);
    return Vvalue;
}

I tried to do these calculations directly using float variables like :

	uint8_t Vint8;
	float V = (0.439f * R) - (0.368 * G) - (0.071 * B) + 128.f;
	int Vint32 = (int)V;
	if (Vint32 > 255) {
		Vint8 = 255;
	} else if (Vint32 < 0) {
		Vint8 = 0;
	} else {
		Vint8 = (int)Vint32;
	}
	return Vint8;

I saw that the resulting frame is correct with these change. When I use this transformation in streaming, the framerate decreased dramatically.

I can't find out why the preceeding CalculateV function not working as expected.

Do you notice any problem with CalculateV function ? 

By the way saturate_cast function called from the CalculateV function and here is its definition:

static uint8_t saturate_cast(int32_t Value, int32_t offset) {
    // Right shifting Value 15 times to get the integer part
	
    int Value_int = (Value >> 15) + offset;
    unsigned char Value_uchar = 0;
    if (Value_int > 255)
        Value_uchar = 255;
    else if (Value_int < 0)
        Value_uchar = 0;
    else
        Value_uchar = (uint8_t)Value_int;

    return Value_uchar;
}

Thanks for your help,

0 Kudos
bsarbb
Contributor
Contributor
190 Views
Registered: ‎03-11-2021

BTW, When I copy the original CalculateV function to other test project, compile and execute it for x86 and arm64 with predefined R,G,B values, the calculations works fine for both architectures.

So, I am not sure but, It seems that HLS related problem exists.

Thanks again.

0 Kudos