Coral from Google - Camera Drivers - Troubleshooting

Coral from Google
Error creating thumbnail: Unable to save thumbnail to destination
Introduction
Getting Started SoM Overview
GStreamer
Example Pipelines
GstInference
Introduction Why use GstInference? Example Pipelines Demos
Camera Drivers
Introduction MIPI Camera connection How to flash Google Coral How to recompile kernel Adding a new camera driver Troubleshooting
Reference Documentation
Contact Us

Clock issues

This driver had a lot of issues with the clocks inside the board, after some testing of different clocks we discovered the different clocks that affect the camera capture:

Clock	Frequency	Effect
IMX8MQ_CLK_CSI1_CORE	250MHz	Affects the capture o bytes from the camera reducing the amount of jumps between values if increased
IMX8MQ_CLK_CSI1_PHY_REF	200MHz	Affects the writing of bytes to memory if increased too much the values get replicated.
IMX8MQ_CLK_CSI1_ESC	Not changed	Doesn't affect the frame captured.

To change the clocks you can go to these files:

drivers/clk/imx/clk-imx8mq.c
arch/arm64/boot/dts/freescale/fsl-imx8mq.dtsi

The .c file contains the code that defines the clocks and their address, this address needs to be concatenated with 3038 to access the register, when you see the base + <ADDRESS>, that address can be used to write the registers of the clocks to change them on the fly.

clks[IMX8MQ_CLK_CSI1_CORE] = imx8m_clk_composite("csi1_core", imx8mq_csi1_core_sels, base + 0xbd00);
clks[IMX8MQ_CLK_CSI1_PHY_REF] = imx8m_clk_composite("csi1_phy_ref", imx8mq_csi1_phy_sels, base + 0xbd80);
clks[IMX8MQ_CLK_CSI1_ESC] = imx8m_clk_composite("csi1_esc", imx8mq_csi1_esc_sels, base + 0xbe00);

Now we can look for the parent of the clock searching, for example, IMX8MQ_CLK_CSI1_CORE:

clk_set_parent(clks[IMX8MQ_CLK_CSI1_CORE], clks[IMX8MQ_SYS2_PLL_1000M]);
clk_set_parent(clks[IMX8MQ_CLK_CSI1_PHY_REF], clks[IMX8MQ_SYS2_PLL_1000M]);
clk_set_parent(clks[IMX8MQ_CLK_CSI1_ESC], clks[IMX8MQ_SYS1_PLL_800M]);

If we want to change the parent we need to look for possible parents searching for example csi1_core:

static const char *imx8mq_csi1_core_sels[] = {"osc_25m", "sys1_pll_266m", "sys2_pll_250m", "sys1_pll_800m",
					      "sys2_pll_1000m", "sys3_pll2_out", "audio_pll2_out", "video_pll1_out", };

static const char *imx8mq_csi1_phy_sels[] = {"osc_25m", "sys2_pll_125m", "sys2_pll_100m", "sys1_pll_800m",
					     "sys2_pll_1000m", "clk_ext2", "audio_pll2_out", "video_pll1_out", };

You can search for every parent name for the ENUM that represents them, and set it with the respective clk_set_parent for the clock that you want to modify.

To read the clock register you need the <ADDRESS> mentioned before and concatenate it with 0x3038, then you can do the following:

mendel@jumping-orange:~$ sudo memtool md -l 0x3038bd00
3038bd00: 14000007 14000007 14000007 14000007                ................ #This is the output

The output is going to display the values read in hexadecimal, being the first 2 bytes a representation of the parent and the last hex value being the divisor of the parent clock, the divisor is the hex value in decimal value plus 1.

Now, for example we want to divide a clock by four, you need to only modify the last hex value,writing:

sudo memtool mw -l 0x3038bd00 0x14000003

Now the parent clock is being divided by 4, allowing you to modify the clocks while the system is running.

To modify them in the dtb you can go to the following lines of code inside the .dtsi file mentioned before:

	mipi_csi_1: mipi_csi1@30a70000 {
		compatible = "fsl,mxc-mipi-csi2_yav";
		reg = <0x0 0x30a70000 0x0 0x1000>; /* MIPI CSI1 Controller base addr */
		interrupts = <GIC_SPI 44 IRQ_TYPE_LEVEL_HIGH>;
		clocks = <&clk IMX8MQ_CLK_DUMMY>,
				<&clk IMX8MQ_CLK_CSI1_CORE>,
				<&clk IMX8MQ_CLK_CSI1_ESC>,
				<&clk IMX8MQ_CLK_CSI1_PHY_REF>;
		clock-names = "clk_apb", "clk_core", "clk_esc", "clk_pxl";
		assigned-clocks = <&clk IMX8MQ_CLK_CSI1_CORE>,
				  <&clk IMX8MQ_CLK_CSI1_PHY_REF>,
				  <&clk IMX8MQ_CLK_CSI1_ESC>;
		assigned-clock-rates = <250000000>, <200000000>, <66000000>;
		power-domains = <&mipi_csi1_pd>;
		csis-phy-reset = <&src 0x4c 7>;
		phy-gpr = <&gpr 0x88>;
		status = "disabled";
	};

The system will automatically calculate the closest divisor to create the assigned clock rates. In this case, the clocks were set to allow captures of 1080HD 60fps RAW10 RGGB, you can test them individually to see how they affect the images.

Solving parent drivers isssues

When it comes to the IMX8 platform, sometimes the recycled drivers have some issues or don't support some image formats even though adding two lines solves the issue, so lets go through the modifications that had to be done to enable the driver controls and enable the format.

Enable formats

First,go to the file drivers/media/platform/mxc/capture/mx6s_capture.c and let us add the format that the sensor is going to use, to add it you just need to modify the following structure:

static struct mx6s_fmt formats[] = {
	{
		.name		= "UYVY-16",
		.fourcc		= V4L2_PIX_FMT_UYVY,
		.pixelformat	= V4L2_PIX_FMT_UYVY,
		.mbus_code	= MEDIA_BUS_FMT_UYVY8_2X8,
		.bpp		= 2,
	}, {
		.name		= "YUYV-16",
		.fourcc		= V4L2_PIX_FMT_YUYV,
		.pixelformat	= V4L2_PIX_FMT_YUYV,
		.mbus_code	= MEDIA_BUS_FMT_YUYV8_2X8,
		.bpp		= 2,
	}, {
		.name		= "YUV32 (X-Y-U-V)",
		.fourcc		= V4L2_PIX_FMT_YUV32,
		.pixelformat	= V4L2_PIX_FMT_YUV32,
		.mbus_code	= MEDIA_BUS_FMT_AYUV8_1X32,
		.bpp		= 4,
	}, {
		.name		= "RAWRGB8 (SBGGR8)",
		.fourcc		= V4L2_PIX_FMT_SBGGR8,
		.pixelformat	= V4L2_PIX_FMT_SBGGR8,
		.mbus_code	= MEDIA_BUS_FMT_SBGGR8_1X8,
		.bpp		= 1,
	}, { //Added format
                 //Name of the format
		.name		= "RAW10 (RGGB)", 
                //Format of the camera, in this case RAW SRGGB10
		.fourcc		= V4L2_PIX_FMT_SRGGB10, 
                //Format of the camera, in this case RAW SRGGB10
		.pixelformat	= V4L2_PIX_FMT_SRGGB10, 
                //Format set to the bus
		.mbus_code	= MEDIA_BUS_FMT_SRGGB10_1X10,
                //Bytes per pixel
		.bpp		= 2, 
	}
};

Now you need to add a line to the following method, why?, because for some reason its enabled by default, resulting in a switching line errors.

static int mx6s_csi_enable(struct mx6s_csi_dev *csi_dev)
{
	struct v4l2_pix_format *pix = &csi_dev->pix;
	unsigned long flags;
	unsigned long val;
	int timeout, timeout2;

	csi_dev->skipframe = 0;
	csisw_reset(csi_dev);

	if (pix->field == V4L2_FIELD_INTERLACED)
		csi_tvdec_enable(csi_dev, true);
	else
                //Add this line to disable the interlaced feature
		csi_tvdec_enable(csi_dev, false); 

Now we need to add to the following method the cases to enable the capture format:

static int mx6s_configure_csi(struct mx6s_csi_dev *csi_dev)
{
	struct v4l2_pix_format *pix = &csi_dev->pix;
	u32 cr1, cr18;
	u32 width;
	if (pix->field == V4L2_FIELD_INTERLACED) {
		csi_deinterlace_enable(csi_dev, true);
		csi_buf_stride_set(csi_dev, csi_dev->pix.width);
		csi_deinterlace_mode(csi_dev, csi_dev->std);
	} else {
		csi_deinterlace_enable(csi_dev, false);
		csi_buf_stride_set(csi_dev, 0);
	}

	switch (csi_dev->fmt->pixelformat) {
	case V4L2_PIX_FMT_YUV32:
	case V4L2_PIX_FMT_SBGGR8:
		width = pix->width;
		break;
//Set the width of the image, you need to add this case
	case V4L2_PIX_FMT_SRGGB10: 
		if (csi_dev->csi_mipi_mode == true){
			width = pix->width;
		}
			
		else{
			width = pix->width * 2;
		}
		/* For parallel 8-bit sensor input */
		break;
	case V4L2_PIX_FMT_UYVY:
	case V4L2_PIX_FMT_YUYV:
		if (csi_dev->csi_mipi_mode == true)
			width = pix->width;
			
		else
			/* For parallel 8-bit sensor input */
			width = pix->width * 2;
		break;
	default:
		pr_debug("   case not supported\n");
		return -EINVAL;
	}
	csi_set_imagpara(csi_dev, width, pix->height);

	if (csi_dev->csi_mipi_mode == true) {
		cr1 = csi_read(csi_dev, CSI_CSICR1);
		cr1 &= ~BIT_GCLK_MODE;
		csi_write(csi_dev, cr1, CSI_CSICR1);

		cr18 = csi_read(csi_dev, CSI_CSICR18);
		cr18 &= ~BIT_MIPI_DATA_FORMAT_MASK;
		cr18 |= BIT_DATA_FROM_MIPI;

		switch (csi_dev->fmt->pixelformat) {
		case V4L2_PIX_FMT_UYVY:
		case V4L2_PIX_FMT_YUYV:
			cr18 |= BIT_MIPI_DATA_FORMAT_YUV422_8B;
			break;
		case V4L2_PIX_FMT_SBGGR8:
			cr18 |= BIT_MIPI_DATA_FORMAT_RAW8;
			break;
//Enable the raw10 format to the bus
		case V4L2_PIX_FMT_SRGGB10:
			cr18 |= BIT_MIPI_DATA_FORMAT_RAW10; 
			break;
		default:
			pr_debug("   fmt not supported\n");
			return -EINVAL;
		}

		csi_write(csi_dev, cr18, CSI_CSICR18);
	}
	return 0;
}

Now your camera should capture without issues

Common errors

We don't have the exact output of the system, but sometimes, after installing a driver, the system fails to boot, not with a kernel panic or anything like that, it just stops working, we don't know exactly why this happens. Sometimes after adding a "print" to the driver, the system fails, and after removing it or moving it to a previous line it works. The only way to solve this is by reflashing the board using an already working image.