Coral from Google - Camera Drivers - Adding a new camera driver
| Coral from Google | |
|---|---|
Error creating thumbnail: Unable to save thumbnail to destination  | |
| Introduction | |
|
|
|
| GStreamer | |
|
|
|
| GstInference | |
|
|
|
| Camera Drivers | |
|
|
|
| Reference Documentation | |
| Contact Us |
For the device tree one of the files that you have to modify is the one called arch/arm64/boot/dts/freescale/fsl-imx8mq-phanbell.dts, this file is where you should add the camera dtb.
An example can be seen here:
&i2c2 {
status = "okay";
imx327: imx327@1a { //this changes with the camera, use the respective @ value
compatible = "imx327";
reg = <0x1a>; // Register value for the camera
status = "okay";
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_csi1>;
clocks = <&clk IMX8MQ_CLK_DUMMY>;
clock-names = "csi_mclk";
csi_id = <0>;
pwdn-gpios = <&gpio3 8 1>;
rst-gpios = <&gpio1 12 1>;
port {
imx327_port: endpoint {
remote-endpoint = <&mipi1_sensor_ep>;
data-lanes = <1 2>;
clock-lanes = <1>;
};
};
};
};
// Add it to the mipi csi
&mipi_csi_1 {
#address-cells = <1>;
#size-cells = <0>;
status = "okay";
port {
mipi1_sensor_ep: endpoint1 {
remote-endpoint = <&imx327_port>; //add it to the endpoint
data-lanes = <1 2>;
clock-lanes = <1>;
};
csi1_mipi_ep: endpoint2 {
remote-endpoint = <&csi1_ep>;
};
};
};
You usually find an OV camera here, but you can just replace it with your camera DTB. But this is not enough to make the camera fully work sometimes, but this is going to be discussed later.
Now, to add the driver code you need to add the .c and .h files to this directory drivers/media/platform/mxc/capture/ , in this case, we add the files imx327_mipi.c and imx327_mipi.h to the directory.
In the same directory you are going to find a makefile and kconfig file, to the makefile add the following line:
obj-$(CONFIG_VIDEO_IMX327_MIPI) += imx327_mipi.o
To the kconfig add the following lines:
config VIDEO_IMX327_MIPI tristate "IMX327 camera sensor support" depends on I2C && VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API ---help--- This is a Video4Linux2 sensor-level driver for the IMX327 camera. To compile this driver as a module, choose M here: the module will be called imx334.
Now you need to enable the driver, this can vary from system to system, so you need to sometimes run make menuconfig and sometimes you need to add to the defconfig file the following line:
CONFIG_VIDEO_IMX327_MIPI=y
Now you can compile the packages to install the driver
Enable camera controls
To enable camera controls you need to add the following code to the driver.
Add this to the camera structure, you need a control handler structure, it creates, in the long run, a hashmap with all the control functions.
struct v4l2_ctrl_handler ctrl_handler;
When it comes to functions, you can add the following functions to the driver
//Call used by v4l2 to set the different controls of the camera
static int imx327_s_ctrl(struct v4l2_ctrl *ctrl)
{
struct imx327 * sensor = container_of(ctrl->handler, struct imx327, ctrl_handler); // get the camera structure
int err = 0;
s32 value = ctrl->val; // get the value of the control
switch (ctrl->id) { //switch with the control ID
case V4L2_CID_GAIN:
//Here es where you would set the value, in this case we call another function
err = imx327_set_gain(value,sensor);
break;
case V4L2_CID_EXPOSURE: //same as before
err = imx327_set_exposure(value,sensor);
break;
}
return err;
}
//This sets the function that is called when the controls are trying to be set from v4l2
//You can also implement the get function, but is not necessary
static const struct v4l2_ctrl_ops imx327_ctrl_ops = {
.s_ctrl = imx327_s_ctrl,
};
//This function sets the different control functions,
you can set them in the probe function or create their own function
static int imx327_setcontrols(struct imx327 *sensor){
int err =0;
// this inits the control handler that you added to your camera structure,
the number is the amount of controls you want to enable
v4l2_ctrl_handler_init(&sensor->ctrl_handler, 2);
//This sets the control handler of the subdev structure and is needed
sensor->subdev.ctrl_handler = &sensor->ctrl_handler;
//This is how you enable a new control, the arguments are, the control of the structure, the ctrl_ops structure,
the ID of the control, the minimum , maximum, step and default values
v4l2_ctrl_new_std(&sensor->ctrl_handler, &imx327_ctrl_ops,
V4L2_CID_EXPOSURE, MIN_EXPOSURE, MAX_EXPOSURE,
EXPOSURE_STEP, DEFAULT_EXPOSURE);
v4l2_ctrl_new_std(&sensor->ctrl_handler, &imx327_ctrl_ops,
V4L2_CID_GAIN, MIN_GAIN, MAX_GAIN,
GAIN_STEP, DEFAULT_GAIN);
//Check if the controls were enabled correctly
if (sensor->ctrl_handler.error < 0) {
err = sensor->ctrl_handler.error;
v4l2_ctrl_handler_free(&sensor->ctrl_handler);
return err;
}
return err;
}
Now we need to expose the driver as a subdevice, but for some reason, the parent driver doesn't register its subdevs, so we can do the following.
Add this to the end of the probe function, after setting the controls and before registering the subdevices:
//Enable the subdev flag
sensor->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
sensor->subdev.grp_id = GROUP_ID; //this value should be 678
//Just before this
retval = v4l2_async_register_subdev(&sensor->subdev);
Now we have to modify the parent driver, to enable the subdev to be exposed, we need to add the registering of the subdevs in a callback function, adding it there allows the parent driver to register the subdev after the first stream command.
First lets add a bool to the structure of the parent in the file drivers/media/platform/imx8/mxc-mipi-csi2.h, to not "register" every time we start the stream:
struct mxc_mipi_csi2_dev {
struct v4l2_device v4l2_dev;
struct v4l2_subdev sd;
struct v4l2_subdev *sensor_sd;
struct media_pad pads[MXC_MIPI_CSI2_VCX_PADS_NUM];
struct v4l2_mbus_framefmt format;
void __iomem *csr_regs;
void __iomem *base_regs;
struct platform_device *pdev;
u32 flags;
int irq;
struct clk *clk_apb;
struct clk *clk_core;
struct clk *clk_esc;
struct clk *clk_pxl;
struct csis_hw_reset hw_reset;
struct csis_phy_gpr phy_gpr;
struct v4l2_async_subdev asd;
struct v4l2_async_notifier subdev_notifier;
struct v4l2_async_subdev *async_subdevs[2];
struct mutex lock;
int id;
u32 hs_settle;
u32 send_level;
u32 num_lanes;
u8 data_lanes[4];
u8 vchannel;
u8 running;
//This line
bool registered;
};
Now we just need to modify the next function in drivers/media/platform/imx8/mxc-mipi-csi2_yav.c:
static int mipi_csi2_s_stream(struct v4l2_subdev *sd, int enable)
{
struct mxc_mipi_csi2_dev *csi2dev = sd_to_mxc_mipi_csi2_dev(sd);
struct device *dev = &csi2dev->pdev->dev;
struct v4l2_subdev *sensor_sd = csi2dev->sensor_sd;
int ret = 0;
dev_dbg(&csi2dev->pdev->dev, "%s: %d, csi2dev: 0x%x\n",
__func__, enable, csi2dev->flags);
// TOOO: The below code only works for one CSI camera attached. The old
// code was broken for OpenCV, and not according to V4L2 spec, both for the
// one and two camera cases and this is a temp fix that takes care of the
// one camera case See b/128424247
if (enable) {
if (!csi2dev->running) {
pm_runtime_get_sync(dev);
mxc_mipi_csi2_phy_reset(csi2dev);
mxc_mipi_csi2_hc_config(csi2dev);
mxc_mipi_csi2_enable(csi2dev);
mxc_mipi_csi2_reg_dump(csi2dev);
}
v4l2_subdev_call(sensor_sd, video, s_stream, true);
csi2dev->running = 1;
} else if (csi2dev->running){
v4l2_subdev_call(sensor_sd, video, s_stream, false);
csi2dev->running = 0;
pm_runtime_put(dev);
mxc_mipi_csi2_disable(csi2dev);
}
dev_dbg(&csi2dev->pdev->dev, "%s: %d, csi2dev: 0x%x\n",
__func__, enable, csi2dev->flags);
//This was the section of the code added
if(csi2dev->registered == false)
{
v4l2_info(&csi2dev->v4l2_dev, "Registering subdevice nodes");
//This line register all the subdevice nodes, creating a subdevice file
ret = v4l2_device_register_subdev_nodes(&csi2dev->v4l2_dev);
if(ret < 0 ){
v4l2_info(&csi2dev->v4l2_dev, "Error registering subdevice nodes");
return -EINVAL;
}
csi2dev->registered = true;
}
return ret;
}
