UsingISC (Image Sensor Controller)
Introduction
This page describes how to configure and use of the ISC in SAMA5D2 SoC with the image sensors.
- The Image Sensor Controller(ISC) manages incoming data from a sensor up to 12-bit parallel interface.
- The ISC can generate the isc_mck output clock to the image sensor.
- The isc_mck clock has three selectable clock sources and one programmable clock divider.
- The ISC can handle pixel clock up to 96MHz, with the maximum resolution of 2592x1944.
- The ISC includes a video pipeline consist of the following sub-modules:
- PFE: Parallel Front End to sample the camera sensor input stream
- WB: Programmable white balance in the Bayer domain
- CFA: Color filter array interpolation module
- CC: Programmable color correction
- GAM: Gamma correction
- CSC: Programmable color space conversion
- CBC: Contrast and Brightness control
- SUB: This module performs YCbCr444 to YCbCr420 chrominance subsampling
- RLP: This module performs rounding, range limiting and packing of the incoming data
- The sub-modules in video pipeline can handle the following input pixel formats:
- Raw Bayer convert to YCbCr
- YCbCr (YCbCr 444, YCbCr 422, YCbCr 420),
- Luminance (Black and White) Pixel Format.
- 16 bpp RGB, 32 bpp RGB, 16 bpp.
- For output format,
- For YCbCr output, the ISC includes a DMA interface which allows YCbCr data organized in packed, semi planar or planar format.
- It can also convert YCbCr to RGB.
- It can also be configured as by pass mode, to store original data received from the sensor.
Prerequisites
Starting with
Linux Kernel 4.9
, we provide and support kernel drivers for ISC.
Starting with
Linux Kernel 4.14 (included in Linux4SAM 6.0) , we offer the Device Tree support for the ISC and sensors as an Overlay for the
SAMA5D2 Xplained board only. Overlays are required for extension boards, like camera sensors in our case. For more information, visit the
DT-overlays page.
Starting with Linux4SAM 6.2 , we offer the Device Tree support for the ISC and sensors as an Overlay for the
SAMA5D2 Xplained,
SAMA5D27 SOM1 EK and
SAMA5D27 WLSOM1 EK . Overlays are required for extension boards, like camera sensors in our case. For more information, visit the
DT-overlays page.
Starting with Linux4SAM 2020.04 , we offer additional support for the
MT9V022 monochrome Aptina sensor.
Advanced features
Starting with
Linux Kernel 4.19 (included in Linux4SAM 6.1) , we offer extended support for the
White Balance features.
Configuration in kernel
The ISC driver in Linux kernel provides the upper layer interface to the Video4Linux (V4L2) framework.
To make it work, you should enable the following options in Kernel menuconfig.
- Enable V4L2 framework.
- Under the menu: "Device Drivers ---> Multimedia support"
- Select the menu "Device Drivers ---> Multimedia support ---> Media Controller API"
- Select the menu "Device Drivers ---> Multimedia support ---> V4L2 sub-device userspace API"
- Deselect the menu "Device Drivers ---> Multimedia support ---> Autoselect ancillary drivers (tuners, sensors, i2c, spi, frontends)"
- Select device driver
- V4L platform device can be found in the menu: "Device Drivers ---> Multimedia support ---> V4L platform devices"
- Deselect the menu "Device Drivers ---> Multimedia support ---> V4L platform devices ---> SoC camera support"
- Select the menu "Device Drivers ---> Multimedia support ---> V4L platform devices ---> ATMEL Image Sensor Controller (ISC) support"
- Select sensor driver
- Sensor drivers can be found in the menu "Device Drivers ---> Multimedia support ---> I2C Encoders, decoders, sensors and other helper chips "
- We have tested with OV7670 , OV7740 and OV5640 image sensors. Starting with Linux4SAM 2020.04 we also tested with mt9v022
Configuration in U-boot
Starting with
Linux Kernel 4.14 (included in Linux4SAM 6.0) and U-boot 2018.07 , we offer the Device Tree support for the ISC and sensors as an Overlay for
SAMA5D2 Xplained SAMA5D27 SOM1 EK and
SAMA5D27 WLSOM1 EK . Overlays are required for extension boards, like camera sensors in our case. For more information, visit the
DT-overlays page.
U-boot, if instructed to do so, will load the Device Tree Overlay for the Image sensor controller (ISC) and the Device Tree Overlay for the specific sensor which we support.
To instruct U-boot to load the additional required Device Tree Overlays, proceed with the following steps:
- Step 1: The boot command needs to have an additional overlay for the ISC controller itself. This is an additional FIT image overlay configuration:
bootm 0x24000000#kernel_dtb#isc
- Step 2: The boot command needs to have an additional overlay for the sensor. This is an additional FIT image overlay configuration:
bootm 0x24000000#kernel_dtb#isc#ov7670
or
bootm 0x24000000#kernel_dtb#isc#ov7740
or
bootm 0x24000000#kernel_dtb#isc#ov5640
or
bootm 0x24000000#kernel_dtb#isc#mt9v022
Note that both overlay configurations are required, chained as in the command in step 2 above. the 'isc' overlay will add the setup for the ISC, and the ov7670/ov7740/ov5640/mt9v022 will add the setup for the sensor itself.
The following table summarizes the possible FIT image overlay configurations that are needed:
|
|
| |
Overlay configuration |
Effect |
Requirements |
Starting with Linux4SAM version |
| Image Sensor Controller |
isc |
Adds the Image Sensor Controller setup configuration |
Base Kernel device Tree Blob 'kernel_dtb' configuration |
Linux4SAM 6.0 |
| Omnivision ov7670 |
ov7670 |
Adds the Omnivision ov7670 setup configuration |
Base Kernel device Tree Blob 'kernel_dtb' configuration and 'isc' configuration |
Linux4SAM 6.0 |
| Omnivision ov7740 |
ov7740 |
Adds the Omnivision ov7740 setup configuration |
Base Kernel device Tree Blob 'kernel_dtb' configuration and 'isc' configuration |
Linux4SAM 6.0 |
| Omnivision ov5640 |
ov5640 |
Adds the Omnivision ov5640 setup configuration |
Base Kernel device Tree Blob 'kernel_dtb' configuration and 'isc' configuration |
Linux4SAM 6.1 |
| Aptina MT9V022 |
mt9v022 |
Adds the Aptina mt9v022 setup configuration |
Base Kernel device Tree Blob 'kernel_dtb' configuration and 'isc' configuration |
Linux4SAM 2020.04 |
|
|
|
Note that adding just an overlay for the sensor is
not enough. The 'isc' overlay is a
mandatory prerequisite .
Note that
SAMA5D2 Xplained board is used as a reference, and the software support is created and tested for this board. The SAMA5D2 SoC supports ISC, but to have it up and running, please check your board schematics.
This support is only available starting with
Linux Kernel 4.19 (included in Linux4SAM 6.1)
- Step 1: The boot command needs to have an additional overlay for the sensor. This is an additional FIT image overlay configuration:
bootm 0x24000000#kernel_dtb#ov5640
or
bootm 0x24000000#kernel_dtb#isc#mt9v022
Note that only the support for ov5640 and mt9v022 are currently available.
The following table summarizes the possible FIT image overlay configurations that are needed:
|
|
| |
Overlay configuration |
Effect |
Requirements |
Starting with Linux4SAM version |
| Omnivision ov5640 |
ov5640 |
Adds the Omnivision ov5640 setup configuration |
Base Kernel device Tree Blob 'kernel_dtb' configuration |
Linux4SAM 6.1 |
| Aptina MT9V022 |
mt9v022 |
Adds the Aptina mt9v022 setup configuration |
Base Kernel device Tree Blob 'kernel_dtb' configuration and 'isc' configuration |
Linux4SAM 2020.04 |
|
|
|
Note that the connector on this board
is different from the one on
SAMA5D2 Xplained. Different wiring/extension board are required .
Note that
SAMA5D27 SOM1 EK board is used as a reference, and the software support is created and tested for this board. The SAMA5D2 SoC supports ISC, but to have it up and running, please check your board schematics.
This support is only available starting with Linux4SAM
6.2
* Step 1: The boot command needs to have an additional overlay for the ISC controller itself. This is an additional FIT image overlay configuration:
bootm 0x24000000#kernel_dtb#isc
- Step 2: The boot command needs to have an additional overlay for the sensor. This is an additional FIT image overlay configuration:
bootm 0x24000000#kernel_dtb#isc#ov7670
or
bootm 0x24000000#kernel_dtb#isc#ov7740
or
bootm 0x24000000#kernel_dtb#isc#ov5640
or
bootm 0x24000000#kernel_dtb#isc#mt9v022
Note that both overlay configurations are required, chained as in the command in step 2 above. the 'isc' overlay will add the setup for the ISC, and the ov7670/ov7740/ov5640/mt9v022 will add the setup for the sensor itself.
The following table summarizes the possible FIT image overlay configurations that are needed:
|
|
| |
Overlay configuration |
Effect |
Requirements |
Starting with Linux4SAM version |
| Image Sensor Controller |
isc |
Adds the Image Sensor Controller setup configuration |
Base Kernel device Tree Blob 'kernel_dtb' configuration |
Linux4SAM 6.2 |
| Omnivision ov7670 |
ov7670 |
Adds the Omnivision ov7670 setup configuration |
Base Kernel device Tree Blob 'kernel_dtb' configuration and 'isc' configuration |
Linux4SAM 6.2 |
| Omnivision ov7740 |
ov7740 |
Adds the Omnivision ov7740 setup configuration |
Base Kernel device Tree Blob 'kernel_dtb' configuration and 'isc' configuration |
Linux4SAM 6.2 |
| Omnivision ov5640 |
ov5640 |
Adds the Omnivision ov5640 setup configuration |
Base Kernel device Tree Blob 'kernel_dtb' configuration and 'isc' configuration |
Linux4SAM 6.2 |
| Aptina MT9V022 |
mt9v022 |
Adds the Aptina mt9v022 setup configuration |
Base Kernel device Tree Blob 'kernel_dtb' configuration and 'isc' configuration |
Linux4SAM 2020.04 |
|
|
|
Note that the connector on this board
is different from the one on
SAMA5D2 Xplained. Different wiring/extension board are required .
Note that
SAMA5D27 WLSOM1 EK board is used as a reference, and the software support is created and tested for this board. The SAMA5D2 SoC supports ISC, but to have it up and running, please check your board schematics.
Connect the sensor to ISC
Note that the connector on
SAMA5D27 SOM1 EK is different from the one on
SAMA5D2 Xplained. Different wiring/extension board are required .
SAMA5D2 Xplained includes an image sensor connector:
It uses two peripherals from SAMA5D2:
- ISC: The DVP interface consists of:
- Pixel Clock (VSYNC).
- Horizontal Sync (HSYNC).
- Vertical Sync (VSYNC).
- 8 to 12-bits parallel data interface
- TWI: an I2C interface to control the sensor's behavior.
- And reset and power down control of the sensor.
The ISC driver uses the device tree to specify the ISC and sensor device, please refer to the binding document
atmel-isc.txt.
The binding document explains:
- How the DVP interface of the sensor is connected to SAMA5D2.
- How the sensor's I2C control channel is connected to SAMA5D2.
Note that the hardware connections are highly dependent on your board. Check your board schematics to see how the wiring is done.
Starting with
Linux Kernel 4.14 (included in Linux4SAM 6.0) and U-boot 2018.07 , we offer the Device Tree support for the ISC and sensors as an Overlay for
SAMA5D2 Xplained. Overlays are required for extension boards, like camera sensors in our case. For more information, visit the
DT-overlays page.
The DT-overlay for the ISC and for the sensors can be consulted in
our Github repository.
Here are links to the overlays themselves, for the
SAMA5D2 Xplained board:
Starting with
Linux Kernel 4.19 (included in Linux4SAM 6.1) we offer additional support for omnivision ov5640 for
SAMA5D2 Xplained but also for
SAMA5D27 SOM1 EK.
Here are links to the overlay itself, for the
SAMA5D2 Xplained board:
Here are links to the overlay itself, for the
SAMA5D27 SOM1 EK board:
Starting with Linux4SAM 6.2 we offer support via overlays for all the three sensors (ov7670, ov5640, ov7740) for the
SAMA5D27 WLSOM1 EK board.
Starting with Linux4SAM 2020.04 we offer support for the Aptina sensor monochrome mt9v022 for all the three boards.
If everything is okay, you can see the log while the system boot-up, it indicates the ov7670 sensor successfully probed.
ov7670 2-0021: chip found @ 0x42 (AT91)
After boot up, you can see a character device file named
/dev/video:
root@sama5d2-xplained:/# ls -l /dev/video0
crw-rw---- 1 root video 81, 1 Jul 12 11:56 /dev/video0
The more information is in the sysfs filesystem.
root@sama5d2-xplained:/# ls sys/class/video4linux/video0
dev dev_debug device index name power subsystem uevent
root@sama5d2-xplained:/# cat /sys/class/video4linux/video0/name
atmel_isc
root@sama5d2-xplained:/# cat /sys/class/video4linux/v4l-subdev0/name
ov7670 2-0021
User space software
Since linux4sam-2022.04 , using
Linux Kernel 5.15 , userspace needs to use the media controller paradigm of configuring the image capture pipeline.
without properly setting up the media controller entities and links, image capture will fail.
the ISC driver will no longer setup the sensor with a corresponding format, as in the media controller paradigm, this is no longer possible.
The v4l-utils are a series of packages for handling the video4linux devices.
v4L2-compliance
You can use v4l2-compliance tool to test the v4l2 API compliance of the ISC driver.
# v4l2-compliance
v4l2-compliance 1.22.1, 32 bits, 32-bit time_t
Compliance test for microchip-isc device /dev/video0:
Driver Info:
Driveratmel- : microchip-isc
Card type : Atmel Image Sensor Controller
Bus info : platform:atmel-sama5d2-isc f000
Driver version : 5.15.32
Capabilities : 0xa4200001
Video Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x24200001
Video Capture
Streaming
Extended Pix Format
Media Driver Info:
Driver name : atmel_isc_commo
Model : atmel,sama5d2-isc
Serial :
Bus info : platform:atmel-sama5d2-isc f000
Media version : 5.15.32
Hardware revision: 0x00000137 (311)
Driver version : 5.15.32
Interface Info:
ID : 0x0300000c
Type : V4L Video
Entity Info:
ID : 0x0000000a (10)
Name : atmel_isc_common
Function : V4L2 I/O
Flags : default
Pad 0x0100000b : 0: Sink
Link 0x02000010: from remote pad 0x1000003 of entity 'atmel_isc_scaler' (Video Scaler): Data, Enabled, Immutable
Required ioctls:
test MC information (see 'Media Driver Info' above): OK
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/video0 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
sama5d2-isc f0008000.isc: ================= START STATUS =================
atmel-sama5d2-isc f0008000.isc: Contrast: 256
atmel-sama5d2-isc f0008000.isc: Brightness: 0
tatmefor unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
l-sama5d2-isc f0008000.isc: Gamma: 2
atmel-sama5d2-isc f0008000.isc: White Balance, Automatic: true
atmel-sama5d2-isc f0008000.isc: Red Component Gain: 512 inactive volatile
atmel-sama5d2-isc f0008000.isc: Blue Component Gain: 512 inactive volatile
atmel-sama5d2-isc f0008000.isc: Green Red Component Gain: 512 inactive volatile
atmel-sama5d2-isc f0008000.isc: Green Blue Component Gain: 512 inactive volatile
atmel-sama5d2-isc f0008000.isc: Red Component Offset: 0 inactive volatile
atmel-sama5d2-isc f0008000.isc: Blue Component Offset: 0 inactive volatile
atmel-sama5d2-isc f0008000.isc: Green Red Component Offset: 0 inactive volatile
atmel-sama5d2-isc f0008000.isc: Green Blue Component Offset: 0 inactive volatile
atmel-sama5d2-isc f0008000.isc: ================== END STATUS ==================
test VIDIOC_LOG_STATUS: OK
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 1 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls (Input 0):
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 6 Private Controls: 8
Format ioctls (Input 0):
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK
Codec ioctls (Input 0):
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls (Input 0):
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
test TIME32/64: OK
Total for microchip-isc device /dev/video0: 47, Succeeded: 47, Failed: 0, Warnings: 0
v4l2-ctl
- Display all information available.
# v4l2-ctl --all
Driver Info:
Driver name : microchip-isc
Card type : Atmel Image Sensor Controller
Bus info : platform:atmel-sama5d2-isc f000
Driver version : 5.15.32
Capabilities : 0xa4200001
Video Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x24200001
Video Capture
Streaming
Extended Pix Format
Media Driver Info:
Driver name : atmel_isc_commo
Model : atmel,sama5d2-isc
Serial :
Bus info : platform:atmel-sama5d2-isc f000
Media version : 5.15.32
Hardware revision: 0x00000137 (311)
Driver version : 5.15.32
Interface Info:
ID : 0x0300000c
Type : V4L Video
Entity Info:
ID : 0x0000000a (10)
Name : atmel_isc_common
Function : V4L2 I/O
Flags : default
Pad 0x0100000b : 0: Sink
Link 0x02000010: from remote pad 0x1000003 of entity 'atmel_isc_scaler' (Video Scaler): Data, Enabled, Immutable
Priority: 2
Video input : 0 (atmel_isc_common: ok)
Format Video Capture:
Width/Height : 640/480
Pixel Format : 'AR12' (16-bit ARGB 4-4-4-4)
Field : None
Bytes per Line : 1280
Size Image : 614400
Colorspace : sRGB
Transfer Function : Default (maps to sRGB)
YCbCr/HSV Encoding: Default (maps to ITU-R 601)
Quantization : Default (maps to Full Range)
Flags :
User Controls
brightness 0x00980900 (int) : min=-1024 max=1023 step=1 default=0 value=0 flags=slider
contrast 0x00980901 (int) : min=-2048 max=2047 step=1 default=256 value=256 flags=slider
white_balance_automatic 0x0098090c (bool) : default=1 value=1 flags=update
do_white_balance 0x0098090d (button) : value=0 flags=inactive, write-only, volatile, execute-on-write
gamma 0x00980910 (int) : min=0 max=2 step=1 default=2 value=2 flags=slider
red_component_gain 0x009819c0 (int) : min=0 max=8191 step=1 default=512 value=512 flags=inactive, slider, volatile
blue_component_gain 0x009819c1 (int) : min=0 max=8191 step=1 default=512 value=512 flags=inactive, slider, volatile
green_red_component_gain 0x009819c2 (int) : min=0 max=8191 step=1 default=512 value=512 flags=inactive, slider, volatile
green_blue_component_gain 0x009819c3 (int) : min=0 max=8191 step=1 default=512 value=512 flags=inactive, slider, volatile
red_component_offset 0x009819c4 (int) : min=-4095 max=4095 step=1 default=0 value=0 flags=inactive, slider, volatile
blue_component_offset 0x009819c5 (int) : min=-4095 max=4095 step=1 default=0 value=0 flags=inactive, slider, volatile
green_red_component_offset 0x009819c6 (int) : min=-4095 max=4095 step=1 default=0 value=0 flags=inactive, slider, volatile
green_blue_component_offset 0x009819c7 (int) : min=-4095 max=4095 step=1 default=0 value=0 flags=inactive, slider, volatile
#
- Display all controls and their values
# v4l2-ctl --list-ctrls
User Controls
brightness 0x00980900 (int) : min=-1024 max=1023 step=1 default=0 value=0 flags=slider
contrast 0x00980901 (int) : min=-2048 max=2047 step=1 default=256 value=256 flags=slider
white_balance_automatic 0x0098090c (bool) : default=1 value=1 flags=update
do_white_balance 0x0098090d (button) : value=0 flags=inactive, write-only, volatile, execute-on-write
gamma 0x00980910 (int) : min=0 max=2 step=1 default=2 value=2 flags=slider
red_component_gain 0x009819c0 (int) : min=0 max=8191 step=1 default=512 value=512 flags=inactive, slider, volatile
blue_component_gain 0x009819c1 (int) : min=0 max=8191 step=1 default=512 value=512 flags=inactive, slider, volatile
green_red_component_gain 0x009819c2 (int) : min=0 max=8191 step=1 default=512 value=512 flags=inactive, slider, volatile
green_blue_component_gain 0x009819c3 (int) : min=0 max=8191 step=1 default=512 value=512 flags=inactive, slider, volatile
red_component_offset 0x009819c4 (int) : min=-4095 max=4095 step=1 default=0 value=0 flags=inactive, slider, volatile
blue_component_offset 0x009819c5 (int) : min=-4095 max=4095 step=1 default=0 value=0 flags=inactive, slider, volatile
green_red_component_offset 0x009819c6 (int) : min=-4095 max=4095 step=1 default=0 value=0 flags=inactive, slider, volatile
green_blue_component_offset 0x009819c7 (int) : min=-4095 max=4095 step=1 default=0 value=0 flags=inactive, slider, volatile
#
- Get / Set the value of the controls
# v4l2-ctl --get-ctrl=contrast
contrast: 256
# v4l2-ctl --set-ctrl=contrast=200
# v4l2-ctl --get-ctrl=contrast
contrast: 200
#
Media-ctl is a tool that is used to query and set media controller pipeline (entities, links, etc.). More information about the media controller and a tutorial about it can be found here
here.
Example of media-ctl -p output with mt9v022 sensor connected
# media-ctl -p
Media controller API version 5.15.32
Media device information
------------------------
driver atmel_isc_commo
model atmel,sama5d2-isc
serial
bus info platform:atmel-sama5d2-isc f000
hw revision 0x137
driver version 5.15.32
Device topology
- entity 1: atmel_isc_scaler (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev0
pad0: Sink
[fmt:SBGGR8_1X8/2592x1944 field:none colorspace:srgb
crop.bounds:(0,0)/2592x1944
crop:(0,0)/2592x1944]
<- "mt9v032 1-0048":0 [ENABLED,IMMUTABLE]
pad1: Source
[fmt:SBGGR8_1X8/2592x1944 field:none colorspace:srgb]
-> "atmel_isc_common":0 [ENABLED,IMMUTABLE]
- entity 4: mt9v032 1-0048 (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev1
pad0: Source
[fmt:Y10_1X10/752x480 field:none colorspace:srgb
crop:(1,5)/752x480]
-> "atmel_isc_scaler":0 [ENABLED,IMMUTABLE]
- entity 10: atmel_isc_common (1 pad, 1 link)
type Node subtype V4L flags 1
device node name /dev/video0
pad0: Sink
<- "atmel_isc_scaler":1 [ENABLED,IMMUTABLE]
#
Example of media-ctl -p output with ov7740 sensor connected
# media-ctl -p
Media controller API version 5.18.0
Media device information
------------------------
driver atmel_isc_commo
model atmel,sama5d2-isc
serial
bus info platform:atmel-sama5d2-isc f000
hw revision 0x137
driver version 5.18.0
Device topology
- entity 1: atmel_isc_scaler (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev0
pad0: Sink
[fmt:SBGGR8_1X8/2592x1944 field:none colorspace:srgb
crop.bounds:(0,0)/2592x1944
crop:(0,0)/2592x1944]
<- "ov7740 1-0021":0 [ENABLED,IMMUTABLE]
pad1: Source
[fmt:SBGGR8_1X8/2592x1944 field:none colorspace:srgb]
-> "atmel_isc_common":0 [ENABLED,IMMUTABLE]
- entity 4: ov7740 1-0021 (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev1
pad0: Source
[fmt:YUYV8_2X8/640x480@1/60 field:none colorspace:srgb]
-> "atmel_isc_scaler":0 [ENABLED,IMMUTABLE]
- entity 10: atmel_isc_common (1 pad, 1 link)
type Node subtype V4L flags 1
device node name /dev/video0
pad0: Sink
<- "atmel_isc_scaler":1 [ENABLED,IMMUTABLE]
#
- Video-capture-at91 is a collection of scripts present on Github that is part of the Linux4SAM distribution, and offers examples and scripts that automatically configure the media controller pipeline.
- The scripts are available in your home directory, and are organized in directories corresponding to each sensor supported.
- There is a directory for each of the supported sensors:
# ls -la video-capture-at91/
total 24
drwxr-xr-x 6 root root 4096 May 10 2022 .
drwx------ 3 root root 4096 Jan 1 02:08 ..
drwxr-xr-x 2 root root 4096 May 10 2022 mt9v022
drwxr-xr-x 2 root root 4096 May 10 2022 ov5640
drwxr-xr-x 2 root root 4096 May 10 2022 ov7670
drwxr-xr-x 2 root root 4096 May 10 2022 ov7740
- There is a script that prepares the media controller pipeline for capturing for each supported resolution:
- There are also demo scripts available for each resolution, on how to use fswebcam to capture video in different formats, and how to configure the video format accordingly.
- Example on how to use scripts for sama5d2 + mt9v022 sensor:
# ./video-capture-at91/mt9v022/
752x480_Y10.sh fswebcam_752x480.sh
# ./video-capture-at91/mt9v022/752x480_Y10.sh
Preparing MT9V022 in Y10 10bits mode
Ready to capture at 752x480
#
#
# ./video-capture-at91/mt9v022/fswebcam_752x480.sh
--- Opening /dev/video0...
Trying source module v4l2...
/dev/video0 opened.
No input was specified, using the first.
--- Capturing frame...
Skipping 20 frames...
Capturing 1 frames...
Captured 21 frames in 0.37 seconds. (56 fps)
--- Processing captured image...
Writing JPEG image to 'tinyGREY.png'.
#
- Example on how to use scripts for sama5d2 + ov7740 sensor in raw bayer mode:
# ./video-capture-at91/ov7740/640x480_raw.sh
Preparing OV7440 in RAW BAYER MODE
Ready to capture at 640x480
# ./video-capture-at91/ov7740/fswebcam_640x480.sh
--- Opening /dev/video0...
Trying source module v4l2...
/dev/video0 opened.
No input was specified, using the first.
--- Capturing frame...
Skipping 20 frames...
Capturing 1 frames...
Captured 21 frames in 0.38 seconds. (55 fps)
--- Processing captured image...
Writing JPEG image to 'tinyRGB565.png'.
--- Opening /dev/video0...
Trying source module v4l2...
/dev/video0 opened.
No input was specified, using the first.
--- Capturing frame...
Skipping 20 frames...
Capturing 1 frames...
Captured 21 frames in 0.38 seconds. (55 fps)
--- Processing captured image...
Writing JPEG image to 'tinyYUYV.png'.
--- Opening /dev/video0...
Trying source module v4l2...
/dev/video0 opened.
No input was specified, using the first.
--- Capturing frame...
Skipping 20 frames...
Capturing 1 frames...
Captured 21 frames in 0.38 seconds. (54 fps)
--- Processing captured image...
Writing JPEG image to 'tinyABGR32.png'.
--- Opening /dev/video0...
Trying source module v4l2...
/dev/video0 opened.
No input was specified, using the first.
--- Capturing frame...
Skipping 20 frames...
Capturing 1 frames...
Captured 21 frames in 0.38 seconds. (54 fps)
--- Processing captured image...
Writing JPEG image to 'tinyGREY.png'.
--- Opening /dev/video0...
Trying source module v4l2...
/dev/video0 opened.
No input was specified, using the first.
--- Capturing frame...
Skipping 20 frames...
Capturing 1 frames...
Captured 21 frames in 0.38 seconds. (54 fps)
--- Processing captured image...
Writing JPEG image to 'tinyYUV420P.png'.
#
- ffmpeg can be used to capture video from the v4l2 device.
- To list possible pixel formats use :
ffmpeg -pix_fmts
Media controller pipeline must be properly configured before using ffmpeg
Example at 640x480 , RGB565 format
v4l2-ctl -v pixelformat=RGBP,height=480,width=640 # corresponding v4l2 format
ffmpeg -s vga -pix_fmt rgb565be -f video4linux2 -i /dev/video0 video_vga_rgb565be.avi
Example at 640x480, YUYV format
v4l2-ctl -v pixelformat=YUYV,height=480,width=640 # corresponding v4l2 format
ffmpeg -s vga -pix_fmt yuyv422 -f video4linux2 -i /dev/video0 video_vga_yuyv422.avi
Linux4SAM
