SAMA7G5 and Sony IMX274 examples

Introduction

Sony IMX274 is a 10/12 bit Raw BAYER sensor with a CSI2 MIPI 4-lane connection.
Warning, important To have the sensor working with sama7g5, an additional patch to enable MIPI HALF mode is required.
Warning, important SAMA7G5 uses media controller paradigm to configure the pipeline. Please refer to this tutorial for more information about it. To test this sensor , we used the Leopard Imaging LI-IMX274-MIPI-CS

imx274_mic.png
In the photo above you can see the LI-IMX274-MIPI-CS connected to an adapter with an I-PEX cable, and from the adapter the standard Raspberry Pi ribbon cable.

We configure the imx274 adapter with jumpers as seen in photo below :
MicrosoftTeams-image_17.png
J3 is closed in the position [2,3]

Overlay configuration

To enable the imx274 overlay, it is required to edit the boot command of your U-boot.
The Linux4SAM release already includes the imx274 sensor overlay in the .itb file, ready to be applied at boot time.
The boot command in U-boot must look similar to this: 
bootcmd=fatload mmc 1:1 0x63000000 sama7g5ek.itb; bootm 0x63000000#kernel_dtb#imx274

The important part is adding the #imx274 after all the other overlays that U-boot is applying , at the bootm command.
If the overlay was applied successfully, at boot time we will notice:
## Loading fdt from FIT Image at 63000000 ...
   Using 'imx274' configuration
   Trying 'fdt_isc_imx274' fdt subimage
     Description:  Device Tree blob ISC IMX274 overlay
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x6347cd18
     Data Size:    2534 Bytes = 2.5 KiB
     Architecture: ARM
     Load Address: 0x61500000
     Hash algo:    crc32
     Hash value:   e5a3177c
     Hash algo:    sha1
     Hash value:   0400b15999df1192dd9289b77faf8a25c228290a
   Verifying Hash Integrity ... crc32+ sha1+ OK
   Loading fdt from 0x6347cd18 to 0x61500000
   Booting using the fdt blob at 0x61000000
   Loading Kernel Image
   Loading Device Tree to 7fb70000, end 7fb79f93 ... OK

Starting kernel ...

Media controller pipeline

If probed correctly, the media controller pipeline looks like this: 
Media controller API version 5.17.0

Media device information
------------------------
driver          atmel_isc_commo
model           microchip,sama7g5-isc
serial
bus info        platform:microchip-sama7g5-xisc
hw revision     0x220
driver version  5.17.0

Device topology
- entity 1: atmel_isc_scaler (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
        pad0: Sink
                [fmt:SRGGB10_1X10/3840x2160 field:none colorspace:srgb
                 crop.bounds:(0,0)/3840x2160
                 crop:(0,0)/3264x2160]
                <- "csi2dc":1 [ENABLED,IMMUTABLE]
        pad1: Source
                [fmt:SRGGB10_1X10/3264x2160 field:none colorspace:srgb]
                -> "atmel_isc_common":0 [ENABLED,IMMUTABLE]

- entity 4: csi2dc (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
        pad0: Sink
                [fmt:SRGGB10_1X10/3840x2160 field:none colorspace:srgb]
                <- "dw-csi.0":1 [ENABLED]
        pad1: Source
                [fmt:SRGGB10_1X10/3840x2160 field:none colorspace:srgb]
                -> "atmel_isc_scaler":0 [ENABLED,IMMUTABLE]

- entity 7: dw-csi.0 (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev2
        pad0: Sink
                [fmt:SRGGB10_1X10/3840x2160]
                <- "IMX274 1-001a":0 [ENABLED]
        pad1: Source
                [fmt:SRGGB10_1X10/3840x2160]
                -> "csi2dc":0 [ENABLED]

- entity 12: IMX274 1-001a (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev3
        pad0: Source
                [fmt:SRGGB10_1X10/3840x2160@9/87 field:none
                 crop.bounds:(0,0)/3840x2160
                 crop:(0,0)/3840x2160
                 compose.bounds:(0,0)/3840x2160
                 compose:(0,0)/3840x2160]
                -> "dw-csi.0":0 [ENABLED]

- entity 24: atmel_isc_common (1 pad, 1 link)
             type Node subtype V4L flags 1
             device node name /dev/video0
        pad0: Sink
                <- "atmel_isc_scaler":1 [ENABLED,IMMUTABLE]
The topology can be also represented by a graph:

graph_imx274.png

Capturing configuration

Capturing at 1280x540, mode6 

media-ctl -d /dev/media0 --set-v4l2 '"IMX274 1-001a":0[fmt:SRGGB10_1X10/1280x540@1/30]'
media-ctl -d /dev/media0 --set-v4l2 '"dw-csi.0":0[fmt:SRGGB10_1X10/1280x540]'    
media-ctl -d /dev/media0 --set-v4l2 '"csi2dc":0[fmt:SRGGB10_1X10/1280x540]'      
media-ctl -d /dev/media0 --set-v4l2 '"atmel_isc_scaler":0[fmt:SRGGB10_1X10/1280x540]'
v4l2-ctl -v pixelformat=YUYV,height=540,width=1280                               
fswebcam -p YUYV -r 1280x540 -S 20 tinyYUYV.png

Warning, important Pay attention to the IMX274 requested frame rate. IMX274 trims the frame size if the requested frame rate is too high, thus handing out incomplete frames.

Capturing at 1280x720, mode5 

media-ctl -d /dev/media0 --set-v4l2 '"IMX274 1-001a":0[fmt:SRGGB10_1X10/1280x720@1/20]'
media-ctl -d /dev/media0 --set-v4l2 '"dw-csi.0":0[fmt:SRGGB10_1X10/1280x720]'    
media-ctl -d /dev/media0 --set-v4l2 '"csi2dc":0[fmt:SRGGB10_1X10/1280x720]'      
media-ctl -d /dev/media0 --set-v4l2 '"atmel_isc_scaler":0[fmt:SRGGB10_1X10/1280x720]'
v4l2-ctl -v pixelformat=YUYV,height=720,width=1280                               
fswebcam -p YUYV -r 1280x720 -S 20 smallYUYV.png

Warning, important Pay attention to the IMX274 requested frame rate. IMX274 trims the frame size if the requested frame rate is too high, thus handing out incomplete frames.

Capturing at 1920x1080, mode3 

media-ctl -d /dev/media0 --set-v4l2 '"IMX274 1-001a":0[fmt:SRGGB10_1X10/1920x1080@9/171]'
media-ctl -d /dev/media0 --set-v4l2 '"dw-csi.0":0[fmt:SRGGB10_1X10/1920x1080]'   
media-ctl -d /dev/media0 --set-v4l2 '"csi2dc":0[fmt:SRGGB10_1X10/1920x1080]'     
media-ctl -d /dev/media0 --set-v4l2 '"atmel_isc_scaler":0[fmt:SRGGB10_1X10/1920x1080]'
v4l2-ctl -v pixelformat=YUYV,height=1080,width=1920                              
fswebcam -p YUYV -r 1920x1080 -S 20 fullYUYV.png

Warning, important Pay attention to the IMX274 requested frame rate. IMX274 trims the frame size if the requested frame rate is too high, thus handing out incomplete frames.
Pointing hand The computed 9/171 is the maximum possible before frame is cropped.

Capturing at 3264x2160, mode1 

media-ctl -d /dev/media0 --set-v4l2 '"IMX274 1-001a":0[fmt:SRGGB10_1X10/3840x2160@9/87]'
media-ctl -d /dev/media0 --set-v4l2 '"dw-csi.0":0[fmt:SRGGB10_1X10/3840x2160]'   
media-ctl -d /dev/media0 --set-v4l2 '"csi2dc":0[fmt:SRGGB10_1X10/3840x2160]'     
media-ctl -d /dev/media0 --set-v4l2 '"atmel_isc_scaler":0[fmt:SRGGB10_1X10/3264x2160]'
v4l2-ctl -v pixelformat=YUYV,height=2160,width=3264                              
fswebcam -p YUYV -r 3264x2160 -S 20 hugeYUYV.png

Warning, important Pay attention to the IMX274 requested frame rate. IMX274 trims the frame size if the requested frame rate is too high, thus handing out incomplete frames.
Pointing hand The computed 9/87 is the maximum possible before frame is cropped.
Pointing hand The sensor has a longer width than the maximum supported by the XISC. Thus, the frame is cropped by the atmel_isc_scaler to 3264 width maximum
Pointing hand The resulting resolution is 3264x2160, less than 8 Mpix, because of the width limitation.

Configuring sensor test mode

Querying sensor for test modes 

v4l2-ctl -L -d /dev/v4l-subdev3

User Controls

                       exposure 0x00980911 (int)    : min=14 max=50000 step=1 default=50000 value=14
                           gain 0x00980913 (int)    : min=256 max=46088 step=1 default=5120 value=5120
                  vertical_flip 0x00980915 (bool)   : default=0 value=0

Image Processing Controls

                   test_pattern 0x009f0903 (menu)   : min=0 max=12 default=0 value=0 (Disabled)
                                0: Disabled
                                1: All 000h Pattern
                                2: All FFFh Pattern
                                3: All 555h Pattern
                                4: All AAAh Pattern
                                5: Vertical Stripe (555h / AAAh)
                                6: Vertical Stripe (AAAh / 555h)
                                7: Vertical Stripe (000h / 555h)
                                8: Vertical Stripe (555h / 000h)
                                9: Vertical Stripe (000h / FFFh)
                                10: Vertical Stripe (FFFh / 000h)
                                11: Vertical Color Bars
                                12: Horizontal Color Bars

Setting one test mode to try 

v4l2-ctl  -d /dev/v4l-subdev3  --set-ctrl=test_pattern=11

Example resulted photos with vertical color bars

3264x2160

hugeYUYV.png

1920x1080

fullYUYV.png

1280x720

smallYUYV.png

1280x540

tinyYUYV.png

Using video-capture-at91 to configure the sensor

Linux4SAM already includes specific scripts inside the root file system that will help you automate the configuration of the media controller pipeline and simple photo capture.

Inside the home directory you will find: 

# ls -la video-capture-at91/imx274/
total 40
drwxr-xr-x    2 root     root          4096 Feb 17  2022 .
drwxr-xr-x    4 root     root          4096 Feb 16  2022 ..
-rwxr-xr-x    1 root     root           375 Feb 16  2022 1280x540.sh
-rwxr-xr-x    1 root     root           375 Feb 16  2022 1280x720.sh
-rwxr-xr-x    1 root     root           381 Feb 16  2022 1920x1080.sh
-rwxr-xr-x    1 root     root           380 Feb 16  2022 3264x2160.sh
-rwxr-xr-x    1 root     root           824 Feb 16  2022 fswebcam_1280x540.sh
-rwxr-xr-x    1 root     root           832 Feb 16  2022 fswebcam_1280x720.sh
-rwxr-xr-x    1 root     root           840 Feb 16  2022 fswebcam_1920x1080.sh
-rwxr-xr-x    1 root     root           832 Feb 16  2022 fswebcam_3264x2160.sh
#

Using the configuration scripts, you can have the media controller pipeline ready: 

# ./video-capture-at91/imx274/1280x540.sh
Ready to capture at 1280x540
# media-ctl -p
Media controller API version 5.17.0

Media device information
------------------------
driver          atmel_isc_commo
model           microchip,sama7g5-isc
serial
bus info        platform:microchip-sama7g5-xisc
hw revision     0x220
driver version  5.17.0

Device topology
- entity 1: atmel_isc_scaler (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
        pad0: Sink
                [fmt:SRGGB10_1X10/1280x540 field:none colorspace:srgb
                 crop.bounds:(0,0)/1280x540
                 crop:(0,0)/1280x540]
                <- "csi2dc":1 [ENABLED,IMMUTABLE]
        pad1: Source
                [fmt:SRGGB10_1X10/1280x540 field:none colorspace:srgb]
                -> "atmel_isc_common":0 [ENABLED,IMMUTABLE]

- entity 4: csi2dc (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
        pad0: Sink
                [fmt:SRGGB10_1X10/1280x540 field:none colorspace:srgb]
                <- "dw-csi.0":1 [ENABLED]
        pad1: Source
                [fmt:SRGGB10_1X10/1280x540 field:none colorspace:srgb]
                -> "atmel_isc_scaler":0 [ENABLED,IMMUTABLE]

- entity 7: dw-csi.0 (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev2
        pad0: Sink
                [fmt:SRGGB10_1X10/1280x540]
                <- "IMX274 1-001a":0 [ENABLED]
        pad1: Source
                [fmt:SRGGB10_1X10/1280x540]
                -> "csi2dc":0 [ENABLED]

- entity 12: IMX274 1-001a (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev3
        pad0: Source
                [fmt:SRGGB10_1X10/1280x540@1/30 field:none
                 crop.bounds:(0,0)/3840x2160
                 crop:(0,0)/3840x2160
                 compose.bounds:(0,0)/3840x2160
                 compose:(0,0)/1280x540]
                -> "dw-csi.0":0 [ENABLED]

- entity 24: atmel_isc_common (1 pad, 1 link)
             type Node subtype V4L flags 1
             device node name /dev/video0
        pad0: Sink
                <- "atmel_isc_scaler":1 [ENABLED,IMMUTABLE]

# ./video-capture-at91/imx274/3264x2160.sh
Ready to capture at 3264x2160
# media-ctl -p
Media controller API version 5.17.0

Media device information
------------------------
driver          atmel_isc_commo
model           microchip,sama7g5-isc
serial
bus info        platform:microchip-sama7g5-xisc
hw revision     0x220
driver version  5.17.0

Device topology

- entity 1: atmel_isc_scaler (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
        pad0: Sink
                [fmt:SRGGB10_1X10/3840x2160 field:none colorspace:srgb
                 crop.bounds:(0,0)/3840x2160
                 crop:(0,0)/3264x2160]
                <- "csi2dc":1 [ENABLED,IMMUTABLE]
        pad1: Source
                [fmt:SRGGB10_1X10/3264x2160 field:none colorspace:srgb]
                -> "atmel_isc_common":0 [ENABLED,IMMUTABLE]

- entity 4: csi2dc (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
        pad0: Sink
                [fmt:SRGGB10_1X10/3840x2160 field:none colorspace:srgb]
                <- "dw-csi.0":1 [ENABLED]
        pad1: Source
                [fmt:SRGGB10_1X10/3840x2160 field:none colorspace:srgb]
                -> "atmel_isc_scaler":0 [ENABLED,IMMUTABLE]

- entity 7: dw-csi.0 (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev2
        pad0: Sink
                [fmt:SRGGB10_1X10/3840x2160]
                <- "IMX274 1-001a":0 [ENABLED]
        pad1: Source
                [fmt:SRGGB10_1X10/3840x2160]
                -> "csi2dc":0 [ENABLED]

- entity 12: IMX274 1-001a (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev3
        pad0: Source
                [fmt:SRGGB10_1X10/3840x2160@9/87 field:none
                 crop.bounds:(0,0)/3840x2160
                 crop:(0,0)/3840x2160
                 compose.bounds:(0,0)/3840x2160
                 compose:(0,0)/3840x2160]
                -> "dw-csi.0":0 [ENABLED]

- entity 24: atmel_isc_common (1 pad, 1 link)
             type Node subtype V4L flags 1
             device node name /dev/video0
        pad0: Sink
                <- "atmel_isc_scaler":1 [ENABLED,IMMUTABLE]

#

One can then use the fswebcam_*.sh scripts to perform basic captures using fswebcam tool.

The scripts reside in a repository on our github

I Attachment Action Size Date Who Comment
PNGpng MicrosoftTeams-image_17.png manage 1132.6 K 2022-06-14 - 07:58 UnknownUser imx274 adapter
PNGpng fullYUYV.png manage 240.7 K 2021-11-23 - 11:52 UnknownUser  
PNGpng graph_imx274.png manage 20.8 K 2021-11-22 - 10:20 UnknownUser  
PNGpng hugeYUYV.png manage 865.9 K 2021-11-23 - 11:51 UnknownUser  
PNGpng imx274_mic.png manage 72.1 K 2021-11-19 - 15:34 UnknownUser  
PNGpng smallYUYV.png manage 108.3 K 2021-11-23 - 11:51 UnknownUser  
PNGpng tinyYUYV.png manage 82.0 K 2021-11-23 - 11:52 UnknownUser