Raspberry Pi Autofocus Camera & libcamera

Today we will be checking out a new camera for the Raspberry Pi, the Arducam Autofocus camera.  We’ll also be looking at libcamera, the new camera stack provided with the latest Raspberry Pi operating system.

Introduction

One of the many wonderful features of the Raspberry Pi is that it comes with a CSI camera port, allowing you to add video capabilities to your projects.   All modern Raspberry Pi boards have this feature, from the inexpensive model Zero to the top-of-the-line 8GB Raspberry Pi Model 4.

The cameras used with the Raspberry Pi have evolved over the years. We now have a wealth of cameras to choose from, from the 5MP Version 1 to the 16MP Raspberry Pi HQ Camera.  And the models are not just limited to ones made by Raspberry Pi, other manufacturers such as Waveshare and Arducam also have produced Raspberry Pi cameras, some of them outperforming the originals, with sensors capable of up to 21MP resolution.

Today we are examining one of these cameras, a new camera from Arducam that has some impressive specifications, especially considering its low price.

 

 

And we’ll also be working with a new camera stack packaged with the latest version of the Raspberry Pi operating system, libcamera.

Arducam Autofocus Camera

The Arducam Autofocus Camera (currently it doesn’t seem to have a model number) is a brand-new offering that has just come off of its successful Kickstarter campaign.  

This camera is one of Arducam’s “Pivariety” series of devices, and it’s not the only one of the series with autofocus capabilities.  However, it does stand out from the rest of the offerings due to (a) its low price (same as the Raspberry Pi Version 2 Camera), (b) its 16MP sensor, and (c) its form-factor, which is identical to that of the version 1 and 2 cameras.

The camera is also packaged with a small plastic case that provides fairly effective weatherproofing, and that has a ¼-inch thread to allow the camera to be mounted on a standard photographic tripod.

libcamera

If you are familiar with using cameras with the Raspberry Pi, then you have likely run across two command-line utilities – raspivid and raspistill.  These two applications allow you to take video and still pictures and adjust a wealth of camera parameters like image size and brightness.

With the release of the latest version (as of this writing, of course) of the Raspberry Pi operating system, code-named “Bullseye”, raspivid and raspistill have been replaced by a new camera stack named libcamera.

libcamera (the lower-case spelling is intentional) is an open-source camera stack for controlling embedded camera hardware, and it is available for Linux, Android, and ChromeOS. To adapt libcamera to an operating system requires the OS developer to write a custom pipeline handler, which Raspberry Pi has done.

libcamera supports all the existing Raspberry Pi cameras, as well as a number of devices from 3rd-party manufacturers. Developers can add additional camera modules as well. It can be used at the command line, as well as within Python programs.

To make it easier to switch existing applications to libcamera Raspberry Pi has provided a set of libcamera-apps.  The chart below describes their functions:

Of particular interest are libcamera-still and libcamera-vid, which effectively replace the functions performed by raspistill and raspivid respectively.

We will be using these libcamera-apps today to check out our new autofocus camera.

How Autofocus Cameras Work

Before we get started with our new camera, I thought it might be useful to see how cameras perform automatic focusing.

In order to focus a camera, you need to be able to move the position of the camera lens relative to the image sensor.  You can see this effect with a magnifying glass or another simple lens, move it toward and away from your eye and observe how the target image changes focus.

In a manual focus camera, you can accomplish this by twisting a focus ring, which in turn changes the lens position.

Focus Motor

In an electronic focus design, you replace the manual ring with a motorized one. On some cameras, this is done with an actual small motor, whereas others use a mechanism that works more like a solenoid. In both cases, we’ll refer to this as a focus motor.

To autofocus a camera, you need to have some sort of control system that moves the motor.  The three components are illustrated here:

By adding a motor to the camera lens assembly, we can use software to control the focus. We could write a program to do this manually, or we could use a program to automatically achieve this.

On the Raspberry Pi we have to figure out one detail before we can work on a software-based focus system – how do we electrically attach to the focus motor to control it?

There are two possibilities:

• Make additional connections to the camera from the Raspberry Pi GPIO.
• Find some unused connections on the CSI camera cable.

The second method is preferable, as it would make the camera easier to use. It also avoids any conflict with other devices we might want to hook to the GPIO.

Let’s examine the pinout of the 15-pin camera CSI cable and see if we can find a few unused connections:

Most of the connections are either power, ground, or signal connections. Note that each signal has both a positive and negative component, that is because this is a “balanced” line. It allows the use of a long unshielded cable without picking up extraneous electrical noise.

But there are two other connections that are ideal for the job:

The CSI cable has its own I2C bus, a separate bus from the two that are already on the Raspberry Pi GPIO connector. And on most camera implementations it is unused.

So we can make use of this connection to control our focus motor.

Autofocus Methods

Now that we know how we can move the motor via software, the next thing to figure out is how we can achieve automatic focusing.

Autofocus is a feature that you’ll find on most cameras, and there are a few different methods of achieving it. Some advanced cameras use multiple methods.

• Phase Detection. With this method, the incoming light is sent to different sensors, each mounted at a different angle. The phase difference between the sensors is compared electronically, and the focus motor is moved until the differences are minimized or eliminated.
• Assist Lamp. With Assist Lamp Autofocus, an infrared LED or laser is used to illuminate and to measure the distance to the target subject. This measurement is used to adjust the lens position.
• Contrast Detection. This method examines the individual pixels in the homage and moves the focus motor until the maximum difference between pixels is achieved.

As the first two methods require extra hardware, the simplest method is Contrast Detection. This is the method used in the Arducam Autofocus Camera, it is also used in many phone-based and pocket cameras.

Getting Started with the Arducam Autofocus Camera

The Arducam Autofocus Camera comes packaged in a small box that contains the camera module, the plastic case, a short CVI cable, and an even shorter instruction “manual” (i.e. it’s a piece of paper).

In the unit I received, the camera module was already mounted in the case and the cable was attached.  It is very easy to remove the camera from the case, and you can replace the cable while the camera is still in its case.

Connecting the camera to the Raspberry Pi is as simple as any other camera module, just watch the orientation of the cable. The correct method is to have the blue insulator side facing the USB ports if you are using a Raspberry Pi 4.

Unlike most other Raspberry Pi cameras, however, you’ll need to install some software and a driver to get the camera to work.

Installing the Camera Software and Drivers

Arducam has provided the software you’ll need on GitHub, and it can all be installed at the command line. You’ll need to ensure that you are using a recent version of the Raspberry Pi OS, any version made after March 2021 will suffice.

Log into your desktop, open the Terminal and follow along.  As some of these commands are a bit cryptic you might want to grab the “cheat sheet” that I have prepared, you can copy it to your Raspberry Pi, open it in Genie and just copy and paste the lines into the Terminal.

Download the shell scripts

The first step is to download all the shell scripts that you’ll need to perform the installation. You can do that using the following command:

Set the installation file permission

The installation file will need permission to execute on your Raspberry Pi. You can do this using the chmod function as follows:

Update your Raspberry Pi

Before you get started, you’ll want to ensure that all of your files are up-to-date.

Install the Arducam libcamera updates

Arducam requires a few updates to the existing libcamera dev and apps files. The following two commands will take care of this:

Note the similarity between the two commands. You can take advantage of the arrow key to change between them and just type the differences. Or you can just use my cheat sheet!

Install the Camera Kernal Driver

This is the driver for the Sony IMX519 sensor used in the camera module.

You will need to reboot your Raspberry Pi after this step.

Install the Manual Focus Application

This Python script will allow you to focus the camera using your keyboard arrow keys. It’s optional but extremely useful.  I’ll show you how to use it in a bit.

And you are now finished installing the camera software. Now let’s see how to use it.

Using the Autofocus Camera for Images

Now that you have your camera hooked up to your Raspberry Pi and have installed the software, you’re ready to start using it.  

We will be using libcamera at the Terminal, so you’ll need to open a Terminal window again.

Testing with libcamera-hello

A quick way to test out the camera to make sure it is working is by using the libcamera-hello app. Type it on the command line and press enter.

If everything is working correctly, you’ll see a small preview window open up in the top left corner of the screen, displaying the output of the camera module. It may or may not be in focus, we’ll deal with that later.

If you would like the image displayed to be larger you can use the full-screen parameter, this works with most libcamera apps. You can either use -f or –fullscreen as follows:

Now that you’ve determined that the camera is functional, it’s time to take your first picture!

Taking a Picture

The libcamera-jpeg app allows you to take JPEG images using the camera. For this command you’ll need to specify the output file name, using the “.jpg” or “.jpeg” file extension.

Type the following at the command line in your Terminal:

This will open a window, do a short countdown, and then capture an image that is stored within the directory you executed it from. Assuming that you haven’t changed directories, this image will end up in your Home directory.

You can open your File Manager to find the image. Click on it to display it in the viewer.

The -o parameter is the “output” parameter, and we will be using it frequently with both still images and video files.

Using libcamera-still

The libcamera-still app is the equivalent of the older raspistill app, and it accepts most of the same arguments and parameters.

Let’s start with the following command:

The -t parameter is a timeout parameter, measured in milliseconds. In the above example, it will result in a small image window opening up for five seconds. You can experiment with other time values to get familiar with it.

If you want to open a window indefinitely, which is handy when aiming or focusing the camera, use a value of zero for the time parameter, as follows:

The window will stay open until you close it using your mouse.

The next command combines a number of parameters on one line:

In this case, we will see a full-screen preview window for 10 seconds, after which an image named newimage.jpg will be created.

Manual and Automatic Focus

Of course, the “big new feature” in the Arducam Raspberry Pi Autofocus Camera is its ability to, well, autofocus!  You can also focus the camera manually, using a software utility that you installed earlier.

Actually, there is no way to manually focus the camera mechanically, you have to either autofocus or use software manual focus. Unlike other Raspberry Pi camera modules, there is no focus ring that you can twist.

Manual Focus

In order to demonstrate the manual focus application, you’ll need to open up two Terminal windows, which is simply a matter of clicking the Terminal icon twice.

In one Terminal window, you will just use libcamera-still to display a picture window with an infinite timeout.  You can use this window to examine the image while you focus it.

In the other window you’ll need to change into the directory that the manual focus application lives in, as follows:

Remaining in this window, you can now run the manual focus application:

This will start a Python application that will display in the window. With this application, you can adjust the focus with the up and down arrow keys. The application will display the focus value, from 0 to 4095.

As you make the adjustment, the image focus will change.  Use the keys to adjust it until the focus is the best.

Autofocus

Of course, autofocus is what you really want to see in action, and you can do it directly from libcamera. It’s as simple as using the –autofocus parameter.

This will display a full-screen image for 10 seconds, using the autofocus feature.  You will notice the camera coming into focus at the beginning of the timeout period.

The autofocus works well, but it has one pretty major limitation – it only performs an autofocus operation when the image is first opened.  You can demonstrate this by moving the camera and noting how it does not refocus when aimed at a different object, or when moved closer or further to the object it initially focused upon.

In other words, unlike most autofocus cameras you may have worked with,  it does not do continuous autofocus.  And I must confess, I found that to be a real disappointment!

However, there is a way to force it to do autofocus any time you want to, which does improve the situation a bit.

Still in the Terminal, type the following command:

Again, the picture window will open, and the focus will adjust immediately. Just as it did in the last experiment.

Now go back into the Terminal window, while the image is still being displayed, and type the letter “f” and press “enter”.  When you do, you should notice the camera performs another autofocus operation. 

You can repeat this as many times as you like, aiming at different objects and entering an “f”.

It’s not exactly continuous autofocus, but it will allow you to refocus automatically on different objects.

Using the Autofocus Camera for Video

You can also use libcamera for creating video files. In many respects, it’s just the same as creating still images, except instead of libcamera-still you’ll be using libcamera-vid.

Open a Terminal window and type the following:

As you might expect, this opens a picture window that displays live video from the camera for twenty seconds. The framerate and other parameters will be displayed on the top of the window.

And you can also apply autofocus to the image, using this command:

As with the still images, the autofocus on video is not continuous, it will only happen when the camera module starts sensing video. But, as with the still images, you can use the –keypress parameter to refocus the camera using the keyboard.

You can also use the manual focus Python application with video, giving you another option to focus the camera.

Saving a Video File

To save the video to a file, you can use the following command:

This will autofocus the camera and save a 20-second video to a “myvideo.h264” file in the current directory.

Note the file format is “.h264”, which is playable on some, but not all, video players. VLC Media Player is one player that can handle these files. You can also convert these files to MP4 using the MP4Box utility, which we have used before. 

Instructions for setting up and using MP4Box can be found in the earlier article I created on Raspberry Pi Cameras.

You can adjust many parameters regarding the video, in fact, there are literally hundreds of adjustments you can do to alter the size, quality, white balance, orientation, and more. To see them all, just look at the Help manual by typing the following:

An example of changing parameters is creating an HD video, which you can do as follows:

The resulting video will be in 1080 HD.

Streaming Video

The libcamera-vid application is also capable of streaming video, albeit with a fair bit of latency. This is a great application for a Raspberry Pi camera, as you could use it to monitor a room or a process remotely.

You’ll need to have a (preferably Linux) workstation to receive the video stream, you could even use a second Raspberry Pi. The receiving computer will need to have VLC Media Player installed, and, of course, both computers will need to be on a network where they can “see” one another.

Streaming Server

Before you can do any video streaming, you’ll need to know either the network name or IP address of the transmitting (source) Raspberry Pi.  This information can be garnered from your router, or you could simply look at it using the ifconfig command in the Terminal.

This will display all the IP addresses associated with your network adapter. You can ignore the one that reads “127.0.0.1”, as this is your Loopback Address and is used for diagnostic purposes. 

You will have one or two other addresses, depending upon if you have both WiFi and Ethernet connections to your Raspberry Pi. I would always recommend using an Ethernet connection for the streaming server. If you’re unsure as to which one is which, just unplug the Ethernet cable and run ifconfig again.

Make note of the IP address, as you’ll need it on the receiving side.

Now type the following at the Terminal command line:

You won’t observe anything happening until the receiver picks up the stream, once it does the usual picture window will be displayed.

Viewing the Stream

On the workstation that you wish to video the stream upon, open up VLC Media Player and do the following:

• Click the Media item on the top menu bar.
• Click Open Network Stream from the menu presented.
• Navigate to the Network tab in the resulting dialog box.

Now enter the following string in the network URL box, substituting the IP address you obtained earlier for ipaddress:

Click the Play button. You should now see the network stream.

If you can also observe the Raspberry Pi you’ll notice two things. First, a window will open displaying the stream. And second, there is a lot of latency.

Nonetheless, this is a pretty simple way of streaming video using any Raspberry Pi camera.

Conclusion

I must admit that I have mixed feelings regarding the Arducam Autofocus Camera. 

On one hand, it is an excellent Raspberry Pi Camera module. On the video that accompanies this article, I compared it to the three camera offerings from Raspberry Pi, and its video and image quality was comparable to the expensive Raspberry Pi HQ Camera. And, as it is priced the same as the Raspberry Pi Version 2 camera (which it was quite superior to) it was a clear winner in the value department. Add to that the included plastic case with the semi-functional tripod thread and I certainly would have no hesitation recommending it.

But on the other hand, when I got the camera module I had assumed that “autofocus” meant continuous autofocus, and I think many other people would also have assumed that. So to discover that it doesn’t do continuous autofocus was a big disappointment to me. So much for my security applications!

Nonetheless, it is a great value and an excellent camera module otherwise. And, as the focus is driven through software, I expect someone, perhaps Arducam, will develop a way to add true continuous autofocus to it.

Arducam has other autofocus modules as well in their Pivariety series, some of which have 21MP sensors. So that could well be the beginning of a new standard in Raspberry Pi Cameras.

I would recommend that you pick one up and check it out for yourself.

Stay focused!