Leading Edge Technologies for Smartphone Cameras

In this publication last month an article covered the resources available for the testing of the cameras in smartphones. Building on that, the purpose of this article is to examine some of the technologies that are now appearing in leading edge smartphone cameras.

We can conveniently segment these into front facing cameras (used for ‘selfies’ and facial recognition) and rear facing cameras intended for photography but also used for authentication of stamps or through QR codes.

Front facing camera solutions

As noted above, there are two categories of these with somewhat different use cases and marketing drivers. The first is used predominantly for taking images of the phone user in association with other content, the second with more technical drivers for facial recognition. Both have applications in personal authentication so have a place in this article.

Self-portrait (selfie) capabilities have become a marketing tool for smartphones and have driven the market to ever higher image quality through improvements in image sensors, lenses and software. As these images also find a secondary use in personal authentication, for example as a comparison between a user and their travel document, this could be good news for our industry. However, this does come with two caveats.

The first is that these implementations are increasingly deploying software (often labelled as Artificial Intelligence or AI) to enhance self-image, resulting in a divergence from reality that will be brand dependent.

The second is that this brand dependence means that premium smartphones will continue to deliver different results to more budget models, essentially a continuation of market differentiation. The message from these is that consumer implementation of identity solutions using these cameras will have to tolerate and encompass an evolving spread of capabilities of the selfie cameras in consumers’ hands.

A similar situation applies to the illumination and camera systems used for facial recognition. Here, the primary drivers are towards maximum speed and minimum false negative results, but at illumination levels that are eye safe. This latter criterion looks set to drive facial biometric authentication in some interesting directions.

Facial recognition devices in smartphones commonly use illumination systems so that they not only have enough light to work in dim environments, but this light may also be ‘structured’ to give some element of 3D recognition. This is to counteract using a 2D photograph of the user. There is an increased tendency for this illumination to be in the near infra-red (NIR) end of the spectrum, outside the visible region.

There are certain IR wavelengths where the eye damage threshold is significantly greater, allowing the safe use of higher illumination intensities and better recognition. Although not yet implemented within smartphones, this may yet lead to the use of short wave IR (SWIR) as there are significant camera developments in this area.

Into the near future both these front facing cameras may well be positioned under the display screen, freeing extra display area.

Rear facing camera solutions 

Unlike the front facing cameras, those on the back face of the smartphone find specific usage in product and document authentication. There is a growing trend here for multiple cameras to be deployed on the back face of smartphones with lenses optimised for distance, close up or wide-angle photography.

While one or more of these can find applications in authentication solutions, there is a growing trend towards higher camera resolutions that may also find applications in more detailed product and document inspection.

There are now concept demonstrators that may point the way these developments could go. The term ‘demonstrator’ is used here because they have yet to be placed on general release, and are only sold in limited volumes and markets, and probably as a market test. Good examples come from a collaboration between smartphone manufacturer Xiaomi and iconic camera manufacturer Leica.

So far, these have not progressed beyond demonstrator stage. They do, however, provide an illustration of where technical and marketing developments are heading for smartphone cameras, towards higher and higher image quality. Marketing theory informs us that now smartphones have become a mature product, category innovation becomes a market attacker’s advantage for the lesser known brands.

Again, from marketing theory, the larger brands know they need to respond to protect margins and market share. Perhaps a more realistic example of a response was announced at the Consumer Electronics Show (CES) 2023. It was a smartphone zoom lens prototype from technology giant LG Innotek, a supplier of materials and components for smartphone manufacture.

While smartphone users may perceive that they can zoom into an image, so far this is done with software. What LG Innotek are demonstrating is a lens assembly that moves in the same fashion as a camera lens all within the confines of a slim smartphone body, including an optical assembly to stabilise the image. This brings with it some favourable attributes.

A zoom lens has the potential to reduce the number of cameras needed on the back face of the smartphone, freeing up space for other electronics or bigger batteries. The LG Innotek solution is a good fit to the normal shape and size of the smartphones we have become accustomed to, and is likely to see global availability. The capability to frame a feature across the full imaging sensor, and hence use full camera resolution could be useful for authentication solutions.

Software solutions

Some mention has been made in the above sections of the impact of software on these developments. Software is likely to become increasingly important in all aspects of smartphone photography and with it their use in authentication. It is often discussed in terms of either AI or computational photography, but for our purposes the result is the same.

Smartphone cameras are increasingly reflecting a subjectively pleasing version of reality rather than ‘ground truth’. This looks set to continue and we should be wary of the impact of this for authentication. The implications of a divergence of the smartphone image content and reality is something we need to consider.

The significance of these and other smartphone developments will be covered in greater depth in a seminar at the Optical & Digital Document Security conference, set for Prague 17-19 April. Conference and seminar are now open for registration.

opticaldigitalsecurity.com