International Standards for Smartphone Camera Testing

Many in our communities will be familiar with the benefits of international standards. These have figured in debates on mobile driving licences (mDL), tax stamps and now in establishing sustainability.

The purpose of this article is to explore a less known resource – the work done within ISO Technical Committee 42 (TC 42) that is pertinent to the use of smartphone cameras for authentication.

I should firstly declare some vested interest in this topic. For the last 20 years I have participated in the work of this committee, led the UK delegation for 15 years and for the last year acted as Chair. With that disclosed, the aim of this article is to show the relevance of TC 42 to authentication solutions that utilise the camera systems in smartphones.

The relevant work of TC 42

As stated in the full name of this Technical Committee, TC 42 deals with International Standards for Photography. From a consumer perspective, photography has almost wholly transitioned from dedicated cameras to the smartphone platform and TC 42 has moved in this direction too. As a result, the documents produced by TC 42 now cover the smartphone cameras that are used in many authentication and traceability programmes.

In addition to the direct relevance of this work, a literature search will reveal where these international standards are used to make comparisons of the performance of smartphone devices relevant to our work.

One such instance, published in a peer- reviewed academic journal is referenced in this work as an example. Papers such as this show how these test methods can be used in practice, and also provide an illustration of the level of variation between smartphone models.

There is another benefit implicit in considering this work. Committees such as TC 42 have invested time and effort in understanding and documenting the key issues, in this case governing the performance characteristics of smartphone cameras. As illustrated by the documents cited below, this provides a ready-made resource to identify the relevant performance criteria for our various use cases.

Use cases in authentication

This article is directly pertinent to two specific use cases of authentication solutions.

The first of these is choosing a smartphone model for use by an inspector in the field who may wish to examine multiple types of items.

The second use case is checking a specific printed or optical feature for resilience to consumer smartphone choice.

There is a key point to glean from the work of TC 42 in this area, stemming from the fact that different brands of smartphones have different camera performance. This is Intuitively obvious as it is a marketing differentiation strategy of the manufacturers, but the key point is that just because a feature works with one camera does not necessarily make it universally read by other models. The peer reviewed publication cited at the end of this article uses ISO TC 42 test methods and provides a good example of the differences that can be found between models.

Significant issues found with smartphone cameras occur under challenging lighting conditions and these are explicitly addressed in the work of TC 42. TC 42 has done a lot of work to define what these categories are – camera flare issues and low light performance. We can illustrate this with reference to four ISO standards, two linked to flare issues and two on low light.

The use cases

So, now we can drill into some examples where this work could be used and, here, we should consider the light level of the use case as a key criterion.

If the protocol is aimed at high light level situations (airport duty free or behind glass in a display case are good examples) flare issues become a priority, but for dimly lit stores low light is the issue. We also have to be aware that an authentication regime may need to cater for both of these.

If inspection commissioners are considering buying one model of smartphone for their team, these criteria guide the choice of TC 42 assessment methods to make the model choice. Conversely, if inspectors are considering a ‘bring your own device’ model, it points out some of the issues that may be faced by the use of multiple models of smartphones.

For those designing features for authentication, TC 42 provides a resource to inform an assessment of smartphone performance, particularly in what to look for in terms of camera performance, which may be particularly relevant in emerging markets with budget smartphones.

In high light situations, an issue known as image flare can become important and this is the topic of ISO 18844:2017, ‘Photography – Digital cameras – Image flare measurement’.

In essence, flare is produced in an imaging system by stray (unwanted) light such as from spotlights and reflections. Smartphone camera systems deploy image processing algorithms to try and compensate for this, but flare can be quite variable and image processing algorithms are not always successful at removing it.

In a smartphone camera system, all elements (lens, sensor, software etc.) can contribute to the end result from image flare. An older international standard, ISO 9358 from ISO Technical Committee 172, covers ‘veiling glare’, the flare from lens systems alone. But because of the complexity of the smartphone camera imaging system, I would recommend the use of ISO 18844.

For low light performance, the relevant document is aptly named ISO 19093:2018, ‘Photography – Digital cameras – Measuring low-light performance’. Reference is made within this document to ISO 15739, ‘Photography – Electronic still- picture imaging – Noise measurements’, as noisy images are often a problem at low light levels. Both these documents provide a resource for the issues around low light, but I would recommend starting with ISO 19093 as this encompasses the noise issue.

Further resources

One of the other learning points worth emphasising from this study is that ISO (and indeed IEC and ITU) standards are a valuable resource for our community. TC 42 welcomes new members and I am happy to discuss this with our community – [email protected]. Or we can discuss this at the 2023 Optical & Digital Document Security™ Conference, set for 17-19 April 2023 in Prague, Czech Republic.

Peer review paper reference: V-T Peltoketo, ‘Signal to Noise Ratio and Visual Noise of Mobile Phone Cameras’, Journal of Imaging Science and Technology, 59(1): 010401-1–010401-7, 2015.