Page translated by Claude — switch to Italian to read the original article.

One of the most debated and least understood issues among non-specialists is the so-called linearity of electronic image sensors.

CCD and CMOS imaging sensors should possess one fundamental characteristic if they are to be used for scientific purposes or even just for advanced astronomical imaging: the sensor’s response to light must be linear.

Linearity of response means that the sensor’s output is linearly correlated with the intensity of the radiation that strikes it.

If we denote by

ADU the intensity read on the image output from the sensor

I the intensity of the radiation

t the exposure time

K a fixed proportionality constant, depending on the technical characteristics of the sensor (mainly quantum efficiency and gain).

then, for an image calibrated with its corresponding dark frame, the following relationship holds

ADU=K*I*t

Therefore, in an ideal sensor, if the intensity of the light source or the exposure time doubles, then the ADU value obtained on the final image also doubles.

This characteristic is fundamental in photometry, where it is absolutely necessary to ensure an accurate measurement of magnitudes, but it is also extremely important in astronomical photography.

In fact, the application of the flat field, used to correct the multiplicative defects introduced by the optics (vignetting, dust, variations in quantum efficiency), presupposes that the sensor is linear: if it isn’t, calibration is destined to fail.

So if you want to use your sensor for photometry, or if you have problems applying flat field correction, it becomes necessary to verify the actual linearity of the sensor.

The best way to carry out this analysis is to expose the sensor to known intensities of radiation, for a defined time, and read the corresponding ADU value after calibration.

Unfortunately, especially at the amateur level, it is very difficult to obtain a perfectly controllable light source, so an alternative method is used: instead of varying the intensity I, in the equation above one varies t, assuming that the control of exposure times is precise and accurate.

The method is described very well in the article by Daniele Gasparri that you can read on the Teleskop-Express website.

Compared to the method described by Daniele, I apply three variations:

Instead of using the absolute residuals, given by the difference between the value calculated through linear regression and the one actually measured, I use the relative ones (percentage differences), that is, the absolute residual divided by the ADU value of the point it refers to.

This is, in fact, a more meaningful value than the absolute deviation from linearity: if at 100 ADU I have a residual of 2 ADU and at 30,000 ADU I have one of 300, using the absolute residuals it would seem that the sensor at 30,000 ADU is much worse than at 100; in reality, at 100 ADU the deviation is 2% (2/100) while at 30,000 ADU it is 1% (300/30,000), therefore much lower than the one at 100.

The second modification is that, by taking several measurements for each exposure time, in addition to calculating the mean I also calculate their standard deviation.

The standard deviation, in fact, allows the measurement uncertainty to be assessed: a measurement without its associated uncertainty is difficult to interpret.

Finally, as a last modification, in the residuals plot I use the ADU value instead of the exposure time as the x-axis: in this way it is possible to easily identify at what exposure level (in ADU) any nonlinearity phenomena occur, and furthermore it becomes possible to compare, on the same plot, data acquired in different sessions, with different source brightnesses.

I then tried to apply this modified method to my Moravian G3 16200, a CCD that should not have linearity problems over much of its dynamic range.

Here is the raw data

and here is the corresponding linearity plot.

If desired, a second series of data can be added by decreasing the intensity of the light source so as to “explore” lower ADU values.

Keeping the same exposure times for the second series and adding the linear regression lines, the following plot is obtained.

At first glance there would seem to be no problems; a slight inflection of the last point of the “high brightness” data is noticeable, due to sensor saturation, but in general it seems to behave well.

When you analyze the residuals, however, the surprises arrive.

As an example, here is the high-ADU data (the data highlighted in orange are those used for the linear regression)

and here is the residuals plot of two data sets with two different brightness levels (and error bars at 95% confidence).

What went wrong?

Not only does the sensor of the Moravian 16200 seem to be nonlinear (with differences at low ADU approaching 30%), but it seems to have an erratic behavior dependent on the brightness of the panel used to illuminate the sensor: in fact, even if the sensor had a pronounced nonlinearity at low ADU, in the regions where the two data sets overlap, between 1000 and 10000, it should behave in the same way (the blue and orange points should overlap within the error bars).
A sensor like that would be totally unusable for astronomical photography (besides being incomprehensible from a physical standpoint).

Since I use this camera without major calibration troubles, the problem must lie in the measurement method and not in the instrument.

The solution to the mystery comes from verifying the assumptions underlying this type of analysis.

The conditions are:

1. The intensity of the light source must be constant throughout the execution of the test
2. The exposure time set by the software must be rigorously reproduced by the camera

If one of these two conditions fails, then the experiment is destined to fail: in this case the weak link is precisely in the accuracy of the exposure time.

The Moravian G3 16200 has a mechanical rotating-blade shutter; this type of shutter takes a non-negligible time to open and close: this means that the sensor is typically exposed for a time shorter than the “nominal” one.

Since, whatever the nominal exposure time, the “transit time” of the shutter blade is always the same, this effect is all the more important the shorter the exposure time, undermining the very foundations of the linearity test based on residuals.

This is the reason why the two residuals plots do not overlap: for exposure times shorter than 60 seconds the data are affected by a significant systematic error.

To minimize this error, it is necessary to use exposure times longer than 60 seconds, which means having to use very faint light sources.

Such sources, however, risk becoming unstable and causing condition 1 to fail, bringing us back into uncertainty; furthermore, the test execution times would rapidly explode.

It therefore becomes necessary to develop a method that can be independent of the exposure time.

The local gradient method

The linearity equation written at the beginning of the article is valid only if the sensor is struck by incident radiation that is perfectly uniform over all pixels.

In practice, however, this never happens: any optical system is affected by “defects” that reduce this intensity inhomogeneously across the sensor surface.

Vignetting and dust grains produce a non-homogeneous illumination of the sensor: the local gradient method exploits precisely this effect.

The trick consists of producing an image affected by strong vignetting so that the sensor is illuminated inhomogeneously.

let us denote by Gp the “map” of the vignetting.

This function tells me how the intensity of the vignetting varies as the position on the sensor varies, therefore the linearity equation becomes

ADU=K*I*Gp*t

One then measures the ADU value in two different regions of the image: one near the center, where the signal will be greater (ADUc), and one near the edge of the image, where the signal will be lower (ADUb).

The local gradient LG(adu) is defined as the ratio between these two values.

In general this value could depend on the ADU value, but, if the sensor is linear, the equation above can be applied and therefore

LG(adu)=ADUc/ADUb = (K*I*Gc*t)/(K*I*Gp*t)

As can be seen, in the ratio above, K, I and, above all, t cancel out (being a single image, both the exposure time and the source intensity are the same), therefore.

LG=Gc/Gp

We thus arrive at the remarkable observation that, if the sensor is linear, the local gradient is independent of the characteristics of the sensor (K), of the brightness of the light source (I) and, above all, of the exposure time (t), and therefore independent of the ADU: it depends exclusively on the “geometry” of the vignetting.

That is, in plain terms, if the sensor is linear the ratio LG will have the same value whatever the brightness of the image used to calculate the ratio.

WARNING: the linearity of the sensor is a sufficient condition for the local gradient to be constant, but it is not necessary (there exist other functions of the intensity that could produce a constant ratio), therefore the local gradient method cannot completely replace the linear regression method, but can only complement its results.

A complete analysis of a CCD sensor should therefore include both tests.

The operational method

The acquisition method is similar to the one used for the linearity test with the residuals method.

Instead of illuminating the whole sensor homogeneously, however, I try to produce strong vignetting.

In my case I used a threaded cap with a small hole in the center, so as to produce intense illumination at the center with a rather pronounced brightness gradient toward the edges.

I then rested the sensor on a flatbox so as to have a constant illumination source.

I then exposed the sensor, using the Hα filter to further attenuate the source, for times varying between 5 and 400 seconds.

This allowed me to obtain ADU values approximately between 80 and 6000.

Then I increased the brightness of the panel and repeated the sequence, obtaining values from 1000 ADU up to sensor saturation.

for each time-brightness combination I acquired 5 images so as to have better statistics and estimate the experimental error.

In addition, for each exposure time I acquired 10 dark frames so as to be able to calibrate each image with its own dark.

This is a NECESSARY condition for the ratio method to work; in fact it is necessary to remove from the image every additive contribution, keeping only the multiplicative ones described by the linearity equation.

In the image below you can see one of these calibrated light frames used for the test.

The data extraction was performed thanks to PixInsight: it is in fact extremely important to evaluate the ratio at exactly the same positions in all the images.

Using the DynamicCrop process I defined two 100x100 pixel regions, one at the exact center of the image, the other near the top edge and, using its process icons, I extracted two sets of images:

the C series (center) and the B series (edge).

Thanks to the BatchStatistics script I measured, for each “crop”, the median value, saving the data in a CSV file.

The two files were then imported into Excel and, thanks to a trivial pivot table, the tables were obtained indicating, for each exposure time, the mean value and the standard deviation of the sample.

N.B. PixInsight handles normalized values (between 0 and 1), therefore, to obtain a representation in ADU, more understandable for most amateur astronomers, it was enough to multiply by 65535 (since the sensor is 16-bit)

Thanks to this data it was finally possible to obtain the following table:

and the corresponding ratios plot.

Interpreting the data

Well, the time has come to interpret the data in our possession: for convenience I reproduce here the residuals plot to be compared with the local gradients plot shown just above

Let us recall some fundamental points useful for interpreting these tests:

• All the plots are presented using the ADU of the image center as the independent variable: this allows data taken under different illumination conditions to be compared.
• The ADU axis is represented in logarithmic form to better separate the data at low ADU.
• The exposure times of the high-ADU series are 5s, 10s, 20s, 30s, 40s, 50s, 60s, 70s, 80s, 90s, 100s, 200s, 300s, 400s
• The exposure times of the low-ADU series are the same as those of the high-ADU series except for the 40-second point, which is missing because of a technical mishap.
• The residuals are presented as relative residuals, that is, normalized to the respective ADU value; this allows an effective assessment of the magnitude of the deviation from linearity
• In the residuals plot, the closer a value is to zero, the better the linearity of the sensor in that region.

Having made these premises, how can the data be interpreted?

The residuals plot tells us that, at least over part of the dynamic range, the sensor is perfectly linear (between 1,000 and 40,000 ADU); observing the local gradient plot, we notice that even below this value the sensor remains perfectly linear, in fact the ratio remains constant.

The apparent deviation from linearity is therefore explainable by a problem of accuracy in the exposure times: the Moravian’s mechanical shutter is decidedly unreliable, but this obviously does not constitute any problem for astronomical photography.

At high ADU, a slight inflection is noticeable both in the residuals plot and in the local gradient plot.

This inflection is attributable to the activation of the antiblooming barrier typical of cameras designed for “aesthetic” astronomical photography.

This inflection is in any case very contained up to about 40,000 ADU, where it settles around 1% (this would be the magnitude of a possible systematic error in magnitude measurements if the limit of 40,000 ADU were reached).
Beyond 50,000 ADU the deviation finally becomes increasingly pronounced until it culminates in sensor saturation, where there is the definitive loss of linearity.

Finally, in the local gradients plot, the first point of both series (5 seconds of exposure) seems noticeably higher than the others (although it is still compatible within the experimental error)

This is a further effect due to the rotating shutter: not only is the exposure time considerably lower than the nominal one, but the sensor turns out to be exposed inhomogeneously.

The portion of the sensor closest to the shutter’s center of rotation remains “in shadow” for a longer time than the more distant portion, therefore a brightness gradient forms on the sensor that produces an apparent nonlinearity.

This leads to a remarkable observation: when acquiring the calibration images, when capturing the flat fields, it is advisable to use exposure times of at least 10 seconds to prevent the phenomenon described above from creating fictitious gradients on our images.

In conclusion, the Moravian G3 16200, according to this test, is an excellent sensor for astrophotography, with excellent linearity in the range between 50 and 40,000 ADU and with a marginal deviation up to 55,000.

Given these characteristics, it can also be used for photometric studies, provided you do not get too close to the limit of 40,000, where the intervention of the antiblooming barrier produces an underestimation of the flux of over 1%.