REVIEW ARTICLES

Comtuter screen photo-assisted techiniques for chemical sensing

Filippini,D.; Lundström, I.

Division of Applied Physics, Linköping University, S-58183 Linköping, Sweden,
Fax: 39 6 2020519, E-mail:danfi@ifm.liu.se , www.ifm.liu.se/-danfi

Received November 05, 2004. In final form May 27, 2005

Abstract
The paper deals with the use of the computer screen as a light source in colorimetric experiments (light absorption, reflection, scattering, emission) with a web camera as a detector. This combination will be described in some detail and some examples of its applications given, related mainly to biomedical analysis. It is e.g. shown how the computer/web camera platform can be used to read arrays of biomedical samples and to perform colorimetric diagnostic tests. It is furthermore shown that the set-up can give fingerprints of coloured substances through a simultaneous measurement of both absorption and emission of light. The use of the computer screen photo- assisted technique as an electronic nose is also described. Finally further developments and possibilities for the computer screen photo-assisted techniques are speculated upon.

Resumen
El artículo trata del uso de pantallas de computadoras como fuente de luz en experimentos colorimétricos (absorción de la luz, reflexión , dispersión, emisión) con una cámara web como detector. Esta combinación se describe con detalles y se dan ejemplos de sus aplicaciones, relacionados principalmente con análisis biomédicos. Se muestra, por ejemplo, cómo se puede usar la plataforma computadora/ cámara web para leer mosaicos o matrices de muestras biomédicas y cómo realizar pruebas de diagnóstico colorimétricas. Se muestra, además, que el sistema puede brindar impresiones digitales de sustancias coloreadas mediante la medición simultánea de ambos procesos, absorción y emisión de la luz. Se describe también el uso de la técnica foto-asistida de pantalla de computadora como nariz electrónica. Finalmente, se especula sobre otros desarrollos y posibilidades de las técnicas foto-asistidas de pantalla de computadora.

Introduction
We have recently shown that a part of a computer screen (or in general displays based on the use of the so called RGB colors) can be used as a programmable light source for bio- and chemical sensing [1-5]. Two methods have been demonstrated for the creation of chemical images from large area field effect structures with catalytic metal gates using a computer screen as a programmable light source. One of these approaches, the so called scanning light pulse technique is emulated exploiting the two dimensional positioning and chopping intrinsically provided by the screen [1]. In the second method we make use of the large area, intensity (or color) modulated light source to homogeneously illuminate a sensor sample consisting of regions with different catalytic activities.
By simultaneously modulating the light intensity (or colors) and the biasing potential of the catalytic metal gate it was possible to create “chemical images” of molecules in a new way [2]. In this contribution we describe another use of a computer screen along with a web camera to generate chemical images. We propose to utilize the chemical sensitivity of an array of optical chemical sensors (as commonly composed bioassays [6, 7] or as thin layers as suggested for the “smell camera” [8] concept) placed between the screen and the web camera. We point out that such a concept applied to the computer web camera combination (henceforth referred to as computer screen photo-assisted technique, CSPT) can lead to an electronic nose technology based on an already available and widely spread platform, mainly the personal computer together with a web camera and expert data evaluation via internet [9].

1. Optical fingerprinting
Computer screens generate colors by linear combination of the spectral radiance of three primaries which excite the human perception of red, green and blue light (RGB colors). This weighted sum determines the color and brightness of the emitted light.
The primaries themselves are not monochromatic but a particular spectral distribution instead. Although the physics of the color generation differs from cathode rays tube (CRT) screens to liquid crystal displays (LCD) the principle of tri-color addition [10] is the same (extensively discussed in the color science literature [11,12]).
CSPT illustrates the possibility to use part of the computer screen as a (large area) programmable light source. The different illuminating colors are displayed in a sequence but in principle does not require any software beyond a free available media player, since the sequence itself can a be a digital video file. The resolution, and consequent size, of this file is irrelevant since it can be expanded according to size of the assays just by simple click and drag procedures.
Since several aspects of the CSPT concept are already described in the references given, we limit the description here to one detail of large importance in electronic nose applications based on color indicators, namely fingerprinting through absorption and emission spectra. Fig. 1 shows a schematic of the measuring platform and a representative result illustrating the CSPT ability for spectral fingerprinting of color substances [13]. A set of 24 substances is evaluated by standard visible absorption spectroscopy and in this figure compared with the CSPT signatures. Although not expected to be exactly the same, the similarities make evident that main spectral characteristics of these substances are retained in CSPT.
By contrast with well established spectral reconstruction techniques or spectral imagers that uses spectrally characterized setups, CSPT aims at retaining as much as possible spectral information without this knowledge since such information is unattainable considering the vast number of existing computer sets and web cameras already available as measuring platforms. Instead the assay itself contains auto calibration and reference rows, which are used to compensate for this aspect making the determinations independent of the particular platform used. This method can also be used to minimize instrumental drift effects, since sample and reference are always measured together.
The ability of CSPT for tracking kinetic target reactions (e.g. hormone detection) has also been demonstrated, using pigment particle containing cells with membrane receptors specific to certain drugs. Reflectance measurements involving opaque substrates as in commercial test strips have been recently demonstrated.
Web cameras are not the only possible detectors. The CSPT approach can also work with custom designed chemically sensitive light detectors using the screen as a intensity modulated, 2D position controlled (within ~200µm resolution determined by the typical pixel pitch) and shape configurable light source. Thus, as already mentioned in the introduction, CSPT can be used for similar purposes as the so called scanning light pulse techniques (SLPT) for chemical image generation. SLPT is an established analytical technique for odor recognition, which becomes equivalent to high density sensor arrays in electronic nose applications; particularly relevant here for body odors and breath analysis. The alternatives for gas phase sensing related to colorimetry are, however, those using web cameras as detectors for arrays of indicators, that change colors upon target gas exposure, as illustrated in Fig.1.

Fig. 1 Schematic of the computer screen photo-assisted technique (CSPT). Lower left: Computer screen with a light source window(indicated by the yellow light behind a multi-spot assay). A web camera collects the image of the multi-well assay. Upper left: 24 colored spots of different colors and hues used to compare CSPT with standard visible spectroscopy. Right hand drawings: Example of spectral fingerprinting using CSPT and its comparison with standard spectroscopy. The 200 different color indices were chosen to produce an illuminating color sequence mimicking that of human visual perception.

There are several ways to evaluate the information contained in the measurement. Actually the information recorded in each frame of the final video is subdivided at time in 320 x 240 = 76800 pixels. In this particular work, each spot in each frame is evaluated over the 314 pixels inside a circle smaller than the spot to minimize the influence of edge effects and uneven light distribution (Fig. 2).
Along all the colors of the sequence each particular pixel of the image will record spatially correlated information which at the end of the process will display a distinctive statistical signature of the spot (Fig. 2). If some of the spots are intentionally left blank, they can be used as a measure of the light source alone as the common practice in visible absorption spectroscopy. An example of the observed intensity for one red spot in the array is shown in Fig. 2 together with a reference “spectrum” obtained for an empty spot. From such curves it is possible to calculate normal quantities like the transmittance of the spot j along the i = 1...N colors of the illuminating sequence:

The results in Fig.1 were obtained in the way described above where the transmittance was calculated for the sum of the intensities in the three channels of the web camera. By comparison with results from visible absorption spectroscopy, it becomes clear that beyond differences in range and inherent convolution with camera filters and the particular spectral response of the detector both techniques express the same information for the same samples in an striking correlated fashion. If we compare the shapes of these curves, much more information is not found in the standard technique, basically due to the smooth nature of the visible transmittances of colored substances subject of interest in sensing applications. In an electronic nose or tongue approach it is possible to use these features to train the computer to recognize color changes related to the given analytical situation using already available pattern recognition methods.
It is natural to suggest that in many situations these methods can be provided by a suitable site on the internet which can be designed to both controlling the light source and web camera, and to transmit the information for further decoding, expert analysis and report from a single centralized server supplied with state of the art data mining capabilities.

2. Example of results
Colorimetric CSPT has so far been used mainly for the elucidation of biomedical applications and not for electronic nose applications. We reproduce below some of the possibilities for medical diagnostics, which we have recently demonstrated.

Cell viability tests
Fig. 3 shows the result of a cell viability experiment performed in a 96 wells micro titer plate. In such experiments bacterial cell viability is tested under the influence of different drugs. Such tests are made to determine the antibiotic resistance of the bacteria and determine the efficiency of different drugs and corresponding minimal doses.
The assay is made by adding to the growth medium a yellow organic salt which is metabolized into a blue product as long as the cells are alive [10]. Fig. 3 shows results for the two different kind of drugs which were tested. The plates were read with a standard microplate reader (at l= 650 nm) and with the computer screen light source displaying pure red (color index (255, 000, 000)). The images in Fig. 3 b) are produced to give a direct visual appearance of the viability of the cells. The darker the spots (more blue metabolic product) the more resistant are the cells to the antibiotic for a given drug concentration. Accordingly, control wells without antibiotics (ctrl) will look blue and with the high absorbance (close to 1), at increasing drug concentrations the decimation of bacterial cells reduce the ability of the cells to metabolize the yellow salt into the blue product subsequently reducing the absorbance. Beyond the biological significance of this assay in particular (daunorubicin is more effective than doxorubicin for these bacterial colonies) we want to exemplify the performance of CSPT for these determinations.
The accuracy of CSPT to comply with microplate reader measurements is seen Fig. 3a), but in this contribution we would like to stress the intrinsic ability of CSPT for pattern generation and its capacity to reproduce the reference measurement (Fig. 3b).

Fig. 2. a) Intensity distribution of each pixel composing the recorded image of a single spot along the illuminating sequence. b) Transmitted intensity features obtained as the average and standard deviation of all the pixels involved in the image of each well (spot j) for each color i of the illuminating sequence. Results are shown for a reference blank spot and a red one. (In the example the color sequence contained 250 different indices).

Reflectance measurements
Many simple colorimetric tests are based on solid test strips and reflectance measurements. CSPT can be used also for this purpose with a suitable optical arrangement allowing for reflectance measurements. Fig. 4 summarizes the results of reflectance measurements done to elucidate the possibilities of CSPT in this mode of operation [14]. The reflectance measurements can be evaluated in a similar way as the transmission experiments in Fig.1. We now obtain the reflectance spectra through the sum of the intensities of the three channels of the web camera and divide by the intensity from the reference white spots. Again the CSPT-spectra resembles the spectra obtained with a standard spectroscopic reflectometer. In many diagnostic situations we are more interested in the classification of the reflectance. For this purpose it can be more efficient to compile a data file of the measured intensities in each of the individual web camera channels as illustrated in Fig. 4 (upper left diagram). Different pattern recognition algorithms can be used on this data file. We have evaluated in a pilot test a multi parameter test strip which measures up to eleven different parameters in urine at the same time[15]. Principal component analysis (PCA) [16] was used to decide from data like those in Fig. 4 (upper left) the classification of the concentration of the different urine parameters in the intervals specified by the test.

Fig. 3. Comparison between a cell viability test monitored by a normal microplate reader and CSPT [3]. a) Picture of the array used for the analysis (a total of 30 wells) and the absorption measured by both methods (open circles for the reference technique and error bars for CSPT). The circle in the right hand corner of the array defines the area of each well used for the calculation of the absorbances. b) The absorbances are color coded in 5 levels to produce chemical response images. Each patch corresponds to a sample well. The assay is described in the text.

Fig. 4. Reflectance measurements with CSPT [11]. Upper right: Colored strips with different hues produced on white paper. The black circles show the regions used for further evaluation, Upper left: The measured intensities in each individual web camera channel put together in a joint data set. Data for the strongest hues of each color are shown. (250 different illuminating colors were used). Lower left: The reflectance of the different spots calculated from the sum of the intensities in the three web camera channels using the reflected intensity from white paper as a reference. Lower right: Reflectance of the colored spots (strongest hues) measured with a standard reflectance spectrometer.

The results in Fig. 5 demonstrate clearly the abilities of CSPT in this context. The position of the different classifications of the tested parameters, as suggested by the calibration colors given by the manufacturer, are shown in PCA plots generated by the CSPT measurements. They are well separated in the plot. Fig. 5 illustrates also a possible evaluation procedure for a representative commercial test strip used in the determination of glucose in urine. The range of possible outcomes in this test is not just a single color variation, but different glucose concentrations are indicated by different colors between yellow and green.

Fig. 5. Principal components plot of the different pads in the calibration label of a Glucose tests for urine samples. A potential test result is indicated as “test” together with the most relevant distances for classification purposes. dt minimum distance to a reference spot, dm mismatch distance and dc calibration distance (i.e. maximum spread of calibration points for a given classification) [17]

The strip contains ten other tests together with glucose, but we will concentrate in only one for clarity; the others are simultaneously measured and evaluated in the same way.
The test is placed in front of the screen and illuminated with a rainbow color sequence of 50 colors. Together with the strip, the references (as described below) are measured. These strips are suitable for visual inspection and come provided with a color chart for being evaluated by comparison with the exposed test. Fig. 5 reproduces the scanned image of the calibration label which is measured simultaneously with the test.
In standby, the CSPT device can periodically measure the label, keeping a fresh track of the references and of the properties of the computer web camera platform. When a determination is requested by the user, label and test are processed as described above producing PCA plots like that in Fig. 5.
When exposed to urine, the test generates a color closer to one of the references (represented by ‘test’ in Fig. 5) and the classification and concurrent evaluation is given just by the shortest Euclidean distance to the references (d_t). For a result like in the example the estimated value would be 100 mg/dL of glucose. The error can also be estimated e. g. as d_t divided by the distance of the corresponding reference to the nearest neighbor reference (the mismatch distance d_m). Other distances in the plot can be used to launch automatic longer recalibrations (for instance, repeating several illuminating cycles), e. g. when the references in subsequent standby evaluations drift more than a certain calibration diameter (d_c). One obvious improvement is to use a model (e.g. like partial least square regression [16, 18]) to more accurately determine the concentration of a given analyte. Certainly these and many more still unexplored possibilities exist for improving the CSPT performance. Typical test patches and labels are about 5 mm by side to be visually observable. The number of indicators per assay is limited by the same reason. If the same chemistries would be used in a more compact format, e. g. also including the references, they would still be suitable for CSPT evaluation, enabling more complete and sophisticated assays to be evaluated at homes.
It should be noted that the results described do not tell if CSPT will be a successful tool for the reading of the test strips in real practice. We are therefore presently engaged in a study with a primary health care unit where CSPT is compared with standard optical reading of test strips.

3. Electronic nose possibilities with CSPT
We have so far not used colorimetric CSPT for gas phase measurements. It is obvious, however, that any of the methods described above, based on measurement spots changing color upon gas exposure can be used for a colorimetric electronic nose concept (Fig. 6). Since there are many color indicators to choose from, we envisage that a versatile new platform for gas phase chemical sensing can be build on CSPT. Phtalocyanines, porphyrins and (fluorescent) polymers are interesting candidates in such a development. The color indicators may be used either in reflectance or transmission. The evaluation of the array for gas phase applications is similar as for the medical diagnostics demonstrated above. In Fig.6 we have indicated the use of 20 (different) color indicators and five reference spots in an array, where the color indicators react differently on different molecules in a gas mixture. Fig.6 shows the anticipated color changes, as calculated by the computer from color scans before and after the exposure to the mixtures. Such a concept is similar to that suggested by Rakow and Suslick [8], but where we use the computer screen as the light source and not a separate light source or ambient light. The scanning of the computer screen colors provides an extra degree of freedom for the evaluation of the color changes produced by the exposure to a given gas nixture. In principle a color change image can be produced for each color index in the color sequence used.

Fig.6. Schematics of an electronic nose application of CSPT.

Summary and conclusions
The concept of CSPT is very versatile and consists of many more possibilities than described in this presentation. The CSPT-platform, as schematically illustrated in Fig.1, is characterized by:

Based on already available platforms as standard computer sets of any kind: desktops, laptops, palm tops or even mobile telephones.
It is an imaging technique, suitable for several layouts of the assays or sensing matrices, the software recognizes and determines the regions of interest containing the responses.
Natural internet embedded operation suitable with distributed evaluation and centralized administration.
Inexpensive devices as web cameras of less than 50 USD are compatible with the technique.
Highly customizable: the application is defined by the sensing assay (e.g. medical diagnostics or environmental monitoring), the platform is always the same.
It does not require to install any specific software, just the regular internet browser would be enough, enabling centralized secure processing.
Compatible with commercially available sensing technologies (e.g. ELISA assays, fluorescent assays, test strips, lateral flow devices, etc.)
By using blank (white, transparent) wells/spots and specific calibration rows (or by comparison with reference spots) in the assay CSPT becomes independent of the computer/web camera platform.
Measurements start by tuning the assay to the specific screen/camera combination used.
Any ”colorimetric” phenomena may be evaluated (absorption, reflection, light scattering, emission).
Any material (polymer, hydrogel, insulator, semiconductor, solution,...) can be used as indicator material.
Any color reaction can be employed as indicator.
Applications possible both in gas and liquid phase: Medical diagnostics based on blood, saliva, urine, sweat, breath and body odours and Environmental monitoring of air and water (in the future also in individual homes?)
A few other observations are that by optimizing the computer generated color sequence defined molecules can be efficiently recognized in comparison to other molecules, i.e. providing a kind of molecular fingerprinting. Furthermore since more than three different colors in the sequence provide better classification there must be some non-linear effects in the color generation and/or measurements at many colors, although a linear combination of the three primaries, increase the signal to noise ratio giving a safer classification of the tested samples. More work has thus to be done to understand the performance of computer screen generated light in physical experiments and colorimetric tests. Regardless of the level of detailed understanding CSPT can, however, be used as a practical tool in many applications involving multiple sensing spots, like in optical electronic noses and for multi parameter diagnostic tests.

Acknowledgements
Financial support from the Swedish Research Council and the Swedish Agency for Innovation Systems for the development of the physics and applications of CSPT is greatly appreciated

References

[1] Filippini, D.; Lundström, I., “Chemical imaging by a computer screen aided scanning light pulse technique”, J. Appl. Phys. 2002, 81, 3891-3893.         [ Links ]

[2] Filippini, D.; Lundström, I., “Chemical images generated by large area homogeneous illumination of metal–insulator–semiconductor structures”, J. Appl. Phys. 2003, 82, 3791-3793.         [ Links ]

[3] Filippini, D.; Svensson, S.P.S.; Lundström, I., “Computer screen as a programmable light source for visible absorption characterization of (bio)chemical assays”, Chem. Commun. 2003, 2003, 240-241.         [ Links ]

[4] Filippini, D.; Lundström, I., Desktop biomedical analysis, Proceedings of EMBEC'02, 2nd European Medical & Biological Engineering Conference, 2002, Part I, 310-311.         [ Links ]

[5] Filippini, D.; Andersson, T.P.M., Svensson, S.P.S.; Lundström, I., "Microplate based biosensing with a computer screen aided technique", Biosensors and Bioelectronics, 2003, 19, 35-41.         [ Links ]

[6] Engvall, E.; Perlmann, P., “Enzyme linked immuno-sorbent assay (ELISA): quantitative assay of IgG”, Immunochemistry, 1971, 8, 871-876.         [ Links ]

[7] Schena, M.; Shalon, D.; Davis, R.W.; Brown, P.O., “Quantitative monitoring of gene expression patterns with a complementary DNA microarray”, Science. 1995, 270, 467-470.         [ Links ]

[8] Rakow, N.; Suslick, K., “A colorimetric sensor array for odour visualization”, Nature, 2000, 406, 710-713.         [ Links ]

[9] Filippini, D.; Lundström, I., Test method, PCT/SE02/00207 patent pending.         [ Links ]

[10] Mossman, R., “Rapid colorimetric assay for cellular growth and survival: application to proliferation and cytotoxicity assays”, J. Immunol. Methods, 1983, 65, 55-63.         [ Links ]

[11] Jackson, R.; MacDonald, L.; Freeman, K., Computer generated color, John Wiley & Sons, 1994.         [ Links ]

[12] Wyszecki, G.; Stiles, W., “Color science: concepts and methods, quantitative data and formulae”, John Wiley & Sons, New York, 1982Links ] Times, serif">.

[13] Filippini, D.; Lundström, I., “Spectroscopic information retained in screen photo-assisted techniques”, Anal. Chim. Acta, 2004, 512, 239-246Links ] Times, serif">.

[14] Filippini, D.; Comina, G.; Lundström, I., “Computer screen photo-assisted reflectance fingerprinting”, Sensors and Actuators B, 2005, 107(2), 580.         [ Links ]

[15] Filippini, D.; Lundström, I., “CSPT evaluation of colorimetric 10 parameters urine tests”, to be published.         [ Links ]

[16] Joliffe, I., “Principal Component Analysis”, Springer-Verlag, New York, 1986.         [ Links ]

[17] Filippini, D.; Lundström, I., “Computer Screen Photo-assisted Techniques”, Encyclopedia of Sensors, in Grimes, C.; Dickey, E.; Pisho, M., (Eds), American Scientific Publishers, 2005, in press.         [ Links ]

[18] Wold, S.; Ruhe, A., Wold, H.; Dunn, W.J., “The collinearity problem in linear regression. the partial least squares (pls) approach to generalized inverses, SIAM”, J. Sci. Stat. Comput., 1984, 5(3),735-743.         [ Links ]