Why are oranges orange?
A spectroscopic and statistical analysis of fruit pigmentation in a supermarket

Gordon Thomas, Department of Physics, MIT, Cambridge, MA
Alan Gelperin, Mark Hansen, and Don Sun, Bell Laboratories, Lucent Technologies, Murray Hill, NJ

 

Outline:

1. Intro

2. Orange is easy to distinguish from green.

Figure 1: Penalized discriminant analysis shows approximate orthogonality of the colors and allows an explanation of the comparison of the absorption spectrum of the pigments and the diffuse reflectance and the components.

Figure 2. Penalized discriminant analysis separates the colors really well. A primate brain does the discrimination differently, but the result is probably the same.

3. the wavelength range of info is similar to the range of sensitivity of primate eyes.

Figure 3. Analysis of the total set of fruits and vegs (with PDA) allows measurement of ID accuracy vs wavelength range

4. the number of colors needed for moderate ID accuracy is not high, similar to the eye

Figure 4. Analysis of ID accuracy vs number of colors

5. the spectroscopic resolution needed is low, similar to primate eyes

Figure 5. Analysis of ID accuracy vs resolution

 

 

 

We present a statistical analysis of spectroscopic data obtained from a large collection of fruits and vegetables. Over the past several years, we have complied more than 50,000 measurements on 100 varieties of produce commonly stocked by supermarkets in the United States. Our goal was to determine how reliably we can distinguish these items based solely on their color. In addressing this question, we uncovered support for more fundamental issues in evolutionary biology1, namely how plants have evolved their colors to enhance the propagation of their species. At a time when some educational administrators still question the importance of teaching evolution2, we find it fascinating that a quantitative analysis of the contents of something as ordinary as a supermarket supports evolution.

We measure mathematically the extent to which the color of oranges and, similarly, other fruits and vegetables, differs from the color of green leaves. We also show that the information content in the pigmentation matches the detection characteristics of primate eyes: the wavelength range, the chromatic resolution and the number of color components. We argue that the results fit an evolutionary model because this set of characteristics increases the probability that primates will pick the fruit, spread the seeds effectively, and contribute to successful plant propagation.

We have measured diffuse reflectance spectra of over 50,000 samples, including about 100 varieties of fruits and vegetables. We have developed an electronic eye3 for this purpose, composed of a set of receptors linked to electronic recording and analysis circuitry. This e-eye has characteristics that we can manipulate electronically. Unlike our visual system4 or that of other living things, the e-eye has an adjustable number of receptors (up to 1024) that record different wavelengths of light. By contrast, humans have 3 (or sometimes only 2) types of color receptors4 with which to record broad ranges of wavelengths centered around red, green and blue. Our e-eye performs object recognition in a way that differs significantly from the primate eye-brain system. The primate uses combined information5 from color, shape and other cues. Our e-eye isolates the color aspect, and therefore allows us to analyze that pure component of information.

We initially carried out this study to answer the following request from the NCR division of AT&T (now NCR Corp.): "Can fruits and vegetables be identified reliably by their color alone?" We assessed the statistical probabilities (with results implicit in the data presented here), patented an identification system2 and then noticed the interesting evolutionary connections discussed here. Experts on pigmentation have noted for some time that the colors are important and have discussed the chemicals that give rise to them. We expand the prior work by adding some quantitative statistical analysis that adds further evidence to the notion that the colors have value to the plants in an evolutionary framework.

A good example of our results is the unsurprising, but illustrative finding, shown in Figure 1, that our e-eye can easily distinguish an orange from a green leaf. What we believe is interesting is our quantitative analysis of the spectroscopic distinguishability. The comparison of orange and green represents a good example of our analysis of the colors of all the fruits and vegetables. Figure 1 shows the absorption spectra of the pigments chlorophyll and carotene. The primary photosynthetic pigment of all higher plants is chlorophyll-a, whose absorption spectrum is shown as the dashed curve in the upper left of Figure 1. Chlorophyll-b, solid line, is also present in virtually all higher plants. The absorption spectrum of both has two strong peaks, one in the red part of the spectrum (near 650nm) and the other in the blue (near 450nm). Between the two peaks, in the green, the absorption is weak and the reflected light is relatively strong, leading to the green color. The diffuse reflectance spectrum of green-leaf cabbage, labeled green, is shown below the chlorophyll spectra and the weak reflected light in the red and blue, as well as the strong reflection in the green is clear.

At the bottom of this figure, we plot results obtained from a regularized version of classical linear discriminant analysis (LDA). LDA is a well-known statistical tool for understanding how a collection of variables differs between classes. In our context, LDA constructs a sequence of linear transformations of the reflectance spectra that "maximally separate" the measurements obtained from different types of fruits and vegetables. For this simple example, we have chosen four classes of green items (green cabbage, granny smith apples, green leaf lettuce, and romain lettuce), and 5 of orange or red items (sunburst tangerines, juice oranges, plum tomatoes, red delicious apples and bunch carrots). In Figure 1, we plot the coefficients of the first (left) and second (right) LDA transformations as curves. (That is, the first discriminant variable is obtained by taking the dot product of a spectrum and the coefficient curve.)

Alan. There are two samples here. One Gordon/Don cooked up to give pretty pictures for Figure 1. It does not use the entire dataset, but a handful of selected items as I indicated in the text. From what I recall, results for the full data set are similar, but perhaps not as strikingly interpretable as "looking just like" an absorption spectra. I will check with Don on monday for the curves from the full data set and see if I can do some more informal descriptions. I personally think that the case can be made for the components "responding to" different chemicals (as I have put in the text below) without having to say that they look just like each other. The coefficients from the PDA are orthogonal and hence have sign changes, while the absorption spectra are positive.

Before providing specifics about LDA and the regularized version we applied in this context, we first discuss features of the transformation that are evident from Figure 1. The first curve can be seen as largely a difference between the reflectance at 550 nm, a point with little chlorophyll absorption, and the reflectance at 650nm, a point with high chlorophyll absorption. This result holds up for the regularized LDA obtained from all the classes of fruits and vegetables in our data set. We learn from this that the dominant color of fruits and vegetables in the typical supermarket is green. Although there are some variants of the chlorophyll molecule, the green in all these varieties, particularly in the leafy vegetables is similar. {We can make this similarity quantitative by considering the identification accuracy. We find that the largest probability of confusion, .12 ? , occurs for the green items, accounting for most of the confusion in the entire data set, where the total confusion probability is .15.} In other words, green is a well-defined reference color of the plant world – again a case in which our results provide quantitative confirmation of an important botanical precept.

The second transformation (lower right panel of Figure 1) is, by construction, orthogonal to the first. This orthogonality represents a multidimensional, mathematical representation of Newton’s concept of complementary colors and of the layman’s concept of a "very different" color. The second transformation seems to capture mainly the height of a reflectance curve near 600nm, the point where carotene has a broad absorption peak. Therefore, just as the first component is similar to green, the second is similar to orange. The fact that this component is the second one indicates quantitatively that this color is the second most common. The orthogonality of this component indicates that this color is almost as different as possible from green. So, this multidimensional analysis shows that the distinctive orange (and the similar red) colors are statistically nearly orthogonal to the green that dominates leaves and the leafy vegetables. Through this informal discussion, we see that a data-analytic tool like LDA is capturing features in the data that can be understood in terms of the underlying chemistry of plant pigments.

Our primary interest in LDA is as a tool for classifying new observations. For this, we would like the different colors to be as separate as possible. For the green, orange and red items listed above, we have constructed the transformation represented by the first three LDA components (the first two of which are given at the bottom of Figure 1). The colored plotting characters indicate the color of the item (for example, green for cabbage and lettuce, orange for the tangerines and red for the tomatoes and red delicious apples). Notice that the first two components easily separate the green produce from the rest, while greater separation between the red and orange items can be achieved by including the third component.

We now briefly describe the regularized LDA procedure applied throughout this paper. First, we introduce some notation. Assume we have g different categories of produce, and that we have collected n1 , ... , ng spectra for each group. Because we record reflectance at w fixed wavelengths, each curve can be thought of as a vector in w-dimensional Euclidean space. Let sij denote the j-th curve in the i-th group, where i=1,...,g and j=1,...,ni. Let s1., ..., sg. be the mean spectra calculated for each class; that is, for each variety of produce, we form an average of the reflectance values wavelength by wavelength. Viewing the curves as vectors, these class means are simply obtained by the coordinatewise average

si. = (1/ni) sum j=1...ni sij
In discriminant analysis, we attempt to find combinations of the input variables that make the separate groups appear as different as possible. Therefore, we consider new variables bT s. To measure the difference between these derived variables, we can appeal to the usual F-ratio for testing the equivalence of the means for each group. That is,
F = (n-g) SSB / SSW (g-1),
where n = n1 + ... + n g is the total number of samples collected on all the varieties. Aside from the scaling by (n-g) and (g-1), this ratio captures the difference between the so-called within- and between-group sum of squares. These quantities are defined as SSB = bTSB b and SSW = bT SW b, where the within-class covariance matrix is defined by
SW = sum i = 1 ... g sum j = 1 ... ni (sij - si. )(sij - si. )T
and the between class covariance matrix is defined by
SB = sum i = 1 ... g sum j = 1 ... ni (si. - s.. )(si. - s.. )T = sum i = 1 ... g ni(si. - s.. )(si. - s.. )T
With these definitions, the F-ratio becomes
F = (n-g) bTSB b / bT SW b (g-1),
Large values of this ratio indicates that the separate group means (the average spectral reflectance curve associated with the different varieties of produce) are different. The scaling by (n-g) and (g-1) assures that when there is no difference between the groups, the ratio has an F-distribution with (n-g) and (g-1) degrees of freedom. To find new variables that are good for discrimination, Fisher suggested maximizing this ratio with respect to b , and it is in this sense that we mean the different groups are well separated.

Either by differentiation or by elementary facts from matrix analysis it should be clear that maximizing the F-ratio gives rise to the eigen system

S W -1S B b = l b
Hence, the coefficients b of the transformation that maximally separates the different groups is just the first eigenvector of the matrix S W -1S B. The subspace of dimension k that provides the greatest separation is obtained by taking the first k eigenvectors. It is common to interpret the LDA coefficients as we have for the curves in Figure 1. As such, this transformation can provide us with new insights into the structure of our data. By taking only a modest number k of the largest eigenvectors, we also accomplish a dimension reduction. Classification in this reduced space is performed by a simple nearest-neighbor rule.

Underlying our use of the F-ratio is an implicit assumption that the measurements for each class follow a multivariate normal distribution with possibly different means but a common variance-covariance matrix (dropping the common covariance assumption leads to quadratic discriminant analysis). While LDA is known to be fairly robust to this assumption, it is known to suffer when faced with many highly-correlated input measurements. In our case, this occurs when the reflectance spectra are smooth. Because the chemicals that make up plant pigments have broad-band absorption spectra, the reflectance data tend to be very smooth in the visible range (with our e-eye, the measurement error increases outside of the visible range, making the curves rougher). When the data are smooth, the within-class covariance matrix will have strong positive correlations, so that a spectral decomposition of S W will assign larger eigenvalues to smoother eigenvectors. The quantity of interest for LDA, S W-1, however, will have the opposite behavior, favoring rough eigenvectors.

Therefore, when analyzing smooth signals, it is preferable to offset this effect. Regularization can aid both interpretability as well as boost classification rates. Many authors have considered this problem, and we have chosen a relatively straightforward procedure called penalized discriminant analysis (PDA). Essentially, PDA differs from LDA by replacing S W with S W + O, where O is a "roughness" penalty matrix. In our application, we take O to approximate the usual smoothing-spline functional

int (f'') 2
Because our curves are sampled at discrete wavelengths, we use an approximation to this quantity based on second differences. In so doing, we can write the above penalty in the form sT O s.

The next result of our statistical analysis of the e-eye data shows that the wavelength range of information content in the pigmentation of fruits and vegetables is maximized in the range of wavelengths that matches the receptors of color vision. In this case, our data analysis shows some aspects of the colors shown in Figure 1 that cannot be discerned by our eye. The pigments in fruit and vegetables don’t have much "color" outside the visual range of wavelengths. In keeping with the sum rule on optical absorption spectra, this efficiency of the pigments allows more information to be concentrated in the visible. Both these factors give a quantitative underpinning to explanations of what makes the fruits (and vegetables) easier to distinguish.

Figure 2 illustrates our quantitative results on the wavelength range of the spectral information contained in the colors of the fruits. We found that the wavelength range of useful information in the color matches the range of sensitivity of primate vision. The data were taken using our e-eye, with appropriate calibrations with dark, white, and standard color references to make the results independent of the sensitivities of the detector arrays, the light source and the details of the dispersing elements. We measured nearly the entire contents of the produce section of a supermarket, including nearly a hundred varieties of fruits and vegetables, many examples of each and 5 to 20 parts of each item. We later repeated the measurements for different supermarkets and for different times of the year, but found the same results within our statistical uncertainty. Initially, we recorded the spectra over a relatively wide spectral range of wavelengths from 350? to 900?nm. One of the first questions that we asked was "how wide a range is needed to identify the varieties?"

We used the accuracy of identification as our criterion for evaluation, as shown by the abscissa in Figure 2. We measured this accuracy by dividing the data into a reference set and a test set (2/3 to training and 1/3 to test), although the identity of each spectrum was known. Then, after transforming the spectra into PDA coordinates derived from the test set as outlined above, we apply a simple nearest-neighbor rule to compute the most likely identity of the members of the test set. Using the data over the entire wavelength range, we obtained an average accuracy of .85, shown as the maximum value in Figure 2. We then eliminated data for wavelengths longer than a set value and recalculated the analysis of accuracy. These results are indicated in the figure using abscissa values of the cut-off value. Similarly, we investigated the short wavelength limit by including all data except that below a cut-off wavelength, again indicated by the cut-off values.

The results show that the identification accuracy by our e-eye is reduced by less than 5% if data is included in the wavelength range from 400 to 725nm. A direct examination of the spectra confirms this conclusion qualitatively. For wavelengths longer that the visual range, the spectra of all fruits and vegetables are nearly flat, indicating the absence of absorption modes. For wavelengths shorter than the visual range, the absorption has features, but the absorption is so strong that the intensity of diffusely reflected light that we measure is very weak. For comparison, Figure shows the range of sensitivity of a human eye. For sensitivities above 5%, the wavelength range that we see is 400 to 700nm.

Discussion of figure 3 on number of colors vs. ID accuracy.

Discussion of figure 4 on resolution.

 

 

 

Conclusion:

As early as the work of Darwin, evolutionary biologists have noted that the range of visual sensitivity in primates has evolved in a symbiotic relationship with the evolution of the range of colors taken on by plants in their environment. Our results on the matching of the wavelength range over which the information content is high, merely provide new quantitative confirmation of these important evolutionary precepts.

cabbage, granny smith apples, green leaf lettuce, romain lettuce sunburst tangerine, juice orange Plum tomato, red delicious apples, and bunch carrots examples are green cabbage and common juice oranges