LED-Induced fluorescence and image analysis to detect stink bug damage in cotton bolls
© Mustafic et al.; licensee BioMed Central Ltd. 2013
Received: 14 June 2012
Accepted: 11 January 2013
Published: 20 February 2013
Skip to main content
© Mustafic et al.; licensee BioMed Central Ltd. 2013
Received: 14 June 2012
Accepted: 11 January 2013
Published: 20 February 2013
Stink bugs represent a major agricultural pest complex attacking more than 200 wild and cultivated plants, including cotton in the southeastern US. Stink bug feeding on developing cotton bolls will cause boll abortion or lint staining and thus reduced yield and lint value. Current methods for stink bug detection involve manual harvesting and cracking open of a sizable number of immature cotton bolls for visual inspection. This process is cumbersome, time consuming, and requires a moderate level of experience to obtain accurate estimates. To improve detection of stink bug feeding, we present here a method based on fluorescent imaging and subsequent image analyses to determine the likelihood of stink bug damage in cotton bolls.
Damage to different structures of cotton bolls including lint and carpal wall can be observed under blue LED-induced fluorescence. Generally speaking, damaged regions fluoresce green, whereas non-damaged regions with chlorophyll fluoresce red. However, similar fluorescence emission is also observable on cotton bolls that have not been fed upon by stink bugs. Criteria based on fluorescent intensity and the size of the fluorescent spot allow to differentiate between true positives (fluorescent regions associated with stink bug feeding) and false positives (fluorescent regions due to other causes). We found a detection rates with two combined criteria of 87% for true-positive marks and of 8% for false-positive marks.
The imaging technique presented herein gives rise to a possible detection apparatus where a cotton boll is imaged in the field and images processed by software. The unique fluorescent signature left by stink bugs can be used to determine with high probability if a cotton boll has been punctured by a stink bug. We believe this technique, when integrated in a suitable device, could be used for more accurate detection in the field and allow for more optimized application of pest control.
The southern green stink bug (Nezara viridula L.) (Hemiptera: Pentatomidae) is one example of a stink bug that feeds on and thereby damages plants. This species is thought to originate from Ethiopia, but is now found in many tropical and subtropical regions of the world. This species is commonly found in the southern United States where it is considered a major pest attacking agricultural crops. Stink bugs feed on a variety of wild and cultivated plants and crops including but not limited to legumes, nuts, and various fruits and vegetables [1–4]. One of the extensively planted crops in the southeastern U.S. susceptible to stink bug infestation is cotton (Gossypium hirsutum L.) . Most stink bugs feed on developing cotton bolls, which occur after anthesis. Both nymphs and adults are capable of feeding and damaging crops, and mature nymphs tend to cause more damage than younger instars .
The feeding mechanism of stink bugs includes the extension of a piercing mouthpart (proboscis), with which it penetrates the outer wall to reach the developing seeds. Pathogenic bacteria may be introduced during feeding, thereby adding an additional destructive agent resulting in further degradation of bolls, especially in less mature bolls [6, 7]. Cotton bolls damaged by stink bugs result in reduced fiber length, quality, and uniformity [8–10]. Another effect of stink bug feeding on cotton bolls is the lower rate of seed germination . These types of damage greatly diminish the value of cotton lint and can cause substantial impediments to a stable cotton supply. The most common approach to reduce stink bug damage is to wait until damage exceeds an economic threshold and then apply insecticides to mitigate the population.
While highly specific insecticides and genetic engineering of the host crop have become preponderant methods in agricultural pest control, these methods have not been effective against stink bugs. Therefore, stink bugs are generally managed using broad spectrum insecticides, such as pyrethroids and organophosphates, that kill the intended pests but also many non-target beneficial insects. Proper timing and application of insecticides is critical to reduce the risk of secondary pest outbreaks [12, 13]. To prevent financial losses, insecticides need to be applied only when the estimated damage exceeds an economic threshold. Moreover, the choice of insecticide chemistry can have a profound effect on efficacy. Along with choosing appropriate types of insecticides for application, timing is important, because organophosphates and pyrethroids have limited residual activity in the field. In other words, insecticides need to only be applied at the correct time and place or the grower risks erosion of financial gains.
It is of essential importance to continue research in this field because of the economic importance of cotton. The United States are the world’s largest exporter of cotton with 11.6 billion bales per year in 2012, which is nearly three times more than the next closest country, Australia, with 4.25 billion bales per year . The southeastern United States, which is the production region with the most stink bug damage, produces approximately 25% of the US upland cotton crop .
To develop an appropriate plan for pest control in the field, two requisite and codependent challenges regarding sampling and damage detection exist. The currently recommended practice of sampling cotton bolls is time consuming, burdensome, and destructive to the test population of cotton bolls. Recent studies describe various issues with monitoring stink bug populations which are predisposed to aggregate in fields and can be difficult to detect with the naked eye [16, 17]. Even with optimized sampling plans, cotton boll examination for potential damage is an invasive procedure which renders the bolls useless because it requires them to be cracked open. Development of a new method with proven accuracy and efficiency to detect stink bug feeding is necessary.
In this article, we propose an image analysis technique that builds on previous research by Xia et al. : the presence of different fluorescence emission was reported, depending on whether the cotton boll was damaged by stink bugs or not. Stink bug feeding results in a unique fluorescent signature on the cotton boll wall that can be imaged and analyzed for discriminating properties. One of the advantages of this technique is that it is non-invasive and nondestructive. Here, we report on an image segmentation and analysis process capable of distinguishing feeding marks from stink bugs versus those caused by other factors.
Southern green stink bugs were reared in the laboratory on a diet of fresh green beans or okra pods following the methods of Harris and Todd . Briefly, adult stink bugs captured in the field were brought into the lab and placed in 37.9 liter glass aquaria lined with paper towels. Adults oviposited on the paper towels. These were transferred to ventilated petri dishes until the eggs hatched. Nymphs were maintained in petri dishes or small plastic wide-mouth jars at 25.0°C and 65% relative humidity. Adults used for infesting bolls were less than two weeks old and of mixed sex.
Cotton plants were started from seed and grown in the greenhouse to produce bolls for the study. Three cotton seeds (FM 9063 B2RF) per 11.35 liter plastic pot were sewn in Metro Mix 300 growing medium. After germination, two of the plants were culled and the remaining plant was fertilized monthly with Osmocote 14-14-14 and Micromax fertilizer (the Scotts Co. LLC, Marsville, OH). Once the plant began flowering, individual white flowers were marked daily using flagging tape and then allowed to develop normally for 10 to 14 days. When the bolls reached 10 to 14 d past anthesis, a single southern green stink bug adult was caged in a mesh bag on each boll and subtending leaf for a period of 48 h.
After treatment with the stink bug, bolls were excised from the plant and shipped with overnight service to the Athens campus for imaging. At the time the bolls were cut from the plant they had an external boll diameter of 2.3 to 2.5 cm. Control bolls were treated exactly as described, except that no stink bugs were introduced into the mesh bags. A total of 136 bolls were harvested that were exposed to a stink bug, and 109 control cotton bolls were harvested.
Sixteen additional cotton bolls were punctured with a sterile syringe needle (31 Ga, 8mm long, Beckton-Dickinson, product no. 328418) as a control. These bolls were treated exactly as described in the previous section with two differences: (a) no stink bugs were introduced into the mesh bag. and (b) before harvesting, the bolls were gently cleaned using rubbing alcohol on a cotton swab to remove any microorganisms that could contaminate the boll. Puncturing was done manually, and care was taken that the needle penetrated into the lint tissue. The punctured bolls were harvested immediately. Bolls with needle punctures were processed in the same manner as the other bolls. The purpose of these controls were to examine the difference between purely mechanical puncture damage and the specific puncture damage caused by stink bugs.
All images were acquired with lens aperture set at f/11 and an exposure time of 15 seconds. All automatic camera functions were disabled, the white balance was fixed to 5500 K, camera sensitivity kept at ISO 400, and the camera set to acquire raw images in MRW (Minolta Raw) format. Each cotton boll was placed on the rotary stage and rotated 90° after each image acquisition, resulting in four images per cotton boll. Raw-format images have a depth of 12 bits per pixel, and all images were converted with UF-Raw (http://ufraw.sourceforge.net) to 16-bit TIFF images with the 12-bit depth fully retained.
Candidates for stink bug puncture marks were identified with a local maximum filter with the a priori settings of a peak-to-peak distance of 15 or more pixels and a minimum intensity difference of 10 to the average inside a 15-pixel radius and segmented with a variant of the greedy snake algorithm . This snake algorithm is a special variant of active contour models. A circle is placed near the fluorescent mark that is larger than the actual mark, and whose placement is uncritical. The circle is discretized into 32 vertices, and the vertices are iteratively displaced to minimize the total snake energy. The snake energy is defined as the sum of the vertex distance (stretching energy), the path first derivative (bending energy) and the negative image gradient (external energy). With this energy formulation, the snake tends to contract in a regular (i.e., circular fashion) until it locks onto the image gradient, and the iterative displacement ends near the gradient maximum of the boundary of the fluorescent mark.
Statistical analysis was performed in R (http://www.r-project.org). The ability of the area metric and the intensity metric to distinguish true and false positives was tested individually with the Kruskal-Wallis test and Dunn’s multiple comparison test. The choice of the Kruskal-Wallis test over one-way ANOVA was dictated by the nonparametric distribution of the data. Deviation of a sample’s median value from a hypothetical value was tested with the Wilcoxon signed rank sum test. A significance level of α = 0.05 was assumed.
The purpose of this study was to provide proof-of-principle that LED-induced fluorescence combined with image analysis can provide an indicator whether a cotton boll has been fed on by stink bugs or not. Here, the most important component was the ability to detect stink bug-related fluorescent marks on the outer carpal wall without the need to manually open the boll.
The starting point for this study is an earlier publication , in which we reported on the observation of a characteristic green fluorescence emission near stink bug puncture sites. This fluorescence was prominently visible with the unaided eye under ultraviolet illumination on the lint and the inner carpal wall. We also observed that some fluorescence becomes visible on the outer carpal wall, while at the same time the red chlorophyll emission recedes.
Any bolls that were punctured with a sterile needle exhibited a similar fluorescent pattern on the outside carpal wall and wart development on the inner carpal wall, although the outside diameter of a 31 gauge needle (0.26mm) is larger than a stink bug proboscis, estimated at 0.17 mm . Unlike in bolls punctured by stink bugs, lint coloration and the later development of shriveled and dried lint as well as mold was absent. We conclude that the fluorescence is a by-product of the plant healing process, but that stink bug feeding introduces additional damage due to pathogen contamination. For the image analysis process, however, this observation is of no relevance.
One challenge for the image analysis method is the presence of fluorescent regions that are not related to stink bug feeding (Figure 5E and F). In fact, out of 109 control cotton bolls that were not exposed to stink bugs, 41 showed fluorescent marks, and a total of 49 fluorescent marks met the inclusion criteria and were included in the analysis. Conversely, out of 136 cotton bolls that were exposed to stink bugs, 38 did not show any fluorescent marks, and manual examination of the inner carpal wall and lint confirmed that these bolls had not been fed upon by the stink bugs (i.e., true negatives). Of the remaining 98 cotton bolls that exhibited fluorescent marks, a total of 169 marks were included in the analysis. Manual examination of cotton bolls not included in this study led to the inclusion criteria, namely, a minimum area, a minimum average intensity, and a strictly convex shape. It is conceivable that small marks are caused by non-piercing insects, such as spider mites , and large, elongated marks are the consequence of mechanical abrasion.
Clearly, any image analysis method must detect false-positive fluorescent marks. The first step of the image analysis chain was aimed at detecting all fluorescent marks, irrespective of their origin. Local maximum detection combined with either gradient-based segmentation (e.g., the hill-climbing algorithm  or active contours (“snakes” ) or intensity-based segmentation, such as region-growing, can be used in a straightforward manner to provide the initial set of candidates. Based on the initial set of cotton bolls, we defined all fluorescent marks that were found on the control group of cotton bolls as false-positives, and all fluorescent marks that were found on the exposed group of cotton bolls as potential true positives. The fluorescent marks associated with needle punctures (total of 81 marks) were examined as a separate group.
A second quantitative metric was obtained by measuring the intensity drop-off towards the center of the puncture mark. False-positive marks tend to have an irregular intensity distribution with the highest intensity near the center. Conversely, plant tissue that was directly damaged by the puncture was almost black under visible-light examination, and its corresponding fluorescent emission lower. It can therefore be expected that an intensity ratio of the center intensity I1, relative to the average intensity of the entire fluorescent spot I2, tends to be less than 1 for true-positives and greater than 1 for false-positives. In some cases (such as the one displayed in Figure 5G and H), the dark center region was not as pronounced as in most cases, and we accepted an intensity ratio of I1/I2 < 1.2 as indication of a positive (i.e., stink bug-caused) fluorescent mark.
It is interesting to note the relative similarity between fluorescent marks caused by stink bug feeding and those caused by needle puncture, which is consistent with our earlier publication , where we hypothesized that the fluorescence emission is the by-product of the plant healing process. Equally relevant is the observation that the fluorescent marks do not fade over a period of seven days (i.e., the maximum period over which we examined a cotton boll). This finding confirms those we reported earlier .
Contingency tables for separation of fluorescent marks by their area A , their intensity ratio I 1 / I 2 , and both criteria simultaneously
A< 0.6 and
A≥ 0.6 or
Infested n = 169
Control n = 49
For practical purposes, an 87% detection of true positives with only 8% false-positives is already acceptable when we consider that the main goal of the application of this method in the field is the approximate determination of the level of stink bug infestation. The reason for this conclusion lies in the study by Reay-Jones  who found an average damage rate of 14.8% on a scale from 0% to 100% in a very large sample from commercial cotton fields in Georgia and South Carolina. Moreover, Extension-recommended treatment thresholds range from 15% to 30% internal boll damage, depending on week of bloom . In our previous study of the new fluorescent method , accuracy for the conventional visual inspection was found to be 75%, while the accuracy for the florescent detection method was 92.9% with a 7.7% false positive rate when a human observer analyzed the fluorescent images.
A concurrent development of a method to detect stink bug presence and damage to cotton bolls is an ”electronic nose” sensor for specific volatiles emitted by stink bugs or damaged plants . Under laboratory conditions, the internal boll injury (interior walls of bolls with warts and lint damage) was predicted in 95% of all cases , and the electronic nose was able to identify injured and healthy cotton bolls with an accuracy between 80 and 90% . Notably, the detection accuracy of 84% reported in this study is in a similar range as the one reported by Degenhardt et al. .
In our previous work , we identified a fluorescent wand and a fluorescent image scanner as possible devices to detect stink bug-related fluorescence. Further analysis of the fluorescent marks, as presented in this study, reveals that the wand is likely impractical due to the presence of a large number of false-positive fluorescent marks. The second device that we proposed is a light-shielded enclosure in which fluorescence emission can be detected by pushing the cotton boll into the enclosure. With a suitably engineered device, the cotton boll might even be left on the plant. Irrespective of the final embodiment of a field-deployable device, we can expect a significant increase of cotton boll throughput and analysis speed. Based on the study by Reay-Jones , the appropriate sample size for arriving at a treatment decision is 15 samples (20 bolls per sample) under the assumption of a boll injury rate of 14.3% and the accepted error rate of 30% of the mean. Toews et al.  compared the time required for internal examination of bolls and found that a single 20-boll sample required 7 minutes to collect and process. Since field scouts are unable to invest the required time to reach this level of precision, they tend to collect fewer bolls and consequently accept a much higher error rate. The fluorescent detection promises much faster sampling as image acquisition and analyses could be performed instantaneously. Examining more bolls in less time promises a more accurate estimate of boll damage. These data strongly support that the fluorescent detection method is a major step forward in making more accurate estimates of stink bug damaged bolls for making pest management decisions.
Among the many challenges facing agronomists in growing cotton, two stand prominently: detection of internal damage and determination of its underlying cause. With respect to cotton, current methods include harvesting immature cotton bolls and opening them so any damage can be visualized. Even if no damage is observed, the cotton bolls are lost irretrievably due to destructive sampling. Furthermore, small bolls are nearly always shed by the plant as a result of stink bug feeding . In that case, the damaged bolls would not be included in the samples, because they would have been dropped by the plant before reaching appropriate diameter for sampling. It is also desirable to detect boll damage very quickly so that the stink bug population can be mitigated with insecticides before the bugs move to new hosts.
To overcome these limitations, we present one step towards a field applicable imaging method to estimate the levels of stink bug infestation. The next step of this project is the engineering design of an imaging scanner that can be used in the field. We envision a hand-held imaging apparatus, where the illumination source and an image sensor array are placed in an enclosure. A cotton boll that is placed in this enclosure needs to be shielded from environmental light. One desirable property of this enclosure that allows the boll to stay on the plant, although sampling by harvesting some bolls and inserting them in the imaging apparatus may turn out to be a suitable alternative. An additional motivation for developing a device that can analyze the boll on the plant is its use in research to examine fluorescent marks and stink bug damage over a longer period of time.
Either way, with the image analysis methods described herein, the theoretical foundations towards a field-usable detection device for stink bug damage in cotton bolls are laid.
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.