In 1992, Polish forests were facing unprecedented levels of death amongst European ash. Ecologists were discovering trees with wilting black leaves and dark lesions growing on their bark. 14 years later researchers revealed the culprit — a fungus of Asian origin — making its way to Europe via plant imports and possibly even via natural spore transmission in wildlife. Now, almost 30 years later, the disease is present across 18 European countries, including the UK. [1]
This disease creates not only ecological disturbances, but urban ones too. Local councils need to be aware of decaying trees that often buckle in catastrophic fashion atop of cars or end up blocking roads and other vital infrastructure. Home owners also have a stake here, as they are legally responsible for a tree on their premises which causes damage to others or their property.
“Even if trees do not die from [ash dieback], their growth and reproduction are diminished, and this reduces their value in terms of both ecological function and forestry yield.” [2]
Following the learnings from our previous study on identifying invasive species from roadside images, and continuing our partnership with UKCEH, we set out to devise an effective operational pipeline for surveying the ash tree population and diagnosing ash dieback.
Our plan was to create an object detector capable of spotting ash trees from roadside imagery, then inspecting the detected trees for the disease.
Scaling Up
We’ve come a long way from the woodland surveys of the past, abandoning waterproof paper for tablets and smartphones [3], but the crux of the problem remains the same: manpower. Covering large areas of land is an inherently time consuming process.
Our previous data acquisition approach, a car-mounted Canon 5D Mark IV, drastically reduced the time to gather data. Another benefit to this strategy is that it is much safer, removing the need for surveyors to walk along the roadside.
Dealing with vegetation can be very difficult for AI — a key takeaway from our first run was to focus on visually distinctive data — for ash that meant ensuring that the seed pods were visible.
We initially played around with the idea of creating a custom object detector but considering that popular high-performance convolutional neural networks are typically trained on hundreds of thousands of images [4], we opted to work with a pre-trained model. Our datasets were not only relatively small (1013 summer images, 659 winter images) but we also anticipated that the nature of the target objects (leaves, branches, etc.) may be conceptually difficult for any neural network.
We chose to use Faster RCNN ResNet101 at an input size of 640×640 and later 1024×1024. For reference, our original image size was 6720×4480 but we compressed the images to 1800×1200 before feeding them into each model. Part of the training pipeline included a fixed size rescaling step to match the required input size.
Training took roughly one hour on a powerful cloud instance equipped with:
Tesla P100 16GB GPUIntel Xeon CPU @ 2.20GHz13GB RAM
Results
The winter model achieved a precision of 94% and a recall of 67%. The summer model did not perform as well, achieving a precision of just 63% and a recall of 26%.
We were curious whether a larger input size will improve things so we ran the winter dataset through a 1024×1024 version of ResNet101. After tinkering with the data augmentation steps we achieved a slightly lower precision of 89% but saw improvements in the recall, at 77%. This improvement came at a cost however; the larger model was making more correct predictions but was less confident about each guess on average. For comparison, the smaller model had an average confidence of 76%, and the larger 52%.
Challenges Faced
Labelling a tree may sound like a simple task, but that would only be true for trees in isolation. In the wild, tree crowns don’t grow nicely into a box. They overlap with other trees, making it conceptually difficult to decide where one tree ends and another begins. Because of this we had decided to label regions of interest in each medley of crowns, resulting in multiple bounding boxes per tree. The idea was to give the model as much data about the structure of the tree as possible.
In reality, this may have confused the model, and more importantly it made evaluation difficult. The model was clearly detecting the trees but it used fewer bounding boxes to encompass the areas of interest. Calculating a meaningful IoU (intersection over union) was difficult.
Equipped with this knowledge we think it may be better to label with as few bounding boxes as possible. An even better, but more difficult to prepare, approach may be to use segmentation.
As demonstrated by the poor results, summer ash is difficult to label and difficult to detect. We suspect that adjusting the data augmentation steps in our training pipeline may yield better results. The larger input sizes may also contribute to finer details being picked up. Perhaps a fundamentally different technique is required for this challenge; one thought we had was to use the high-resolution images to give the model a detailed look of the leaves and use that in some way as the deciding factor.
Next Steps
Despite the winter ash model performing reasonably well, there is room for improvement. The summer model leaves a lot to be desired. We’ll be looking into changing our labelling strategy, as well as adjusting the data augmentation steps and other hyperparameters.
Extending the pipeline with dieback analysis on the detected trees is the logical next step. Two commonly seen symptoms are wilting black leaves and diamond shaped lesions.
One challenge that we’ll have to solve for is that certain symptoms, like the wilting leaves, are more prevalent and easy to spot during the summer months. That’s unfortunate for us since our summer model needs to do better to detect the trees in the first place. The lesions on the bark are also difficult to spot from the camera’s perspective. In any case, we may find that we need to change our data collection strategy if a competent analysis cannot be performed.
For future updates on the project, follow us on Twitter.