The falling cost of digital cameras - and storage - have opened up a whole range of applications for computer vision. One area which is just emerging is that of precision modelling in biology - the data to build these models (sizes and shapes for example) are often measured from images.
Smart camera systems are watching all sorts of biological entities, from whales to wheat. This links to phenomics, a discipline within biology, looking at the large-scale measurement of the physical and biochemical traits of an organism: these traits are determined by an interaction between the genetic makeup of the organism and its environment.
A plant’s genome is the genetic code lying within its cells, but the phenome is the way in which that gets expressed - how tall it is, what shade of green, how it reacts to sunlight, and so on. Many of these characteristics can be caught on camera, and if you can take an image of it, computer vision can try to interpret that picture and create a measurement.
Plant phenomics lies at the heart of addressing the problem of food security - how to feed an increasing population - enabling us to understand the relationship between genetics and the environment that will lead to the ability to breed plants with better resistance to drought, or crops with higher yield. Plants are at the centre of food supply chains, but of increasing importance is their use as biofuels.
At the moment, understanding how plants grow is still largely carried out manually by measuring physical aspects such as height, growth rate, number of flowers, root system spread, leaf size and shape etc.
Within plant science, recent technical developments in gene sequencing mean that for many species the genetics are becoming well understood; our ability to make many measurements characterising the plant (the phenotyping) has lagged behind and become the bottleneck. Computer vision systems are ideally suited to solving this kind of problem, automating measurements on the images and thereby removing subjectivity and increasing throughput of data analysis.
It is becoming the case that biologists will no longer have to measure thousands of wheat plants by hand to produce the phenotyping data. Instead, they can take photographs as they grow, and computer scientists will be able to align the images, calibrate and correct for camera geometry, then automatically measure the plants, detecting those that are faster to establish or slower to wither when starved of water. Being able to image and analyse both above ground shoots and below ground roots is crucial to understanding the condition of the plant as a whole.
In the UK there are two main centres where computer scientists and biologists work hand in hand to solve these kinds of problems. In Aberystwyth, there is the National Plant Phenomics Centre, a large automated greenhouse with many monitoring systems (including cameras). In Nottingham, the Centre for Plant Integrative Biology has focused on understanding root growth and development.
Aberystwyth’s Plant Phenomics Centre
The National Plant Phenomics Centre at the Institute of Biological, Environmental and Rural Sciences (IBERS) at Aberystwyth is home to an automated greenhouse that enables the high throughput analysis of growing plants - exactly the kind of system needed to help tackle the phenotyping bottleneck. The greenhouse, which is funded by the BBSRC and the Welsh Government, has a floor space of 750m2 and provides a national capability for use by research scientists from academia and industry across the UK.
Plants are grown in the greenhouse where different environmental factors affecting their growth, such as temperature, watering and nutrient regimes can all be carefully controlled and monitored. The plants grown and imaged in Aberystwyth include food crops such as oats, wheat, rye, oilseed rape, and maize, but also biofuel crops such as miscanthus. These plants are automatically carried around the greenhouse on a conveyor system in individually RFID tagged carriages (there are 882 in total) and then through to imaging chambers.
The five three metre high chambers can be thought of as a kind of plant photo-booth, each housing a different type of camera. Imaging systems currently installed include standard digital cameras, fluorescence and infrared cameras. As the carriages move into these chambers, the doors on each side are shut and photos are taken of the individual plants from the top and sides in reasonably controlled lighting conditions.
These images are all stored in a large database along with any relevant metadata, detailing the environmental factors the plants were grown under (watering regime, temperature and so on), and the camera settings used during the imaging. The images can then be analysed using various computer vision techniques to rapidly and automatically calculate things such as the height of plants, the number of stems or leaves and the total area (closely related to biomass).
This non-destructive automation provides useful metrics relating to the different phenotypes of plants to their genotype for many more plants than can currently be processed manually. This will give us much more useful information to help us select the best strains of crops for various needs, such as drought-resistant plants that will cope with possible future changes in climate.
Nottingham’s Centre
At the Centre for Plant Integrative Biology (CPIB) in Nottingham, a heavily multidisciplinary group of biologists, computer scientists, engineers and mathematicians are also addressing important plant science questions. Image analysis plays a crucial role in the interactions between these disciplines. Digital imaging allows the plants to be continually monitored, and measurements can be taken from the images automatically.
Phenomena like root growth can be measured from timelapse sequences of images using algorithms developed by computer scientists. This data is then fed back to the biologist for further analysis, or in some cases used in mathematical models. This could be at the cellular scale, right up to modelling the geometry of the entire root system or even the whole plant.
Additionally, aerial field-scale images allow data relating to entire fields of crops to become available for analysis. To enable the high throughput capture of images, the computer scientists need to interact closely with engineers to design and build robotic systems that can move a camera and hence automate the image capture process.
This interdisciplinary work at Nottingham continues with the development of the Hounsfield facility, due to open in May 2014 and funded by the BBSRC, ERC, Wolfson Foundation and the University of Nottingham.
This will be a state-of-the-art root phenotyping facility. Three μ-CT x-ray scanners will be able to image plant roots that are still growing in soil, trying to truly uncover the mechanisms of root growth in as natural an environment as possible.
One of the machines will be automatically operated, fed by an autonomous pickerrobot that selects plants from the attached computer-controlled glasshouse, and passes them to a robotic arm. This in turn will place the soil cores inside the room-sized scanner to produce a 3D image of the roots and surrounding soil structure. The amount of image data this autonomous system will produce is huge (an estimated four terabytes per week), which must be stored and backed up locally before being archived at a remote location.
The local technological challenges of storing and transferring this data are not small, and of course the data must also be analysed to retrieve useful information such as the total length of the root system, the volume of soil it surrounds, measures of root architecture complexity and so on. It falls to the computer scientists to develop image analysis approaches to automate these measurements, as the sheer volume of 4D (3D+time) data precludes any feasible manual alternative.
The future of biology?
Computers are playing a bigger role in many sciences now: big data can come from gene sequencers, or from video cameras, or from X-ray machines. These ideas aren’t new for those who’ve been involved in medical technology, but applying these to plants is a very recent development. Given the pressing issues surrounding food security in the longer term, intelligent agriculture would benefit greatly from the increased ability to understand plant growth, and therefore evaluate interventions (such as fertilisation regimes) objectively based upon image data.
But it is not all about large scale agriculture: the falling costs of sensors open the door to a whole range of applications including growth monitors as mobile phone applications. Imagine a world where hobbyist plant growers are able to photograph seedlings and determine whether they are using optimal watering and feeding regimes for their tomatoes.
