Who had the idea to use this technology for this particular purpose? What's the idea behind the project?
Tilo: It was over four years ago that Peter Barham, a professor of physicists in the physics department here, and penguin enthusiast, approached the computer department here with an observation that African penguins each carry individually characteristic spot patterns on their chests and whether there'd be any opportunity to utilise that fact. And at the time Professor Barry Thomas was my supervisor for my PhD (he's unfortunately passed away now), but at the time he was key to suggesting that I try and do something with this information.
So I was the first person to research the area to come up with an engineering solution to utilise a naturally growing pattern that is unique to each animal, in order to tag the animal, without actually tagging it, without using synthetic markers. By doing that we could help conservationists and behavioural biologists to study, understand, and hopefully, in the end, conserve the species in a better way than is done today.
How do you think the project has gone? Have you achieved what your aims were?
Tilo: We have a prototype working so we've shown that it's possible not just as a concept but as something that works in a practical way. But we're by far not at the end of the road. Our current task, which came with a grant from the Leverhulme Trust three years ago, was to take the prototype that we developed and turn it into an actual monitoring system for a penguin colony on Robben Island, in South Africa. And we've basically begun to collaborate with the University of Cape Town there, with Professor Les Underhill, from the animal demography unit, and we've been working together for one and a half years now.
We started to build a system on Robben Island, which is a work in progress. We now have the first results which show that things work but there is still lots of work to do in installing the infrastructure and testing the system under African conditions, meaning very bright sun, and a high dynamic range of lighting situations, which means that cameras must work in scenarios where the environment changes quickly. So the visual content of the image in the cameras is very dynamic and highly variable, which demands a lot of adaptations to the system to keep it viable.
Innes: Ultimately, the idea is to have a system that you can hand over to biologists, or population ecologists, without the technical know how, and with simple instructions on where and how to install the cameras so they can be up and running and collecting data without needing to have their hands held by computer scientists at every stage of the way. So it has to be a robust system.
What is your own input into all this then Innes?
Innes: Well I got involved through meeting Peter Barham who supervised an undergraduate project on penguins at Bristol Zoo and he knew that I was interested in animal behaviour. So when the automatic recognition project started he invited me to come on board the team as someone who is interested in animal behaviour and who can see the applications and potential for such a system to use with a wide variety of animal groups and situations.
Also much of my personal research, at the moment, is on animal vision and I'm increasingly using computational techniques to try and understand how animals see the world. There's a geeky aspect to the project that appeals to me as well, independent of any animal behaviour interest.
Is this utilising newly developed software and, if so, who developed it, yourself?
Tilo: Yes. I basically put the software together for work contained in my PhD. It originates from two previously existing areas; one is human biometrics, which one might know as the fingerprint and iris recognition systems and appears not only in police stations but also at airports where people are processed and uniquely identified using naturally grown patterns on us.
However, we took that work ahead and took techniques from the field of computer vision, which is more or less the core area in order to apply the fusion of these two fields to what I would call Visual Animal Biometrics to help the biologists. In the end computer vision tries to makes computers see. So that's the large task.
Our specific application of research at the moment is to see if we can identify a species in its natural habitat and once identified, normalise, find and extract the unique patterning that it carries, and then, after extracting this information, compare it against the population database in order to tell which animal is currently in front of the camera and from there trying to reconstruct population dynamics, movements, and fluctuations. Basically, the data that the camera can deliver is 'which animal is where at what point'.
Innes: That might sound simple but to a biologist that might revolutionise what's possible. Currently, to get an accurate estimate of how big a population is, to know a populations 'toings and froings', who's where and when, you have to capture and mark them by hand, with tags or paint spray, and that involves a lot of human effort and can be stressful for the animal and you are severely limited in the number of individuals you can capture.
You're really doing very well to get one or two per cent of your population marked. Then to get the essential data you have to have people sitting there with binoculars identifying these individuals when they're resighted; So that data is very hard won, with a huge amount of human effort and time for a very limited amount of information about the whole population.
Even the system, as it works now, which is far from perfect, would collect many orders of magnitude more, higher quality data than is currently possible using manual techniques.
Were penguins chosen for this study because you were already looking at the previously mentioned population?
Innes: Penguins were chosen because Peter Barham is fascinated by penguins and in his spare time, when he's not being a physicist, he goes out to South Africa and studies penguins. So he got involved in the conservation research out of a personal interest in penguins. But in many ways they're a very good model system for this type of technology because first, in terms of selling the idea, most people would think that penguins look identical so it looks like quite a challenge but in fact clearly they're not because they have unique spot patterns.
What's more, they're black and white so it's a nice high contrast pattern, which makes the image acquisition problem solvable. And they waddle along in nice predictable patterns between their nest site and the beach so you can set up a camera on traditional walkways and be fairly certain that most of the population will stream by at some point in the day.
Are there any species that the system wouldn't work for?
Innes: I would say polar bears are probably difficult! It's got to be an animal with a distinctive pattern.
Tilo: In my PhD I've shown that all animals with so called Turing-like code patterns can be identified with the technology that we've developed. However, beyond that, there are a multitude of animals that carry non-Turing patterns which are still uniquely characteristic to an individual. Though a concept that sits beyond the pure mathematics of what we have show so far is applicable to a great variety of animals; amongst them a great many animals that are either endangered or in decline, like the African penguin.
If you go back to the 1930s there were around a million penguins around and nowadays numbers range from around 100,000 to 150,000. Hence, there's been a significant decline over the course of not even a century and that is another motivation to better understand and find the reasons for that decline.
Is this quite an expensive system to run and could you only really justify its use on an endangered species? Or could it be used on more common animals, such as Zebra?
Tilo: Actually the system has been trialled on zebra. From the very beginning I felt that the system has to be able to run on consumer hardware, which is out now to buy from the shops, for comparatively little money. For a system we need one good camera, which needs to be flexible, in terms of the infrastructure, (which costs approximately £600 to £1000), plus a laptop computer. Then those together give a client system that sits out on site.
Innes: This is all technology that is becoming cheaper by the week. People already have surveillance cameras installed in nest boxes, in their back gardens, so they can watch the breeding birds in spring.
Tilo: From there you need a population data base that captures the uniqueness of the animals, stores the information, and links it to identifiers that we, as humans, can read and use for our biological surveys.
Hence, in the real world, these client systems will consist of a camera, a laptop, a small bit of wiring, and between, hook up to a server, which at the moment is on the island, but hopefully, in the not too distant future, we'll have a worldwide database for such data. This server will do the job of identification in a similar way that fingerprint identification works at the moment.
You've implied that the idea originated in the human sector, with biometrics. Can it now be flipped over and identify people via their facial patterns and bone structures, based on the fact that various indigenous tribes have particular features, etc?
Tilo: In general humans do not have code patterns and if we had them we would hide them with clothes. There's not much point for using code patterns when studying human behaviour and for identification. Therefore, we go for the part which goes for bits that can be acquired such as iris patterns, fingerprints and facial structures. There is an entire science working on this specific part for human applications.
However, certain mathematical methods and algorithms, developed for this project, can feedback into the area of human biometrics. Especially some algorithms which were developed for robust comparison of features and moreover, in some ways, of indentifying the part of the body that is the unique one.
Innes: There's a big parallel interest in the remote surveillance of the human population for security or policing reasons. One big advantage that we have with animals is that they're not trying to disguise their identity.
Looking ahead to the future how do you see the project developing, assuming you continue to obtain the funding for it?
Innes: One thing that I would be keen on would be for it to be about more than just identity and location and information, so they can actually start to reconstruct what the animal is doing from its movements and its posture. Many animals, penguins included, have characteristic display behaviours, so if you could train a system to recognise from an animal's posture what it was doing then you could get far richer data on what was going on in your population - who's interacting with who and so on. My personal interest would certainly be more in that direction.
So mine and Tilo's dream would be for there to be a publically accessible database where, if a member of the public takes some holiday photos of say a penguin, while on holiday in South Africa, they upload it to a centralised server that tells you details about the penguin that you have identified. But the scientists get the benefit of additional information of where that individual was at a particular time. Everyone carries mobile phones around with cameras these days so it could be really helpful. You could use these people as further remote, intelligent cameras, if you like.
Tilo: Penguins turn up at all sorts of beaches in South Africa so holiday maker's snaps could be useful on another level. To help understand where these animals are, where they go, and the good thing about a natural pattern is that it's intrinsic to the individual - they won't lose it - they will have it. You don't have to catch them to tag them. It makes them identifiable even by a member of the public, just by taking a photograph.
In terms of the behavioural aspect we have done some work on identifying locomotive behaviours and that specific work has been done on lions. Using the system for species identification, in order to find the lion, and then track it for a continuous number of shots, enabling us to say what locomotive behaviour the lion is actually showing. And that is the first step in the direction of identification of animal behaviour, which at the moment is still down to scientists sitting in front of videos they have taken and doing it manually.
Tell us about the exhibition you are involved in later this year, in June. What are you hoping to achieve from that?
Innes: It's at the Royal Society, which is just near Piccadilly Circus, and this annual summer science exhibition is meant to showcase the best of British science.
It's selling science on several different levels, so they'll be fellows of the Royal Society, policy makers, government research councils and MPs, all going along to see what's hot in British science, but there'll also be school kids too - 5,000 plus sixth formers going through and hopefully getting excited about doing a science degree.
Hopefully, the good thing about our sort of display is that it shows that you don't need to be pocketed in one particular area. As a biologist you can be using exciting computational techniques, and as a computer scientist you can be applying your technology to a completely different sort of real world problem that most computer scientists would normally probably not think about.
Why do you think less and less students are signing up to science courses in this country at the moment?
Tilo: My personal belief is that there's a stigma attached to it - that science is boring, sitting in a lab and basically doing calculations, which is obviously part of it but it is, by far, not the whole story. And as Innes said, especially now, science is discovering different areas which sit between our classical disciplines, which is really exciting. In our case, the combination of computer science, biology and physics is very exciting. We use transmission networks to get the data from A to B - all that, together, has created a new field, which is using machines to monitor animal behaviours, which is of real value to biologists and conservationists.
The other reason might be because people don't see science as a creative thing to do, like the arts are, but again this is totally not the case because science demands a great amount of creativity in order to step over the borders and boundaries which are there, in order to approach a problem from a new direction.
Innes: Biology application rates remain fairly buoyant, particularly at my end of biology, because a lot of people are passionate about conservation and they like the idea of doing research on big glamorous animals in the wild and it has a high media profile, particularly so in Bristol with the BBC Natural History Unit and many other wildlife TV companies based here.
But a shortfall in our students is that they're not sufficiently interested in, or trained in, mathematics and the numerical side of the subject, which is absolutely vital on all levels and it's often only by the latter stages of their degree, where they are engaged in research projects, collecting their own data, and have to analyse it in a statistical way, that they suddenly think that the maths they learned in school is actually quite useful and they want to know the answers. So it's also quite good to get across to school kids that it's actually very useful to combine maths and the hard sciences with biology because those techniques are very useful, even if your ultimate goal is saving tigers.
What can the BCS and its members do to help?
Tilo: Well funding is obviously one thing that would be very welcome for many reasons. First of all there is cost involved in doing the research and in dispatching the systems themselves. Funding is vital, especially in an evolving field, which possibly doesn't have a strong lobby existing. Funding is basically the trigger for the field to be not only successful, on a small scale, but also applicable to a wider range of scenarios and ultimately to have a stronger impact on saving our animals and understanding different species and their behaviour.
Innes: There's always a difficulty in getting interdisciplinary projects funded, whereby you have to appeal to and impress two different communities. Hence, if we were trying to sell this to biologists they'd say 'well, ultimately, the biology you're doing is counting penguins, is that's not so exciting, really'?! And if you approach a computer scientist they'd say 'well there's people doing this with human biometrics, so is it really that exciting?' where in fact it's the fusion of the disciplines that creates something that could never have been done before. And that's always a difficult set of cards to play, to sell. So part of our agenda, at this Royal Society meeting is to try and educate people, involved in many subjects, so they can see the potential of linking the fields together.
Further information
Tilo Burghardt is a research fellow at the department of computer science, University of Bristol and Innes Cuthill is a professor of behavioural ecology at the School of Biological Sciences, also at the University of Bristol.
