This week I had the privileged to attend two conferences in the US focusing on unmanned aerial systems (UAS) in agriculture. The first was the Delta AgTech Symposium in Memphis, Tennessee and the second was the Precision Ag Aerial Show at Decator in Illinois. These conferences are the beginning of my private study as part of my Nuffield Scholarship, thanks to my sponsor the GRDC.
As a forward to this discussion, I do focus more so on the fixed wing type UAS for large area mapping – see why here – and therefore generally do not comment on multi-rotor systems, GoPros, First Person View (FPV) etc even though these were covered at these conferences.
Both conferences were a combination of educated/independent speakers and vendors. There was a couple 1 hour multi-rotor demonstrations at Delta AgTech and two full days of demonstrations of fixed wing and multi-rotors at PAAS. The audience was extremely diverse – from curious farmers, agronomists, all the way to sensor & software companies and interestingly a large presence from insurance companies.
Themes I picked up on:
Everyone agrees that data needs to be more than pretty pictures – it needs to be actionable. Futhermore, boots still need to be on the ground, ground truthing data, explaining WHY. No one is claiming UAS replaces clever farmers and informed agronomists.
There is broad acknowledgement that UAS in agriculture needs to show a return on investment but only one case study was mentioned with actual figures.
Data processing is an issue. Some are claiming the solution is server based i.e. ‘in the cloud’ but acknowledge internet bandwidth is a bottleneck.
Sensors are a hot topic. Most manufacturers claim they have vastly improved sensors coming. Talk at the moment is around affordable true multispectral combined with irradiance/incident light measurement.
We were reminded a couple times not to dismiss satellite imagery as an option. Satellite sensors are always improving and getting cheaper. Will we have 10cm GSD from multispectral satellite in 10 years?
Generally frustration with the FAA on the time it is taking to develop rules around UAS. In saying this, there is acknowledgement that rules are needed.
Some other interesting points picked up over the last week:
We are starting to see some UAS companies offering early versions of vectorization (i.e. points, lines and polygons) in their server solutions using imagery collected via UAS with the main example variable rate application map of fertilizer in crop based on NDVI. This is not new technology – it has been done with satellite imagery for years.
With the speed of technology rapidly changing are we considering the upgrade path of current UAS? Can the sensor and GPS be upgraded without buying a whole new system?
Most of the commercial grade fixed wing UAS seem to be able to fly well, have autopilots that just work and acceptable ground control software. Product differentiation will probably be around sensor integration and options, support, innovation and whole system workflows including speed of data processing. This is just a snapshot of what was covered in the last week. I will be compiling a report that encompasses my whole USA/Canada trip which will be available towards the end of the year.
Built with a 3DR Pixhawk (APM:Plane 3.0.2), Finwing Penguin 2815 fixed wing air frame, S100 Canon camera and RFD900 Radio (excellent Australian product), my budget agricultural unmanned aerial vehicle (aUAV) has kept me busy and learning lots in any spare time over the last few months.
In my last post, Unmanned Aerial Vehicles (UAV) in Precision Agriculture, I outlined the main components of a UAV for precision agriculture focusing on a fixed wing platform for collecting high resolution paddock scale data. In this followup post I will attempt to log some of my experiences. Note that this is just a learning experience – there are many commercial UAV options available for agriculture that are less time consuming and provide similar or better results right away.
I needed a fixed wing platform that is readily available, cheap, with potential for long battery life, stability in the air and plenty of space for electrical components. I chose the Finwing Penguin. With the standard 2815 Finwing motor, 60 amp electronic speed controller (ESC), 9×6 propeller combined with a 4400Mha 3 cell Lipo battery I was only able to achieve about 20minutes flight time or enough to map about 40ha at 12m/s. I have a CXN engine mount which enables me to go to a 10″ prop which some of the gurus recommend. I could also increase my battery capacity and battery cell count to get longer flight time.
The Penguin is rather unique in its class as it has inbuilt landing gear. This consists of a pair wheels at the front and a single wheel at the back. I think this assists in preserving the plane when there is nowhere soft to belly land. The landing gear also allows you to take off like a traditional plane rather than hand launching. After making weight or centre gravity (CG) changes I will often take off from the ground. The downside is that these wheels to block up with mud quite easily if you land in a wet paddock.
The wings (including tail wing) come off for transport. I usually remove the main wings but leave tail wing in place as it is quite hard to get on and off due to awkward wiring and attachment arrangement.
The Penguin UAV does come with a pre-cut hole to install a down facing camera but it does not suit to place the camera top facing forward which is desirable. It was also very awkward to get the camera in and out as I had the Pixhawk autopilot installed above the camera position. I decided to go at the plane with a hacksaw and build a camera mount that would allow the camera to be installed from underneath the plane and also enough space to mount the camera top face forward.
Autopilot, GPS & Radio modem
As far as autopilot is concerned 3DR Pixhawk with APM:Plane 3.0.2 was the best option. At first I had issues getting my plane to fly well but once I upgraded to version 3.0.2+ the autotune feature changed the game altogether. This allowed the APM to adjust the PID settings in the plane as I manually flew it around. It works really well! During my latest flight I had an 8km/h cross wind that the APM was able to fly against successfully.
The GPS is a Ublox LEA-6M. It works well considering the price point. I did not attempt autonomous landing which is when GPS accuracy is more important. This GPS is able to get a fix within seconds of start-up and generally no issues throughout flight.
I initially used the 3DR radio modems but had all sorts of problems keeping a solid connection with my GCS. I decided to bite the bullet and buy a quality radio modem that should last me a long time and exceed all my range requirements. The RFD900 Radio pair is compatible with 3DR equipment and slotted in quite well. I did have to manufacture a cable to connect it to the Pixhawk and it took a bit of searching to figure which wires went where but I got it sorted within an hour or so. The RFD900 did have some driver issues on Windows. I had to install an old driver before I could get Mission Planner connected to the Pixhawk through RFD900. This all equates to time spent mucking around… BUT once working this product is excellent and I always have strong telemetry signal.
Ground Control Station (GCS)
The Mission Planner software which runs on the ground station laptop allowing you to program the UAV and monitor it in flight is very good – especially the Auto Waypoint Survey Grid feature. This allows you to draw the area you want to on the map. Simply load in a photograph from the camera you will be using and the target elevation. From this information is draws a flight path with your desired overlap.
Sensor & Image Processing
Canon S100 is my sensor of choice as it is a great balance of quality, price functionality and size. I started with a Canon D10 but many of the photos came out under exposed. The S100 has a larger sensor and inbuilt GPS so it is a better choice for aerial mapping. The downside to the S100 is that the lens protrudes from the camera which exposes it to damage in a rough landing.
With UAV aerial mapping you need a way for the camera to trigger every few seconds on its own. With a Canon camera it is easy using the Canon Hack Development Kit (CHDK). This updates the camera firmware, allowing you to use intervalometer scripts to trigger camera every few seconds. CHDK also offers what seems like unlimited settings for the camera. It seems difficult to find a complete set of settings to use with CHDK, but for my next flight I will try using the DroneMapper documentation to setup CHDK.
In my last flight approximately 30% of my photos came out blurry. I discarded the worst of the photos but still had to use some poor quality photos to ensure the map had no blank spots. This is probably due to a combination of camera settings, camera mount and propeller slightly out of balance.
Using desktop software trial of Agisoft Photoscan I was able to product a 40ha orthomosaic. The application works surprisingly well considering all images are taken at slightly different angle and are only provided with one GPS point for each photo. It is a very computer intense process and if I was to do a significant amount of processing would need to upgrade my computer. Alternatively I could use DroneMapper, but my dataset did not meet their requirements because I had to cull some images. I hope to try DroneMapper next time.
I took my data a step further and set up a web server to host the data as tiles. You can check it out here. How to store and share data collected by UAVs is something I have been thinking about. An orthomosaic for a single paddock can be several gigabytes and take a powerful computer to view in its raw form. The web seems like a good way to display data having a server store the data and only send bits of the image that the end user requests as they zoom in and move around.
This is my second flying UAV. My first was a Finwing Penguin as well. I spent a couple days putting my plane together and all the components. It is a nervous time flying your brand new plane for the first time. The first time out my plane few OK in manual mode but since I am a very ordinary pilot I like to use assisted flying modes. I changed to Fly By Wire mode and due to a APM setting (that I had to set) the autopilot had reversed the elevator sending it crashing into the ground. This snapped the plane in half and bent up the fuselage. Thankfully this durable foam returns to shape when you put it in boiling water and the pieces can be glued back together, reinforced with carbon fiber and fiberglass tape. Now I follow the suggested checks in the APM:Plane instructions more closely. I’ve had no crashes since but have landed in mud which can be painful to clean out of the landing gear.
Putting together this UAV I have learned how all the components of a UAV fit together, challenges faced by the commercial suppliers, and a better understanding of the enormous potential on offer. I think the biggest challenge is not the UAV platform itself but collecting high quality consistent data that can be quickly processed and given a useful, profitable application. The setup I have discussed here not including laptop or the countless hours of time comes to about AU$1200. Obviously for mapping large areas on a consistent basis, a commercial UAV would be preferred or even essential.
After an eye-opening Nuffield global focus program through South Africa, Kenya, Russia, Czech Republic, Poland, Germany and USA I am in the final stages of planning for my study topic travels.
My study question aims to investigate: the potential of unmanned aerial/ground vehicles (UAV/UGV) in the grains industry
Note: I will keep an eye open on UGVs but the topic is too big if I try and focus on both! Therefore, the primary focus will be UAVs.
Investigate the perceived benefits of UAVs
Understand the individual components that contribute to a UAV solution
Consider strengths, weaknesses and standout features of different UAV manufacturers
Evaluate end user experience so far with UAV technology
Record current development and future use for the technology
Beneath these over arching objectives include many of the FAQs many people have such as:
How do we handle the data?
Which sensor is the best?
Will UAVs actually help farmers make better decisions or just generate more data?
What about the rules and regulations?
Can a farmer own his own UAV or is it a specialist occupation?
To achieve these objectives I will be traveling the the USA and Canada in July-August 2014. During my travels I will attend three conferences, meet with at length with three UAV companies, discuss end user experience with farmers and advisers, and lots more…
As per Nuffield requirements I will be producing a comprehensive report on my findings which I hope to have completed by September 2014. Once edited and approved it will be available at the Nuffield International Reporting page.
Subscribe to agmapsonline.com for updates (See right hand side of this page) and follow me on Twitter @BenBoughton1.
#SocialWeatherFeed allows anyone to create a Twitter account (or use your own) to give a weather update for a selected weather station in Australia. It uses BoM weather data to give text updates every 3 hours and a 3 day charted history every morning.
It has taken me a while to figure out the best way to implement it and I am not sure if I have found it – but this is it for the moment.
Here is a list of the active Twitter accounts using #SocialWeatherFeed:
Technology in farming is constantly evolving. Collecting accurate, reliable georeferenced (location in terms of GPS coordinates) data is essential to capitalise on technologies such as variable rate application of chemicals and fertiliser and aid in crop monitoring at a level once not imagined. Some current forms of collecting georeferenced paddock data include:
Combine harvester – yield maps (crop yield as harvester works through paddock)
Satellite imagery – colour and near infrared (NIR) bands to produce natural images & vegetation indices such as Normalised Difference Vegetation Index (NDVI)
Aerial imagery – similar to satellite but offers higher resolution at higher price & some other sensor options
Tractor – Greenseeker (plant biomass), digital elevation model (DEM) collected from high accuracy GPS
Utility vehicles e.g. Soil sampling pH & nutrition, electromagnetic conductivity, Greenseeker, DEM
Handheld with GPS – Greenseeker, soil sampling
Stationary – moisture probe, weather station
Unmanned Aerial Vehicles (UAVs) are emerging as a cost effective way to collect data with many advantages over the traditional forms listed above. UAVs are as the name suggests an unmanned vehicle which flies over the paddock to collect data. These machines are generally compact, can be cheap, mechanically simple, fly below cloud cover and are on there way to being easy to operate with advanced autopilot systems.
Over the last 6 months I have begun researching civilian UAVs and their application in agriculture as part of my Nuffield Scholarship. Furthermore, I have been testing a budget UAV platform which I will discuss in a later post. The aim of this post is to aggregate key information and ideas on the topic into one space. It is by no means comprehensive – more of a beginning. Note that I am not a pilot or lawyer. This article is general in nature and does not give permission to fly or legal advice. Lets start with a sky-high view.
The Agricultural UAV Solution
It is important to consider all aspects pertaining to the agricultural UAV (aUAV) Solution which I define as a robust, timely, cost effective way to collect usable data to improve yields and overall profitability in sustainable farming systems. Consider the following formula:
All components of the formula need to be working well and working together for the product to be successful technology. Now enough of inventing acronyms and formulas that will inevitably change, it’s time to flesh out the components of the aUAV Solution.
There are two main platforms available: fixed wing and multi-rotor. A fixed wing platform has the advantage of covering large areas efficiently, whereas a multirotor shines in being able to remain very stable in challenging conditions with large payloads.
Due to the scale of broadacre grain growing in Australia, my interest lies predominately with the fixed wing platform type, as paddocks often exceed 250ha (~620ac). ConservationDrones has an excellent list of budget fixed wing platforms they have used as an example.
Global Positioning Systems (GPS) are the backbone of most spatial technologies. GPS on the UAV tells the autopilot where it is at all times. In addition, GPS links the data collected to it’s spatial position (aka geo-referencing).
Many UAVs are equipped with a u-blox GPS receiver or similar which is compact and provides <5m horizontal accuracy. These systems are affordable and are accurate for most situations.
An exciting development is the Piksi by Swift Navigation, which is a low cost Real Time Kinetic (RTK) GPS receiver that promises to sell for around $1000 which is unheard of in the world of GPS. The Piksi offers centimetre level accuracy inside a compact design ideal for small UAVs. The improved accuracy will be invaluable for autonomous landings and improved accuracy of geo-referencing data.
We are seeing UAV autopilots improve very quickly with increased reliability, especially within the open source community. Autopilots are essential for being able to effortlessly fly over a whole area to collect the desired data. DIY Drones‘ APM:Plane is often the autopilot of choice for hobbyists and entry to mid level platforms. It uses the same hardware and similar software to the APM:Rover I built last year.
There are several other autopilots available, commercial and open source, that are worth checking out. Google it.
Usually the UAV is communicating with a ground control station (GCS) via radio link. GCS is usually just a laptop computer with software such as Mission Planner. Mission Planner is also used to set the flight paths for the UAV missions.
The most complex part of collecting good data is having the correct sensor. For plant biomass data, the most important spectral range is in the near infrared spectrum. The two most common options include Tetracam ADC Lite built specifically for UAVs or a digital camera modified to capture within this spectrum (MaxMax for example). The latter option is the most cost effective solution. Some preliminary studies show that some good results can be achieved.
Researchers are working hard improving sensors for UAVs. For example, TerraLuma, is a research group at the University of Tasmania. Projects of interest include high accuracy geo-referencing of imagery ‘on the fly’ and the use of a hyperspectral pushbroom scanner to collect data.
Public Lab (an open source community) is also working at modifying cameras similar to MaxMax but also on cheaper devices such as web cams. The recently achieved funding through a Kickstarter campaign. Maybe we will have another cost effective solution soon. See also Pi NoIR.
It is worth mentioning that it is very common for UAVs to have a GoPro camera (or similar) mounted to capture high definition video footage. This video footage is valuable for visually monitoring crops from the sky but is generally not processed to geo-referenced data. There is always exceptions such as shown in this video over a construction site where video footage is used to generate a 3D model.
Data Processing & Integration
Although collecting good data is the most challenging part, the most time consuming (and/or expensive) part can be processing it to a point where it can be integrated into precision agriculture systems. Generally the UAV will follow a lawn mower track collecting an image images at a defined interval with a generous defined overlap. The raw data will usually be images (up to several hundreds – think gigabytes) with a single GPS position and maybe a bearing per image. The challenge for the data processing is to stitch these images together to generate one homogenous data set. Every image is affected by the differing roll, yaw and pitch of the UAV as the image is captured. Some of the more common applications include:
Drone Mapper is a successful web based startup which effectively filled the affordable yet professional data processing gap
Microsoft ICE is a web based and free to use but only stitches images without offering geo-referencing or 3D modelling unlike the above mentioned applications
VisualSFM, CMVS, and CMPVMS – Flight Riot does the hard work explaining how to use this software to generate a 3D model from digital camera photos. This is probably one of the more complex processes but uses all free(ish) software.
Once a geo-referenced, homogenous, data set over a paddock is achieved it could possibly undergo further post processing to determine NDVI. This raster data may then, for example, be used to define zones for in crop variable rate fertiliser application.
As mentioned, some of the above software is able to create 3D models from 2D photographs. These 3D models could be used to create digital elevation models (DEM) which is valuable in farming for determining water movement.
Legal & Operation
In Australia, the Cival Aviation Safety Authority (CASA) rule the sky. CASA has rules governing the use of UAVs (which they call unmanned aerial systems or UAS) and is in the process of re-evaluating some regulations. See a summary of a recent speech from CASA here.
To operate a UAV/UAS commercially in Australia you need to have a certified operators certificate. A list of those certified is available here.
CASA have done well to have a system set up for UAS. The USA is lagging behind and is just now establishing rules and regulations are UAVs.
You can buy a calibrated, tested, ready to fly system built from budget readily available components and open source autopilot. For example Event38 and Flight Riot.
The third option is to go fully DIY. I have tried this using a Finwing Penguin fixed wing platform, APM:Plane autopilot, ordinary Canon digital camera as sensor. I am yet to process any images into geo-referenced datasets. I will post more about this soon. Here is an image from one of my first flights.
It has been a season of ups and downs for us. We had good rain over summer but a dry April and May led to marginal planting conditions. We planted dry and were blessed with some rain in May and June which led to good establishment. We had 23mm for July and just 6mm for August. Crops have been coming along well relying on stored moisture. The mild winter meant crops we madly flowering by late August.
On the 21st of August Moree Aero recorded -1.1°C followed by -0.5 and 0°C. The effect on the barley is marginal and it seems to be filling well. The chickpeas lost all their flowers but they have begun to reflower and are just now starting to set their first pods. Yield potential in the peas has been reduced but should hopefully still see a reasonable crop. The wheat has been most affected with stem frost leading to ring barking. Other tillers that are not ring barked have the head turning yellow and aborting seed set. Again other tillers seem to be OK and should produce seed. The extent of the damage is varied with higher areas less impacted. We will know more in the coming weeks and the yield map will tell the true extent.
We are now looking for a decent fall of rain the keep the crops going through to harvest. Below are a few pictures I took this morning.
It is not only the commercial world that has taken to this technology. There is a whole open source DIY community focused on building autonomous vehicles. Although not specifically aimed at agriculture, DIY Drones along with it’s partner 3D Robotics has developed hardware and software capable of controlling unmanned planes, …copters and rovers. The beauty of this is that the software is essentially free and the hardware is reasonably priced and readily available. Also, there is a community that is openly discussing and evolving the project so it is relatively easy to troubleshoot and learn a bit of what goes on behind the scenes.
A few months ago I took the plunge and ordered the hardware needed to build an autonomous vehicle. I designed and built a small rover that uses two 500W electric motors as it’s ‘powerhouse’. It works with tank style steering. It is essentially a large remote controlled car. The ArduPilot Mega (APM) sits between the remote control receiver and the motor controller (Sabertooth 2×60). The APM has an internal compass, external GPS and an external radio transmitter/receiver.
Using the Mission Planner software you can set waypoints that the rover follows. The rover’s GPS position, speed and other information can be monitored from within the Mission Planner software as the rover completes it’s mission.
I have put together a small video that shows the basic operation of the autonomous vehicle.
The question you are probably asking is ‘Why is this significant?’ or ‘Whats the point?’. The answer is that it is part of the next step in agriculture. Queensland University of Technology (QUT) is involved in similar research. Imagine a fleet of small autonomous vehicles sent to patrol the summer fallows to keep them weed free. For me, building a simple autonomous vehicle of my own shows that the technology exists, it is not that expensive and commercially, shouldn’t be to far away.
My next steps are to use sonar sensors to implement object avoidance. Further down the track I will think about mounting a camera or some other sort of sensor to collect data.