Challenges facing UAS in agriculture

After just finishing a post on some of the applications for UAS in agriculture, I thought I would share what I believe to be some of the biggest challenges the industry faces. As I continue my Nuffield Scholarship studies (thanks to GRDC) I am learning that unmanned aerial systems and the data produced is increasingly complex. In fact, many people will study a PhD just on one aspect of these systems alone (e.g. flight characteristics or remote sensing). Some of the more obvious challenges include:

1. Repeatability

If you were to go out and map a paddock at 11am and then again at 2pm, the resulting pixel values would be different. Even using the Normalised Difference Vegetation Index (NDVI) which by definition gives a normalised value, the data will be different. This is probably because the atmosphere and clouds do not seem to block/transmit/reflect wavelengths in parallel amounts.We see this effect in satellite imagery where the image is affected by cloud shadow. The table below is pixel value of Red and NIR from two points in the same barley paddock that has minimal variability.

Cloud shadow vs no shadow NDVI, Landsat 8, barley
Cloud shadow vs no shadow NDVI, Landsat 8, barley

Despite crop growth activity and biomass being very similar, these areas produce significantly different NDVI value due to the cloud shadow.

This can be applied to UAS imagery. The atmosphere, clouds and sun angle is constantly changing throughout the day affecting reflectance of all wavelengths differently. Therefore for data to be compared like to like it needs to be calibrated.

2. Calibration

This section follows on nicely from repeatability. If data is going to be used for more than just scouting then some sort of calibration will need to take place. As discussed in a previous blog post, calibration of NDVI could be potentially be achieved using an active ground device such as a GreenSeeker. There is also the new payloads that are true multispectral and have upward looking sensor to measure irradiance which could be used to calibrate data for true reflectance. Other forms of calibration/ground truthing include biomass cutting and weighing, tissue testing, the list goes on.

3. Algorithms & Indices

When it comes to remote sensing and vegetation, NDVI is the most famous index used, and for good reason, but it is not without its concerns. If you look on Wikipedia, the authors cover some of the issues with NDVI:

Users of NDVI have tended to estimate a large number of vegetation properties from the value of this index. Typical examples include the Leaf Area Index, biomass, chlorophyll concentration in leaves, plant productivity, fractional vegetation cover, accumulated rainfall, etc. Such relations are often derived by correlating space-derived NDVI values with ground-measured values of these variables. This approach raises further issues related to the spatial scale associated with the measurements, as satellite sensors always measure radiation quantities for areas substantially larger than those sampled by field instruments. Furthermore, it is of course illogical to claim that all these relations hold at once, because that would imply that all of these environmental properties would be directly and unequivocally related between themselves.

Thankfully we are not pigeon-holed to NDVI. Agribotix claim they get better results using the Difference Vegetation Index (DVI). Another example is Soil Adjusted Vegetation Index (SAVI).

4. Position

Without ground control points, the positional accuracy of data will be mediocre at best. Expect XY accuracy of a few meters and even more on the Z axis. GPS will record the position each frame is captured (+/- delay error), but the pitch, yaw and roll of the UAS which affects how the image is framed on the ground, is determined by an inertial measurement unit (IMU). The quality of the IMU will have a bearing on the positional accuracy if the processing software takes these variables into consideration. Expect to have to lay out minimum of 4 ground control points for high accuracy data.

UAS image processed with no GCP
UAS image processed with no GCP

5. Reliable data collection

As a consequence of the process involved in collecting data with a UAS there are several factors that contribute to the ability to reliably collect data. The process usually involves the UAS following a set lawnmower style track, conducting swaths up and back as it moves across the area of interest. As the vehicle is flying, it is capturing an image every few seconds depending on its speed. A forward and side overlap greater than 50% is required. Although processing software can handle images with some angle to the ground, extreme pitch and roll will affect the overall product.

In the data collection process if the UAS is hit by a gust of wind, it may put 2 or 3 images off target, while the autopilot makes adjustment. This can lead to a hole in the data set. This is not uncommon. Some UAS manufacturers allow you the rapidly transfer the raw data off the vehicle to a laptop in the paddock to check data coverage and quality in field (e.g. Precision Hawk).

So remember when someone claims their machine can fly in high winds, it is probably true, but the quality of the data being collected may not be of much use.

Sample flight path and image footprints
Sample flight path and image footprints

6. Data processing

Reliably collecting the data is just the first step in the process. Once all these images have been collect they need to be stitched together. It is amazing that stitching 300+ 12MP images together which are all taken at slightly different angles to the ground is even possible. Even more so that 3D surface models can be constructed from these 2D images. Given the complexity of this task it takes large amounts of computer power and time (think several hours for 100ha). For this reason there are several cloud based platforms which can offer this service (e.g. DroneMapper and PrecisionMapper). Processing on your own desktop computer versus online services do have both their pros and cons. A downside to the cloud services is the internet bandwidth required to first transfer the raw data to the server and then retrieve it once it has been processed. A downside to the desktop solution is the upfront cost of hardware and software and the required skill set may not be available in house.

7. Storage & sharing

Once the data is processed it needs to be stored somewhere and somehow distributed. Often one scene can be more than a gigabyte. If the processed data is not cut into tiles it can require a powerful machine just to view it. This is where online solutions come into play. Again the same issues exist as above around bandwidth requirements. At some point if the data is going to be used for more than just looking at it will most likely need to be transferred onto a local machine. This is an online map of one of the first areas I mapped with my DIY Finwing Penguin UAS. If for some reason the data is to be printed, it needs to be formatted as such which takes time and software.

8. System integration

Integrating data generated from UAS into existing precision agriculture software should be possible but not likely in its highest available spatial resolution. Software such as Farmworks and SST were not designed for such intense data sets which are able to be generated from UAS sensors. Resampling data to 0.5m resolution may be required.

9. Safety and legal

A whole other post could be written on this but basically in Australia we have the Civil Aviation Safety Authority (CASA). They require anyone who wants to fly UAS (they call them remotely piloted aircraft or RPA) commercially to have an Operators Certificate and Controllers Certificate. This, among other things requires a theory component equivalent to a private pilots license, comprehensive manuals written, and a stringent approvals process to ensure your vehicle is fit for service and your piloting skills are sufficient. CASA is at the moment reviewing this process and should hopefully have a revision out by the end of the year. This process seems strict, but it important to keep the UAS industry safe and professional.

Processing harvest yield data to grid / raster

The process to convert point data to raster (or grid) for can differ dramatically depending on who you ask and the purpose of the conversion. Generally data is processed for two main functions. The first being ‘stacking’ or ‘layering’ to later develop prescription maps or other further processing. The second is improved visual representation of the data. My process aims to satisfy both with minimal processing time. My method does contain some compromises; since I use a coarse resolution it is not as pleasing on the eye as some other methods. In addition, my data smoothing technique is quite broad which means you will loose some detail in the grid.

At this point in time I follow these steps:

  1. Clean up point data either manually manually or with a filter to remove any obvious errors such as where header turns in paddock. EDIT: I am currently trying Yield Editor for this step.
  2. Produce a grid over the top of the point data. Any cells that share a points average the point values.
  3. Gaps in the grid are filled in.
  4. Gaussian Filter is used to ‘smooth’ the data.

In QGIS this is the process:

Note: You will need to have SEXTANTE and additional toolboxes setup to follow these steps. Instructions are available here.

  1. Load paddock boundary (shape file polygon)
  2. Load yield data points (You will need these in shape file format – use export function in SMS or FOViewer if your yield monitor does not produce ESRI shape file)
  3. SEXTANTE > SAGA Toolbox > Shapes – Points > Points Filter
    1. Radius: 100
    2. Minimum Number of Points: 25
    3. Maximum Number of Points: 200
    4. Quadrants: No
    5. Filter Criterion: Remove Below Percentile
    6. Percentile: 15-20
  4. SEXTANTE > SAGA Toolbox > Shapes – Points > Points Filter (use output from step 3)
    1. Radius: 100
    2. Minimum Number of Points: 25
    3. Maximum Number of Points: 200
    4. Quadrants: No
    5. Filter Criterion: Remove Above Percentile
    6. Percentile: 90
  5. SEXTANTE > SAGA Toolbox > Grid – Gridding > Shapes to Grid
    1. Preferred Target Grid Type: Floating Point
    2. Cell Size: 15
    3. Method for Multiple Values: Mean
  6. SEXTANTE > SAGA Toolbox > Grid – Tools > Close Gaps
  7. SEXTANTE > SAGA Toolbox > Shapes – Grid > Clip Grid with Polygon
  8. SEXTANTE > SAGA Toolbox > Gaussian Filter
    1. Standard Deviation: 3
    2. Search Mode: Circle
    3. Search Radius: 50

Make sure you do a visual comparison with original point data to ensure the final grid is a good representation of the original data.

Visually check to see that processed grid file is a fair representation of the original point data.
Visually check to see that processed grid file is a fair representation of the original point data. Note I was lazy in this photo and used un-filtered yield data.

To generate useful color maps to represent the grid data try http://colorbrewer2.org/. One day I will explain this process, but it is not too hard to try it yourself.

Now when we get clever we can use the SEXTANTE Modeler to automate this process. More on this later! –Edit: Here is the modeler info.