Using MultiSpec to composite Landsat images

As most GIS users will know the Landsat archive of satellite images is available for free download from USGS. This is a valuable resourse of historic and recent course resolution images covering most of the earth. I find the USGS Global Visualization Viewer the best way to sort through and download images. You need to add images to the basket. Often you can download them directly from the basket, but sometimes they need processing time which can take a day or two. Note that images from L7 2003 onwards have lines of no data caused by a faulty sensor. Up until recently L5 was still working producing quality images but is now out of service. Shortly we will have Landsat 8 – it has been launched and more free imagery is not far off!

Once the image file is downloaded you will have a compressed .tar.gz file which can extracted using built in Windows functions or 7Zip. Once extracted you have a TIFF (.tif) file for each of the bands of the image. For L7, that is 9 TIFF files about 60mb each except band 8 which is higher resolution band for pan-sharpening usually 200mb+.

The next step is to composite these bands to make a coloured image to load into your GIS. If you are a Windows or Mac user I suggest you try out MultiSpec. Download and extract to a folder of you choice. I keep it at C:MultiSpec for simplicity.

A comprehensive guide on how to composite images has been written up by the creators of MultiSpec and is available here. The process I give below is a simplified explanation based on the MultiSpec tutorial.

1. Launch MultiSpec.

2. Go to File > Open and select all bands you want to composite into one image.

Open multiple files
Open multiple files

3. For the ‘Set Display Specifications’ dialogue press OK (Try setting Bands to Red: 5, Green: 4 and Blue: 1. This is not critical for this exercise but is useful to visually inspect the image inside MultiSpec).

4. Press Ctrl + R to open ‘Set Image File Format Change Specifications’

Set Image File Format Change Sepecifications
Set Image File Format Change Specifications

5. You can select Channels and reorder by choosing ‘Subset…’ (I usually include bands 1-5) and also select file format in the ‘Header’ drop box (I always stick with GeoTIFF format). Press OK and choose where you want your composite image to reside and give it a useful name (do not forget to add the .tif extension).

6. Load up your favorite GIS desktop software, QGIS for me, and add your new TIFF file.

7. Edit layer properties and try a few different band combinations and contrast enhancements.
For good visibility of vegetation try Red: Band 5, Green: Band 4 and Blue: Band 1.
For a more natural look try Red: Band 3, Green: Band 2 and Blue: Band 1.

Try different band combinations and contrast enhancement
Try different band combinations and contrast enhancement

I hope this helps make Landsat imagery more accessible to everyone.

Publishing GIS data on the web: Lizmap

Lizmap is Open Source software that has the capacity to display GIS data on the web with minimal effort once it has been configured correctly. This software sets itself apart by making use of the QGIS Server in a unique and user friendly way. In essence this allows the user to configure a map in QGIS desktop software and simply run a basic plugin to publish their data on the web with a Google Maps satellite or street layer as a base map.

I have setup Lizmap software on my server to demo its capabilities. I left the original demo map of transport in Montpellier, France and added my own with data from Lindon – one of our properties. I hope to demo its mobile device capabilities soon.

Check out my Lizmap demo here

 

Lizmap Web Client displaying Lindon property map.
Lizmap Web Client displaying Lindon property map.

More information about Lizmap:
Lizmap Web Client Documentation
Lizmap QGIS Plugin Documentation

http://rcm-na.amazon-adsystem.com/e/cm?t=agma-20&o=1&p=8&l=as1&asins=0596008651&ref=tf_til&fc1=000000&IS2=1&lt1=_blank&m=amazon&lc1=0000FF&bc1=FFFFFF&bg1=FFFFFF&f=ifr

QGIS SEXTANTE Modeler: Automate yield data processing

This is a followup post to Processing harvest yield data to grid / raster where I outlined my yield data processing steps. This post shows how I use the Modeler in SEXTANTE to automate this process utilising SAGA algorithms.

The idea of the modeler for me is to save time and keep consistent processes. In the screen shot below you can see that the model requires the yield data point file (where the Dry Yield table column needs selecting), paddock boundary and the grid and filtered points output file location. The model takes care of all the processing steps.

Select yield data point file to be processed and choose output location. The modeler does the rest!
Select yield data point file to be processed and choose output location. The modeler does the rest!

While constructing the model the only tricky part was making sure I had a separate ‘Extent’ field to input into the ‘Shapes to Grid’ algorithm. Even though I always leave this blank, it is required for the algorithm to realise the extent is just the minimum covering extent of the input shape file.

SEXTANTE model editing screen displaying input fields and algorithms involved in the yield data processing
SEXTANTE model editing screen displaying input fields and algorithms involved in the yield data processing
Yield Map: L3 Chickpeas 2012
Yield Map: L3 Chickpeas 2012

A few years ago I used to use model builder in ArcGIS and really missed it when I no longer had access to ESRI software. The SEXTANTE Modeler fills that gap and allows a combination of algorithms from several different toolboxes.

Some more advanced applications of Sextante Modeler are available here.

Processing harvest yield data to grid / raster

The process to convert point data to raster (or grid) for can differ dramatically depending on who you ask and the purpose of the conversion. Generally data is processed for two main functions. The first being ‘stacking’ or ‘layering’ to later develop prescription maps or other further processing. The second is improved visual representation of the data. My process aims to satisfy both with minimal processing time. My method does contain some compromises; since I use a coarse resolution it is not as pleasing on the eye as some other methods. In addition, my data smoothing technique is quite broad which means you will loose some detail in the grid.

At this point in time I follow these steps:

  1. Clean up point data either manually manually or with a filter to remove any obvious errors such as where header turns in paddock. EDIT: I am currently trying Yield Editor for this step.
  2. Produce a grid over the top of the point data. Any cells that share a points average the point values.
  3. Gaps in the grid are filled in.
  4. Gaussian Filter is used to ‘smooth’ the data.

In QGIS this is the process:

Note: You will need to have SEXTANTE and additional toolboxes setup to follow these steps. Instructions are available here.

  1. Load paddock boundary (shape file polygon)
  2. Load yield data points (You will need these in shape file format – use export function in SMS or FOViewer if your yield monitor does not produce ESRI shape file)
  3. SEXTANTE > SAGA Toolbox > Shapes – Points > Points Filter
    1. Radius: 100
    2. Minimum Number of Points: 25
    3. Maximum Number of Points: 200
    4. Quadrants: No
    5. Filter Criterion: Remove Below Percentile
    6. Percentile: 15-20
  4. SEXTANTE > SAGA Toolbox > Shapes – Points > Points Filter (use output from step 3)
    1. Radius: 100
    2. Minimum Number of Points: 25
    3. Maximum Number of Points: 200
    4. Quadrants: No
    5. Filter Criterion: Remove Above Percentile
    6. Percentile: 90
  5. SEXTANTE > SAGA Toolbox > Grid – Gridding > Shapes to Grid
    1. Preferred Target Grid Type: Floating Point
    2. Cell Size: 15
    3. Method for Multiple Values: Mean
  6. SEXTANTE > SAGA Toolbox > Grid – Tools > Close Gaps
  7. SEXTANTE > SAGA Toolbox > Shapes – Grid > Clip Grid with Polygon
  8. SEXTANTE > SAGA Toolbox > Gaussian Filter
    1. Standard Deviation: 3
    2. Search Mode: Circle
    3. Search Radius: 50

Make sure you do a visual comparison with original point data to ensure the final grid is a good representation of the original data.

Visually check to see that processed grid file is a fair representation of the original point data.
Visually check to see that processed grid file is a fair representation of the original point data. Note I was lazy in this photo and used un-filtered yield data.

To generate useful color maps to represent the grid data try http://colorbrewer2.org/. One day I will explain this process, but it is not too hard to try it yourself.

Now when we get clever we can use the SEXTANTE Modeler to automate this process. More on this later! –Edit: Here is the modeler info.

 

Setting up QGIS for Precision Agriculture GIS: Free Software

QGIS rivals ArcGIS as desktop GIS software especially within Precision Agriculture. The difference is that QGIS is Open Source and therefore free to use for personal and commercial use. Open Source has many other advantages. If you are familiar with ArcGIS you should give QGIS a go.

If you run Windows I recommend installing using the OSGeo4W installer. Run the ‘Desktop Express Install’.

‘Out of the box’ QGIS is very capable. But it is not until you install a few powerful plugins that it’s real potential is revealed. So far the plugins I use on a daily basis are: SEXTANTE and Table Manager. SEXTANTE is not much good to me without SAGA. Together these make available a comprehensive list of common Vector and Raster GIS algorithms. Optionally install TauDEM and Orfeo (I have installed these but not yet used them).

Setting up SEXTANTE in QGIS 1.8 (Windows 7 & 8)

Install Sextante Plugin (In QGIS: Plugins > Fetch Python Plugins).

So SEXTANTE has access to the SAGA algorithms it needs to be downloaded and installed: SAGA.

Configure SAGA in SEXTANTE (In QGIS: Analysis > SEXTANTE options > SAGA), insert SAGA folder and check Activate box.

Similar to SAGA, download and install TauDEM 5.0.6 & MPICH2 (make sure you follow install instructions on the download page).

Configure TauDEM in SEXTANTE (In QGIS: Analysis > SEXTANTE options > TauDEM), insert MPICH2 bin directory, TauDEM command line tools folder and check Activate box.

Install Table Manager

Plugins > Fetch Python Plugins

Search for Table Manager and click Install.

With these two plugins, especially SEXTANTE, QGIS becomes extremely capable.

SEXTANTE with SAGA in QGIS 1.8

http://rcm-na.amazon-adsystem.com/e/cm?lt1=_blank&bc1=FFFFFF&IS2=1&bg1=FFFFFF&fc1=000000&lc1=0000FF&t=agma-20&o=1&p=8&l=as1&m=amazon&f=ifr&ref=tf_til&asins=178216748X

My first QGIS plugin: BigButtons

I’ve recently started playing around with QGIS plugins on my Windows 8 (and 7) machines. After working through the initial process which I discussed here I had a go at making my own (very) simple plugin.

The purpose of the plugin was to allow easy access to zoom and pan functions for a touch screen laptop such as my Panasonic Toughbook.

I use the BigButton plugin in conjunction with QGIS Tracker and OpenLayers. I mounted a GPS on a quad bike with my Toughbook and went out to record a trip with live Google Maps underneath. This worked to a point.

Warning: Using these three plugins together is riddled with bugs. Just the ones I notice from 10 minutes use include:

  • BigButtons will not work properly while QGIS Tracker is Tracking GPS Location as it is programmed to keep the marker centered. Once tracking is stopped BigButtons functions as normal.
  • As the marker direction from QGIS Tracker changes in reference to the GPS bearing the Google Satellite from OpenLayers plugin will go white underneath.
  • BigButtons ‘Cycle Layer’ currently only works if you have all layers in a single group – and even then can do some strange things – needs work!

Download QGIS Plugin tested on Windows with QGIS 1.8: BigButtons-Experimental

Photo of my quick and dirty GPS setup on quad bike:

Creating Python QGIS Plugins in Windows 8

After being fresh out of Udacity CS101 – Computer Science course I wanted to put my new Python skills to use. QGIS supports plugins programmed in Python so I thought I would give that a go. What I realised is that there is a missing link between an introduction to programming and how to apply it! After several hours I was able to complete a simple plugin and get it working, with a GUI – all in Windows 8.

Here are the steps I followed:

1. Install QGIS 1.8 using OSGeo4W. Select ‘Express Desktop Install’. I installed all packages even though we won’t be using them all. It is worth having.

2. Download and Install Monkey Studio IDE.

Optional: Download QGIS APIs for auto-complete. You can add these to Monkey Studio by going to Edit > Settings. In ‘Source APIs’ select Python as language and add all the .api files. I sources these files here but stored them on my server in case the drop box goes missing.

3. The first Plugin I made was called Zoom to Point. I used instructions from a book excerpt available here. Chapter 15 – GIS Scripting, starting at page 270.

I was able to follow the instructions with the following additional tips:

  • Windows 8 QGIS plugin directory is located at

C:Users[username].qgisPythonPlugins

  • QT Designer is built into Monkey Studio. When you need to edit ui_zoomtopoint.ui launch Monkey Studio, then open this file to edit it.
  • Run the command line commands in the OSGeo4W console which should be on your desktop from Step 1. Navigate to your plugin directory e.g.

cd C:Users[username].qgisPythonPluginsZoomToPoint

pyrcc4 -o resources.py resources.qrc

pyuic4 -o ui_zoomtopoint.py ui_zoomtopoint.ui

  • On page 284-285 there is code which you will need to copy over to your zoomtopoint.py file. For me lines 41 and 42 produced an error. I was able to fix this by replacing them with:
41     x = dlg.ui.xCoord.toPlainText()
42     y = dlg.ui.yCoord.toPlainText()
  • In the file zoomtopoint.py (around line 27) I changed ‘import resources_rc’ to ‘import resources’. Otherwise I get an error in QGIS.
  • Every time you make a change to your plugin you need to restart QGIS. There is a plugin called Plugin Reloader that speeds this up allowing you to reload plugin with restarting QGIS.
  • Be very careful manually copying code. Most of my errors came from incorrectly copying code!

I hope that saves some others a few hours starting out. I have just scratched the surface and am looking forward to building some handy QGIS plugins as time allows.

Some more tips as I go:

  • For some reason if I create my Class name (ie when creating plugin template in Plugin Builder) in all lower case it does not build properly with pyuic4. If you have this same problem try using CamelCase and see if that makes a difference – it did for me.