The objective of this work was to autonomously detect manually verified features (plumes) in images under onboard conditions. Success enables these methods to be applied to future outer solar system missions and facilitates onboard autonomous detection of transient events and features regardless of viewing and illumination effects, electronic interference, and physical image arti facts. Autonomous detection allows the maximization of the spacecrafts memory capacity and downlink bandwidth byprioritizing data of utmost scientific significance.

What are plumes? The indication of active transient volcanic eruptive events on multi-
ple planetary bodies (Figure 1). These include large scale explosive and effusive eruptions on Jupiters satellite, Io, geyser-like ejections on Saturns satellite, Enceladus, and
Neptunes satellite, Triton, and outgassing of comet nuclei. These phenomena provide key constraints for subsurface processes, interior dynamics, surface-interior interactions,
and models of planetary composition.

The methods:  After experimental evaluation of the efficacy of various autonomous supervised classification techniques. We found that Scale Invariant Feature Transform(SIFT) + K Nearest Neighbour(KNN) gave the best results. (Figure 2). The statistics of the detection of different image sets are in Table 1.

Fig1:Sample images of planets and plumes. Plumes are denoted with squares. Some plumes are enlarged and enhanced for clarity. All of the planets are Io

 

Fig2.  Results of SIFT+KNN. Detected plumes are denoted with blue arrows.Some plumes are enlarged and enhanced for clarity

 

Table 1. Statistics of plume detection for different image sets

 

1. Objective
2. Problem
3. Experiments on B4 dataset
4. Experiments on El Mayor Cucupah dataset
5. Finding the right window size
6. Detecting regions containing the fault
7. Further directions
8. Related publications
9. Source code and Test dataset

Objective

The objective of this project is to determine 3-dimensional, local ground displacements caused by an earth-quake. The technique requires pre- and post-earthquake point cloud datasets, such as those collected using air-borne Light Detection and Ranging (Lidar) or SFM generated point clouds from aerial images. A typical aerial point cloud data of a terrain is shown in the figure below.

We used the publicly available B4 Lidar and El Mayor Cucupah datasets. B4 dataset covers the San Andreas Fault System of Central and Southern California. El Mayor Cucupah dataset gives the pre and post earthquake data of the April 4 2010 earthquake that was felt throughout Southern California, Arizona, Nevada, and Baja California Norte, Mexico.
<< Table Of Contents >>

Problem

This problem is formulated as a point cloud registration problem in which the full point cloud is divided into smaller windows, for which the local displacement that best restores the post earthquake point cloud onto its pre-earthquake equivalent must be found. We used the ICP algorithm to register point clouds.

Initially we introduced a synthetic earthquake on the B4 lidar dataset wherein displacement vectors were introduced on different regions of the terrain and the resulting displacement vectors from ICP were compared with the input displacement vectors. Later we proceeded to test this approach on the El Mayor Cucupah dataset (a real earthquake dataset) and some observations on the resulting displacements were made.
<< Table Of Contents >>

Experiments on the B4 dataset (Synthetic Earthquake)

Two experiments were conducted with different displacement fields and the results are shown in the figures below.  Black arrows are horizontal displacements and coloured circles denote vertical displacements. The artificial fault introduced is shown as a dotted line, and points on the right side of the fault were given a south east transformation and the points to the left were given a north west transformation. A vertical z displacement was also introduced.

<< Table Of Contents >>

Experiments on the El Mayor Cucupah dataset (Real Earthquake dataset)

We considered a 2 km x 2 km region in this dataset and presented initial results for the same. The figure below shows the results after registration wherein. The differences in these displacements are clear on either side of the fault (shaded in green).

<< Table Of Contents >>

Finding the right window size

In the above experiments we assume a fixed window size for registration. We also  investigated an algorithmic way of finding the right window size for registration. We began by choosing an arbitrary window size in the source cloud (e.g. 200 m × 200 m). For each of these windows, the corresponding window in the target data is identified based on x and y coordinates. Next, we computed the rigid body transformation between the source and target windows using the ICP algorithm. This window is split into four smaller windows of equal size and the rigid body transformation is computed on every child window. The transformation is validated after each split and the associated error computed. Based on the differences in error after consecutive splits, we decide whether further splitting is necessary. We verified experimentally that we cannot have small errors for very small window sizes (∼10 m) given the point cloud densities and input displacements. An analysis of this error indicates when to stop splitting yielding the right window size. A detailed description of this approach can be seen in our paper.
<< Table Of Contents >>

Detecting regions containing the fault

An interesting question to answer in earthquake datasets is ‘Can we find the regions containing a fault?’. Our windowing approach to change detection enabled us to examine this and we came up with an information theoretic approach to classify the windows containing a fault. The following image shows the dataset being split up into multiple windows with the thick black line showing the fault. The results of the information theoretic approach extracting windows containing the fault is shown in the right. A detailed description of this approach can be seen in our paper.

These results were submitted to the International Symposium of Experimental Robotics, 2012 and is under review.
<< Table Of Contents >>

Further directions

We are now running experiments on the entire dataset covering 850 sq km. One more direction we are looking at is to use SFM generated point clouds instead of Lidar. This offers significant advantages in terms of the cost involved.

The following image shows the point cloud generated using aerial image sequences over a  strip of land obtained from a helicopter. This strip of land coincides with a part of the data in the B4 lidar dataset.

 

The B4 lidar and SFM pointclouds have different co-ordinate systems and before we do change detection, the first task is to align the two point clouds globally. We are considering this problem as a template fitting problem. The best fit for the SFM point cloud in the lidar point cloud is computed and change detection is done after the template fit. We are still working on the results.
<< Table Of Contents >>

Related publications

• Change detection using Airborne Systems : Applications to Earthquakes   Aravindhan K Krishnan, Edwin Nissen, Srikanth Saripalli and Ramon Arrowsmith. International Symposium on Experimental Robotics (ISER) 2012

• Three-dimensional coseismic surface displacements and rotations from pre- and post-earthquake Lidar point clouds
  Edwin Nissen, Aravindhan K Krishnan, Ramon Arrowsmith and Srikanth Saripalli
  Geophysical Research Letters, 2012

Source code and Test dataset

The source code can be downloaded here. The test dataset can be downloaded here.

Large scale 3D mapping using LIDAR
In this project, we used the Iterative Closest Point (ICP) algorithm to build a mapping pipeline that can construct 3D maps from LIDAR data. This pipeline has been tested on data from Velodyne and RIEGL LIDARs. The Velodyne LIDAR (VLP16) was mounted on a car and was driven around, while GPS was NOT used. Final trajectories and maps were constructed from this LIDAR data.

 

Vision based GPS-denied Object Tracking and Following for UAV
In this project, we present a vision based control strategy for tracking and following objects using an Unmanned Aerial Vehicle. We have developed an image based visual servoing method that uses only a forward looking camera for tracking and following user-specified objects from a multi-rotor UAV continuously while maintaining a fixed distance from the object and also simultaneously keeping it in the center of the image plane; without any dependence on GPS systems. The algorithm is validated using a Parrot AR Drone 2.0 in outdoor conditions while tracking and following people and other static or fast moving objects, while showing the robustness of the proposed system against perturbations and illumination changes and occlusions. Please visit the project page for more details.

 

Ars Robotica
Ars Robotica is a collaboration between Unmanned Systems Lab and the School of Film, Dance and Theatre at Arizona State University. Using the Rethink Robotics Baxter as a test platform, Ars Robotica aims to investigate the possibility of defining and achieving a human quality of movement through robots, and validating it through the idea of viewing a robot as a performer in theater. Training data is obtained through various modes of sensing ranging from simple devices such as a Microsoft Kinect; to high speed precise tracking setups such as a 12 camera Optitrack system; which is then used for defining a vocabulary of human motion primitives, thus helping create a framework for autonomous interpretation and expression of human-like motion through Baxter. Please visit the project page for more details.

Terrain Mapping using UAVs
Three dimensional mapping is an extremely important aspect of geological surveying. Current methods of doing this, however, often poses pragmatic challenges. We introduce a technique for performing terrain mapping using unmanned aerial vehicles (UAVs) and standard digital cameras. Using a photogrammetric process called structure from motion (SFM), aerial images can be used to infer 3-dimensional data. Please visit the project page for more details.

 

Autonomous Kite Plane for Aerial Surveillance
The development of an autonomous fixed-wing motorized kite plane, Autokite, offers a unique approach to aerial photography in the field.  The inexpensive and lightweight nature of the Autokite makes it ideal for deployment in environments that are remote and or extreme.

 
 
 

Autonomous ship board landing of a VTOL UAV
The autonomous landing of Vertical Take Off and Landing (VTOL) Unmanned Aerial Vehicles (UAVs) is a very important capability for autonomous systems. Autonomously landing on a ship deck platform continues to be studied and has only recently been solved for very favorable weather conditions.
Our challenge is to provide the UAV with the capabilities of autonomously landing on ship deck platforms in extreme weather conditions.

 

EGGS (Exploration Geology & Geophysics Sensors) 
The EGGS Project, Exploration Geology & Geophysics Sensors, aims to develop a diverse set of robust self-righting multi-purpose data-collection platforms capable of assisting scientists/explorers in the field on Earth or through remote deployments to near-by asteroids. With an integrated camera, microscope, accelerometer, magnetometer, and configurations for adding other instruments, EGGS are a low-cost 3D printable option for students, researchers, and enthusiasts who want to learn more about an environment remotely.

Using UAVs to Assesses Signal Strength Patterns for Radio Telescopes
In this work the design of flight hardware for detecting the signal strength field pattern of an array of Radio Telescopes is considered.  Utilizing the ultra-stable and robust aerial platform offered by a multi-rotor craft makes this task possible.

 
 
 

Change Detection using airborne Lidar
In the course of this project, we worked with geologists on developing algorithms for finding the local displacements on topographies during earth quakes. Algorithms use the Digital Elevation Models of earthquake sites (before and after the earthquake) obtained from Lidar scanners mounted on aerial vehicles. Please visit the project page for more details.

 

NIR Camera
The objective of the NIR project involved constructing an equivalent MER PANCAM from readily available commercial parts for use of science and study of Earth’s atmosphere and geological features. Please visit the project page for more details.

 
 
 

Path Planning for Ground Vehicles
The objective of this project was to study and devise new means for motion planning for ground vehicles, using a rover named Raven as the prototype vehicle. More specifically, we try to determine smooth paths for Raven to follow, as it traverses waypoints; such paths have wide use in applications, for instance  in following an astronaut as (s)he walks along a random path. Please visit the Project Page for more details.

 
 

Plume Detection
The objective of this work was to autonomously detect manually verified features (plumes) in images under onboard conditions. Success enables these methods to be applied to future outer solar system missions and facilitates onboard autonomous detection of transient events and features regardless of viewing and illumination effects, electronic interference, and physical image artifacts. Autonomous detection allows the maximization of the spacecrafts memory capacity and downlink bandwidth by prioritizing data of utmost scientific significance. Please visit the project page for more details.

 

R.A.V.E.N.
RAVEN (Robotic Assist Vehicle for Extraterrestrial Navigation) was designed for the 2010 Revolutionary Aerospace Systems Concepts Academic Linkage (RASC-AL) contest.  Please visit the project page for more details.

 
 

Road Detection from UAV Aerial Imagery
Using aerial images taken from UAV photography to detect the presence of roads.  In this work, we developed variations of algorithms suitable for different types of roads and detection.  Please visit the project page for more details.

 
 
 
 

Autonomous Sampling
Autonomous Underwater Vehicles have proven themselves to be indispensable tools for mapping and sampling aquatic environments. However these sensing platforms can only travel as far as their stored energy capacities allow them to. Thus we have researched both offline and online adaptive sampling strategies that optimize both the estimation accuracy of the models derived from sampling and the energy consumption of the vehicle through.  Please visit the project page for more details.

The main aim of this project is to create a framework that can perform collaborative localization between groups of micro aerial vehicle (multirotor vehicles) using monocular cameras as the only sensors.

Especially in the context of UAV swarms, which is rapidly becoming a popular idea in robotics, the focus is usually on small platforms with limited sensory payload and computational capacity. In the context of such groups, having each vehicle run its own version of localization algorithms such as SLAM can be computationally intensive. Although monocular cameras are ubiquitous in current day MAV platforms, monocular cameras are unable to resolve scale on their own, which necessitates further sensor fusion. Given these challenges, the idea of collaboration can be desirable as it has the potential to reduce computation as well as improve localization accuracy: relative measurements can be fused with individual measurements to reduce estimation error over a group. Collaboration also allows for localizing all vehicles in one frame of reference, which is advantageous for applications such as formation control.

In this algorithm, we initially detect features from all cameras and extract common features through matching. The matches are then propagated through an adaptive RANSAC technique [ref] which results in the creation of a map. Once created, this map is then accessible from all vehicles, allowing each vehicle to perform its own localization through feature tracking and 3D-2D correspondences. We assume that the metric distance and heading between at least two vehicles is known before initializing the algorithm in order to have an estimate of the scale. As the vehicles move around, if the number of tracked features falls below a certain threshold, the vehicles ‘collaborate’ once more to match common features and update the global map. Thus, continuous communication between vehicles is not necessary. The frequency at which the map update happens depends on factors such as how fast the movement of the UAVs is, how quickly the environment can change etc. We are currently also looking into techniques such as covariance intersection so that two vehicles can use relative measurements between each other if needed, without having to perform a full map update.

Recently, we have tested this algorithm using Microsoft AirSim, the UAV simulator which was modified in order to simulate multiple vehicles. AirSim uses Unreal Engine as its base, a high fidelity videogame enigne with features such as high resolution textures, realistic shadows, post processing and visual effects. A sample video of localization being performed using AirSim images can be seen below.