Knowing Your Data

It seems to me that one of the more important general goals of scientific research is to know ones data. It's straightforward enough (though often laborious in practice) to follow the textbook scientific method you learned in school: 

1. Make an observation.
2. Ask a question.
3. Form a hypothesis, or testable explanation.
4. Make a prediction based on the hypothesis.
5. Test the prediction.
   

However, when one tests the prediction (Step 5), one may end up with a bad result--whether a false confirmation of the prediction or a false rejection of the prediction--unless one knows their data well.

This happened to me a couple years back. I was studying these enigmatic features on Mars that we call Recurring Slope Lineae (RSL). (See picture below and, better yet, see a gif here.) RSL. In a nutshell, RSL are dark squiggly lines that grow down steep slopes seasonally, usually when temperatures are locally warmest, and then fade as temperatures cool. The exact origins of RSL are presently unknown but there are essentially two schools of thought: either (1) RSL form because of the bulk flow of water (or brine) on or very near the surface, or (2) RSL form because dust and/or sand avalanches downslope, removing bright dust in its wake.

When RSL were first discovered about 10 years ago, it was recognized that the slope angle could be an important way to decide between these two schools of thought. Water can flow over very shallow slopes (<1°) but sand stops flowing on slopes below the dynamic angle of repose, which is ~28-35° based on observations of martian dunes. (This is similar to the dynamic angle of repose on Earth, and in fact, the gravity of Mars is expected to have no effect on this critical angle.) Although all RSL are observed only on steep slopes, a critical question is whether they flow on slopes of <28°.

At the time, I was conducting a detailed analysis of RSL at Tivat crater, which included mapping every RSL at the site. I also had a digital terrain model (or DTM, an image where every pixel has a measured elevation) so I could derives slopes from that model and test whether RSL flowed over slopes of <28°. At the time, I preferred the "wet" school of thought and therefore expected RSL to flow over shallow slopes. When I tested that prediction, it seemed confirmed! Not only did RSL frequently flow over slopes of <28°, but in the locations where they ended or locally widened (as though pooling), the slopes were very shallow. Perhaps, below the resolution limits of the DTM, these areas were flat so that even water could not flow! But...

Then I took a closer look and really got to know my data. What I already knew was that there was some undulation in the slope map that I derived (left image in panel d below). Initially, I was inclined to trust that the undulation was genuinely part of the topography because I knew the painstaking care that was used in generating the DTM and that the DTM had high vertical precision (~43 cm or better). However, each pixel in the DTM is only 1 m by 1 m, and the slope map was derived by calculating the pixel-to-pixel slope. An erroneous change of up to 43 cm over a 1 m span could drastically modify the measured slope angle!

To test whether the undulation were real, I decided to re-create a visual image that had been taken of the sight (panel a above). I used the Mars24 Sunclock to determine the elevation and direction of the sun at the time the image was taken. I then used the ArcGIS software to generate a hillshade that illuminates the DTM with the geometry that I'd determined from the Sunclock. The result was an image (left image in panel b above) that reasonably re-created the original image but with some obvious noise, including the undulation that was so important to my original interpretation. I experimented with different ways to smooth the DTM and ultimately settled on a combination that generated the hillshade shown in right image in panel b. That hillshade well re-creates the original visual image, however, it lacks the low slopes that I had originally measured. Instead, the RSL at Tivat were exclusively found on steep slopes and the locations where RSL ended or widened were locations where the slope was 25-35° and shallowed relative to the span immediately upslope (below). Taken together, this suggests that RSL are specifically sensitive to shallowing below the dynamic angle of repose, which is consistent with the "dry" school of thought.