INF550: EcoInformatics data collection tools & products,
Preface
Acknowledgements
Key Contributors
Pre-Course Setup: EcoInformatics Tools
0.1
Pre-Course Skills & Setup
0.1.1
Installing or Updating R
0.1.2
Windows R/RStudio Setup
0.1.3
Mac R/RStudio Setup
0.2
Linux R/RStudio Setup
0.2.1
Install basic packages for this course
0.3
Installing and Setting up Git & Github on Your Machine
0.4
Installing Atom
0.5
Linking RStudio to Git
0.6
How we will be Conducting this Course
0.7
Exercises:
1
Why ‘EcoInformatics’?
1.1
The Framework of this Course
1.2
Final Course Project: Proposed Derived Data Product
2
Introduction to NEON & its Data
2.1
Learning Objectives
2.2
The NEON Project Mission & Design
2.3
The Science and Design of NEON
2.4
NEON’s Spatial Design
2.4.1
NEON Samples All 20 Eco-Regions
2.5
How NEON Collects Data
2.5.1
Specimens & Samples
2.5.2
Airborne Remote Sensing
2.6
Accessing NEON Data
2.6.1
Pathways to access NEON Data
2.7
Hands on: Accessing NEON Data & User Tokens
2.7.1
Via the NEON API, with your User Token
2.7.2
Get a NEON API Token
2.7.3
Use the API token in neonUtilities
2.7.4
Token management for open code
2.8
Hands on: NEON TOS Data
2.8.1
Pull in Tree Data from NEON’s TOS and investigate relationships
2.9
Intro to NEON Exercises Part 1
2.9.1
Computational
2.10
Part 2: Pulling NEON Data via the API
2.11
What is an API?
2.11.1
Anatomy of an API call
2.11.2
Targets:
2.11.3
Observational data (OS)
2.11.4
Instrumentation data (IS)
2.11.5
Remote sensing data (AOP)
2.11.6
Geolocation data
2.11.7
Querying a single named location
2.11.8
Taxonomy
2.12
Stacking NEON data
2.13
Intro to NEON Exercises: Written Questions
2.14
Intro to NEON Exercises Part 2
2.14.1
Written
2.14.2
NEON Coding Lab Part 2
2.15
NEON Coding Lab Part 2
2.16
Intro to NEON Culmination Activity
3
Introduction to USA-NPN & its Data
3.1
Hands on: Accessing USA-NPN Data via rNPN
3.1.1
Introduction
3.1.2
Accessing USA-NPN Observational Data
3.1.3
Status and Intensity Data
3.1.4
Individual Phenometrics
3.1.5
Site Phenometrics
3.1.6
Magnitude Phenometrics
3.1.7
USA-NPN Geospatial Data
3.2
Accumulated Growing Degree Day Products
3.3
Extended Spring Indices
3.3.1
Other Layers
3.4
Putting it all together:
3.5
Combine Point and Raster Data
3.5.1
Step 1: Get the data
3.5.2
Step 2: Preparing the data
3.5.3
Step 3: Define style options and create graph
3.6
Live Demo Code with Lee Marsh of USA-NPN
3.6.1
Basic Utility Functions
3.6.2
Download Observational Data
3.6.3
Magnitude Data
3.6.4
Downloading Geospatial Data
3.6.5
Putting it together
3.6.6
Other Data Sources, e.g. Daymet, MODIS
3.7
USA-NPN Coding Lab
3.8
NEON TOS Phenology Data Lecture
3.9
Understanding Observation Biases and Censoring in Citizen Science Data
3.9.1
Many existing modeling approaches fail in one (or both) regards.
3.9.2
Censoring
3.10
So then how can we model censored data?
3.11
Intro to USA-NPN Culmination Activity
4
PhenoCam: Digital Repeat Photography Networks & Methods
4.1
Digital Repeat Photography Networks Learning Objectives
4.2
The PhenoCam Network Mission & Design
4.2.1
Relevant documents & background information:
4.3
PhenoCam’s Spatial design:
4.3.1
PhenoCam as a Near Surface Remote Sensing Technique
4.3.2
How PhenoCams Pull Data
4.3.3
Leveraging camera near-infrared (NIR) capabilities
4.4
Digital Repeat Photography Written Questions
4.5
Introduction to Digital Repeat Photography Methods
4.6
Pulling Data via the
phenocamapi
Package
4.7
Exploring PhenoCam metadata
4.7.1
Remove null values
4.7.2
Filtering using attributes
4.8
Download midday images
4.8.1
Download midday images for a given time range
4.9
Detecting Foggy Images using the ‘hazer’ R Package
4.10
Extracting Timeseries from Images using the xROI R Package
4.10.1
xROI Design
4.10.2
Install xROI
4.10.3
Launch xROI
4.10.4
End xROI
4.10.5
Use xROI
4.11
Documentation and Citation
4.11.1
Challenge: Use xROI
4.12
Hands on: Digital Repeat Photography Computational
4.12.1
PhenoCam time series
4.12.2
Obtain ROIs
4.12.3
Download time series
4.12.4
Threshold values
4.12.5
Comparing phenology across vegetation types
4.12.6
In Class Hands-on Coding: Comparing phenology of the same plant function type (PFT) across climate space
4.13
Digital Repeat Photography Coding Lab
4.13.1
Quantifying haze and redness to evaluate California wildfires
4.14
PhenoCam Culmination Activity
5
Flux Measurements & Inter-Operability
5.1
Learning Objectives
5.2
Eddy Co_variance Data: What does it actually measure?
5.2.1
Example Eddy Site
5.3
QA/QC Flags
5.4
Examples of Other Flux Networks: AMERIFLUX & FLUXNET
5.5
The Power of Networked Ecology: Bridging to AMERIFLUX and Beyond
5.6
Hands On: Introduction to working with NEON eddy flux data
5.6.1
Setup
5.6.2
Data Levels
5.6.3
Extract Level 4 data (Fluxes!)
5.6.4
Time stamps
5.6.5
Merge flux data with other sensor data
5.6.6
Vertical profile data (Level 3)
5.6.7
Un-interpolated vertical profile data (Level 2)
5.6.8
Calibrated raw data (Level 1)
5.6.9
Convert NEON flux data variables to AmeriFlux FP standard
5.7
Exercises
5.7.1
Computational
6
NEON AOP
6.1
Hyperspectral Remote Sensing
6.1.1
Learning Objectives
6.1.2
About Hyperspectral Remote Sensing Data
6.2
Key Metadata for Hyperspectral Data
6.2.1
Bands and Wavelengths
6.2.2
Spectral Resolution
6.2.3
Full Width Half Max (FWHM)
6.3
Intro to Working with Hyperspectral Remote Sensing Data in HDF5 Format
6.4
Light Detection And Ranging (LiDAR) Data
6.5
Calculating Forest Structural Diversity Metrics from NEON LiDAR Data
6.5.1
Learning Objectives
6.5.2
R Libraries to Install:
6.5.3
Data to Download
6.5.4
Recommended Skills
6.5.5
Additional Resources
6.6
Introduction to Structural Diversity Metrics
6.7
NEON AOP Discrete Return LIDAR
6.7.1
Loading the LIDAR Products
6.7.2
Normalizing Tree Height to Ground
6.8
Calculating Structural Diversity Metrics
6.8.1
GENERATE CANOPY HEIGHT MODEL (CHM)
6.8.2
Combining Everything Into One Function
6.8.3
Comparing Metrics Between Forests
6.9
Matching GEDI waveforms with NEON AOP LiDAR pointclouds
6.9.1
Learning Objectives
6.9.2
Things You’ll Need To Complete This GEDI Section
6.9.3
R Packages to Install
6.9.4
Example Data Set
6.9.5
Getting Started
6.9.6
Downloading GEDI data
6.9.7
Plot GEDI footprints on CHM
6.9.8
Extract Waveform for a single Shot
6.9.9
Download and Plot NEON AOP LiDAR pointcloud data
6.9.10
Clip AOP LiDAR Pointcloud to GEDI footprints
6.9.11
Plot GEDI Waveform with AOP Pointcloud in 3D space
6.9.12
Datum, Geoid, and how to best measure the Earth
6.9.13
Aligning the Vertical Datum
6.9.14
GEOID12A Height Model
6.9.15
Extract vertical offset for GEDI shots
6.9.16
Plot vertically corrected GEDI waveform in 3D
6.9.17
Optional - NEON base plots
6.10
NEON AOP Written Questions:
6.11
NEON AOP Coding Lab
6.12
NEON AOP Culmination Write Up
7
NASA’s Earth Observing System (EOS)
7.1
Learning Objectives
7.2
NASA EOS Project Mission & Design
7.3
NASA EOS Earth Data Account:
7.4
NASA EOS Coding Assignment 1
7.5
Distributed Active Archive Centers
7.5.1
LP DAAC
7.6
The LPDAAC Mission: Process, Archive, Distribute, Apply
7.6.1
How you can use LP DAAC’s data
7.7
AppEEARS
7.8
Hands on: Pulling AppEEARS Data via the API
7.9
Getting Started with the AppEEARS API (Point Request)
7.9.1
Example: Submit a point request for multiple NEON sites to extract vegetation and land surface temperature data
7.10
Topics Covered in this section:
7.10.1
Prerequisites:
7.10.2
AppEEARS Information:
7.11
Getting Started with the AppEEARS API
7.11.1
Load Packages
7.11.2
Set Up the Output Directory
7.11.3
Login to Earth Data
7.12
Query Available Products
7.13
Search and Explore Available Products
7.14
Search and Explore Available Layers
7.15
Submit a Point Request
7.15.1
Compile a JSON Object
7.15.2
Submit a Task Request
7.15.3
Retrieve Task Status
7.16
Download a Request
7.16.1
Explore Files in Request Output
7.17
Download Files in a Request (Automation)
7.18
Explore AppEEARS Quality Service
7.18.1
List Quality Layers
7.18.2
Inspect Quality Values
7.19
Decode Quality Values
7.20
Load Request Output and Visualize
7.21
Load a CSV
7.22
Plot Results (Line/Scatter Plots)
7.23
Submit an Area Request
7.24
Submit an area request using a NEON site boundary as the region of interest for extracting elevation, vegetation and land surface temperature data
7.24.1
Topics Covered in this Tutorial
7.24.2
Prerequisites:
7.24.3
Procedures:
7.24.4
AppEEARS Information:
7.25
Getting Started with an Area Request
7.25.1
Load Packages for an Area Request
7.26
Set Up the Output Directory
7.27
Load your Earth Data Token
7.28
Query Available Products
7.29
Search and Explore Available Products
7.30
Search and Explore Available Layers
7.31
Submit an Area Request
7.31.1
Load a Shapefile
7.32
Search and Explore Available Projections
7.33
Compile a JSON Object
7.34
Submit a Task Request
7.35
Retrieve Task Status
7.36
Download a Request
7.36.1
Explore Files in Request Output
7.37
Download Files in a Request (Automation)
7.38
Explore AppEEARS Quality Service
7.39
List Quality Layers
7.40
Show Quality Values
7.41
Decode Quality Values
7.42
BONUS: Load Request Output and Visualize
7.43
Load a GeoTIFF
7.44
Plot a GeoTIFF
7.45
NASA EOS Coding Lab #2
7.46
NASA EOS Written Questions
7.47
NASA EOS Culmination Write Up
8
USGS National Water Information System (NWIS)
8.1
USGS Mission:
8.2
USGS Water Resources Mission:
8.3
Types of USGS NWIS Data
8.4
USGS R Packages: Collaborative and reproducible data analysis using R
8.4.1
Suggested prerequisite knowledge
8.4.2
Course outline
8.4.3
Software requirements
8.4.4
Lesson Summary
8.4.5
Lesson Objectives
8.4.6
Lesson Resources
8.4.7
Lesson Slide Deck
8.5
Introduction to USGS R Packages
8.6
Data available
8.7
Common NWIS function arguments
8.8
Discovering NWIS data
8.8.1
whatNWISdata
8.9
Common WQP function arguments
8.10
Discovering WQP data
8.10.1
readWQPdata + querySummary
8.10.2
whatWQPsites
8.11
readNWIS functions
8.11.1
readNWISdata
8.11.2
readNWISdv
8.11.3
readNWISgwl
8.11.4
readNWISmeas
8.11.5
readNWISpCode
8.11.6
readNWISpeak
8.11.7
readNWISqw
8.11.8
readNWISrating
8.11.9
readNWISsite
8.11.10
readNWISstat
8.11.11
readNWISuse
8.11.12
readNWISuv
8.12
Additional Features
8.12.1
Accessing attributes
8.12.2
Using lists as input
8.12.3
Helper functions
8.13
readWQP functions
8.13.1
readWQPdata
8.13.2
readWQPqw
8.14
Attributes and metadata
8.15
USGS Coding Lab Exercises
8.15.1
Just how dry was the 2020 monsoon?
8.16
geoKnife - Introduction
8.17
Lesson Summary
8.18
Lesson Objectives
8.19
Lesson Resources
8.20
Remote processing
8.21
geoknife components: fabric, stencil, knife
8.22
Available webdata
8.22.1
GDP datasets
8.22.2
Datasets not in GDP
8.23
Available webgeoms
8.24
Available webprocesses
8.25
Setting up a geojob
8.26
Checking the geojob status
8.27
Getting geojob data
8.28
wait
and
email
8.29
Putting it all together, mapping precipitation from Tropical Storm Colin
8.30
USGS NWIS Culmination Write Up
Frequently Asked Questions:
8.31
Where can I find due dates for assignments?
8.32
How do I submit assignments?
8.33
Do I still have to submit written exercises as .Rmd and .pdf?
8.34
What’s better for code, conciseness or readability?
8.35
How find I find resources to navigate the NEON Data Portal?
8.36
How can I best prepare for class and succeed?
9
Fall 2020 IGNITE Session
Published with bookdown
Environmental Informatics Using Research Infrastructures and their Data
Chapter 9
Fall 2020 IGNITE Session