Difference between revisions of "EnviroSensing Monthly telecons"

Revision as of 15:04, September 26, 2014

Telecons on the fourth Tuesday of every month at 4:00pm ET. Click on 'join'

Next telecon October 28, 2014, 4:00 PM EDT, Wade Sheldon: GCE Matlab data toolbox

November 25, 2014, 4:00 PM EDT. Jordan Read will introduce the R package SensorQC

Notes from past telecons

9/23/2014

Fox Peterson (Andrews LTER) reported on QA/QC methods they are applying to historic climate records (~13 million data points for each of 6 sites).

The challenge was that most automated approaches still produced too many flagged data that needed to be manually checked. Multiple statistical methods were tested based on long-term historical data. The method they selected was to use a moving window of data from the same hour over 30 days and test for 4 standard deviations in that window; E.g., use all data for 1 pm for days 30 - 60 of the year, compute four standard deviations, and set the range for the midpoint day (45) at the 1pm hour to that range.

Josh Cole reported on his system, which is in development and he will be able to share scripts with the group.
Brief discussion of displaying results using web tools.
Great Basin site discussed the variability in their data, which "has no normal"-- how could we perform qa/qc based on statistics and ranges in this case?
Discussion of bringing Wade Sheldon to call next time / usefulness of the toolbox for data managers
Discussion of using Pandas package- does anyone have experience, can we get them on?
Discussion of the trade off between large data stores, computational strength, and power. Good solutions?
ESIP email had some student opportunities which may be of interest
Overall, it was considered helpful if people were willing to share scripts. Discussion of a GIT repository for the group, or possibly just use the Wiki.

8/26/2014

Suggestions for future discussion topics

Citizen Science contributions to environmental monitoring
'open' sensors - non-commercial sensors made in-house, technology, use, best practices
Latest sensor technologies
Efficient data processing approaches
Online data visualizations
New collaborations to develop new algorithms for better data processing
Sensor system management tools (communicating field events and associating them with data)

@@ Line 8: / Line 8: @@
 ===Notes from past telecons===
 ==9/23/2014==
-Fox Petersen (Andrews LTER) reported on QA/QC methods they are applying to historic climate records (~13 million data points for each of 6 sites). The challenge was that most automated approaches still produced too many flagged data that needed to be manually controlled. The best method they found was to use a moving bin of data from the same hour over several days and apply statistical test to them. E.g., use all data for 1 pm for days 30 - 50 of the year and apply range checks etc. then move to the next bin.
+* Fox Peterson (Andrews LTER) reported on QA/QC methods they are applying to historic climate records (~13 million data points for each of 6 sites).
+The challenge was that most automated approaches still produced too many flagged data that needed to be manually checked. Multiple statistical methods were tested based on long-term historical data. The method they selected was to use a moving window of data from the same hour over 30 days and test for 4 standard deviations in that window; E.g., use all data for 1 pm for days 30 - 60 of the year, compute four standard deviations, and set the range for the midpoint day (45) at the 1pm hour to that range.
+* Josh Cole reported on his system, which is in development and he will be able to share scripts with the group.
+* Brief discussion of displaying results using web tools.
+* Great Basin site discussed the variability in their data, which "has no normal"-- how could we perform qa/qc based on statistics and ranges in this case?
+* Discussion of bringing Wade Sheldon to call next time / usefulness of the toolbox for data managers
+* Discussion of using Pandas package- does anyone have experience, can we get them on?
+* Discussion of the trade off between large data stores, computational strength, and power. Good solutions?
+* ESIP email had some student opportunities which may be of interest
+* Overall, it was considered helpful if people were willing to share scripts. Discussion of a GIT repository for the group, or possibly just use the Wiki.
-Josh Cole reported on his system, which is in development and he will be able share scripts with the group.
-Overall, it was considered helpful if people where willing to share scripts. We'll have to look into good ways to keep them, e.g. git or just on this wiki.
 ==8/26/2014==