Difference between revisions of "Sensor Management Tracking and Documentation"

From Earth Science Information Partners (ESIP)
Line 23: Line 23:
  
 
===Documentation at setup time===
 
===Documentation at setup time===
* location lat, long, elevation (and/or depth), direction (e.g. camera facing north), Location from a certain reference point (e.g. tower base)
+
* Location lat, long, elevation (and/or depth), direction (e.g. camera facing north), Location from a certain reference point (e.g. tower base)
* site description
+
* Site description
* site photos with metadata, photos of procedures (how do you change ...), photo of sensor (so others can easily recognize)
+
* Site photos with metadata, photos of procedures (how do you change ...), photo of sensor (so others can easily recognize)
* manufacturers specs and ID of instruments (make, model, serial number, range, precision, detection limit, calibration coefficient)
+
* Manufacturers specs and ID of instruments (make, model, serial number, range, precision, detection limit, calibration coefficient)
* instrumentation (e.g. datalogger, multiplexer, sensor) wiring diagrams (this should be part of the logger program comments, a header section with the wiring description channel by channel)
+
* Instrumentation (e.g. datalogger, multiplexer, sensor) wiring diagrams (this should be part of the logger program comments, a header section with the wiring description channel by channel)
* power wiring diagrams (e.g. how many solar panels, are they hooked up in series or parallel, etc.)
+
* Power wiring diagrams (e.g. how many solar panels, are they hooked up in series or parallel, etc.)
 
* Network topology and IP addresses
 
* Network topology and IP addresses
 
* Instrumentation deployment date (the “go live” date)  
 
* Instrumentation deployment date (the “go live” date)  
  
 
===Infrastructure events to track===
 
===Infrastructure events to track===
* Changes to dataloggers, multiplexers, or datalogger programs
+
* Changes to dataloggers, multiplexers, or datalogger programs (datalogger programs may be archived)
 
* Power problems, including battery voltage
 
* Power problems, including battery voltage
 
* Enclosure temperature and humidity
 
* Enclosure temperature and humidity
Line 41: Line 41:
  
 
===Site level events to track===
 
===Site level events to track===
* Site disturbance  
+
* Site disturbance (e.g., animal, human, weather caused)
* site visits (presence of people may change measurements)
+
* Site visits (presence of people may change measurements)
* site maintenance (cutting brush, cutting trees, etc.)
+
* Site maintenance (e.g., cutting brush, cutting trees, etc.)
* animal or human caused
 
* weather caused
 
 
* Changes to sensor network design, including additions or deletions of sensors
 
* Changes to sensor network design, including additions or deletions of sensors
 
   
 
   
 
===Sub-component  events to track===
 
===Sub-component  events to track===
Here, we include components like individual telemetry, power systems, instruments, sensor components, etc. While each component doesn’t affect the whole system, they still may influence the interpretation of the measurements.
+
Here, we include components like individual telemetry, power systems, instruments, sensor components, etc. While each component doesn’t affect the whole system, they still may influence the interpretation of the measurements. To track individual components a system of IDs may be developed for all components and supported by Barcodes, Geo-Location Tags and Microchip Encoded Sensors.
* sensor failures
+
* Sensor failures
* sensor calibrations
+
* Sensor calibrations
* sensor removal
+
* Sensor removal
* sub-sensor addition, removal, or change (pluggable sub-sensor positions within the main sensor need to be noted and kept consistent)
+
* Sub-sensor addition, removal, or change (pluggable sub-sensor positions within the main sensor need to be noted and kept consistent)
* sensor installation (replacement)
+
* Sensor installation (replacement)
* sensor maintenance (cleaning, change of parts)
+
* Sensor maintenance (cleaning, change of parts)
* sensor firmware upgrades
+
* Sensor firmware upgrades
 
* Enclosure temperature and humidity
 
* Enclosure temperature and humidity
* repositioning of sensor (move up during winter to be above snowline
+
* Repositioning of sensor (e.g., move up during winter to be above snowline
* normal (non extreme) disturbances as they are noted and removed (sticks in weirs)
+
* Normal (non extreme) disturbances as they are noted and removed (e.g., sticks in weirs)
* methodology changes, e.g., temperature radiation shield change
+
* Methodology changes (e.g., temperature radiation shield change)
 +
 
  
  
* Sensor Identification and Tracking
 
** Location Information (Spatial-Temporal)
 
** Service History
 
** Tracking Options
 
*** Bar Codes
 
*** Geo-Location Tags
 
*** Microchip Encoded Sensors (NEON 'Grape')
 
* Maintenance & Tracking of Sensor Events
 
** Deployments (Spatial-Temporal)
 
** Methodology: Frequency, Calibration, Quality Filters, Sampling Program, Equipment Change
 
** Calibrations
 
** Other Service Actions
 
** Failures
 
** Replacement
 
 
* Middleware Capabilities
 
* Middleware Capabilities
 
** Workflow Management
 
** Workflow Management
 
** Proprietary Software
 
** Proprietary Software
* Geo-Location Referencing
 
** Geo-Referenced Data Stream
 
** Photo Documentation
 
*** Hemispheric photos
 
*** Time-Series Image Collections
 
* Data Stream Processing
 
** System Configuration (Hardware/Software/History/Legacy)
 
*** Datalogger Program Archive
 
*** Sensor Wiring Diagrams
 
 
** Data Workflow
 
** Data Workflow
 
*** Processing Procedures: Current Protocol, Legacy Operations
 
*** Processing Procedures: Current Protocol, Legacy Operations

Revision as of 14:14, April 8, 2014

back to EnviroSensing Cluster main page

Overview

Automated observation systems need to be managed for optimal performance. Maintenance of the overall sensor system include anything from repairs, replacements, changes to the general infrastructure, to deployment and operation of individual sensors, and seasonal or event driven site clean up activities. Any of these activities in the field may affect the data being collected. Therefore, consistent and uniform records of maintenance, service, and changes to field instrumentation and supporting infrastructure serve as metadata for long term quality control and evaluation of the sensor data.

In this chapter, we describe the types of management records that should be kept and the various methods for collecting, maintaining, communicating, and connecting this information to the data. It is important to create tracking and documentation protocols early on because these protocols will support and guide communications and work between field and data management personnel.

Real time monitoring of system health and alerting systems are discussed in the middleware, quality control, and transmission sections of this document. Although some of these parameters do not affect the actual data quality, tracking of these system performance diagnostic data may be helpful to detect patterns and prevent future data loss, intervene remotely, and schedule site visits more effectively. Calibration procedures and schedules, maintenance activities, and replacement schedules are hardware specific and will not be covered here in detail.

Introduction

Data are collected to detect changes in the environment, effects of treatments, disturbances etc., and in all data collection great care is taken to not mask the signature of events of interest with impacts from unavoidable, sampling related disturbances. Field notes are usually associated with the raw data to be able to discern a natural event of interest from a management event. Data collection approaches using automated sensing networks are becoming more complex with many people involved in the data gathering, management, and interpretation activities, and communication among all involved parties is becoming more important and more challenging. Field notes can be a useful vehicle for this communication. Everyone using older long-term data knows the value of field note books to help understand and interpret a dataset. Field notes are equally valuable to future users for a sensor data stream, particularly if the notes are interpreted such that information is integrated with the data via data qualifying flags and method description codes.

Currently there are no standards for flag code sets or for defining which events should be flagged and how to efficiently communicate with data users. Here we attempt to present a list of events that are useful to track and that have been helpful in the past to guide data users in the interpretation and evaluation of the data. To manage this information the concepts of a ‘logical sensor’, a ‘physical sensor’, a ‘method’ and ‘event codes’ have proven useful.

A ‘logical sensor’ or a sensor data stream can be defined by a location, height/depth, and measurement parameter, regardless of what exact physical sensor or hardware is used to log measurements. An example would be ‘air temperature at 3 m above the ground at site A’. However, over time the ‘physical sensor’ will have to be calibrated, eventually replaced, and a new type of sensor may be chosen to provide more accurate measurements. If hardware is swapped out for technical reasons, the data stream still represents the site location for that measurement, and the notion of a ‘logical sensor’ allows identification of a consistent data stream over time.

Changes in the type of sensor or ‘method’ might be tracked with a method code associated with the logical sensor. Of course should a replacement sensor be significantly different such that the past and new data stream are not comparable, then a new logical sensor stream should be initiated. Events such as routine calibration might be flagged with an ‘event code’ rather than a change in ‘method’, even if this event has lasting effects on the data, i.e., more accurate data. An event code may serve as a means to link to individual field notes for the event. ‘Physical sensors’ should also be individually identifiable by location and tracked through a calibration or replacement schedule.

Methods

What should be tracked

Basic information on the site and hardware configuration need to be recorded at installation time. During normal operations event tracking can be done at several levels of granularity with respect to a research program. For example, it may be done at the level of the entire infrastructure, at a site, or at a sub-component of a site. The information about each event needs to be propagated or connected to all relevant data streams. Following are examples of what should be tracked at each of the above levels, in terms of impact on the recorded data:

Documentation at setup time

  • Location lat, long, elevation (and/or depth), direction (e.g. camera facing north), Location from a certain reference point (e.g. tower base)
  • Site description
  • Site photos with metadata, photos of procedures (how do you change ...), photo of sensor (so others can easily recognize)
  • Manufacturers specs and ID of instruments (make, model, serial number, range, precision, detection limit, calibration coefficient)
  • Instrumentation (e.g. datalogger, multiplexer, sensor) wiring diagrams (this should be part of the logger program comments, a header section with the wiring description channel by channel)
  • Power wiring diagrams (e.g. how many solar panels, are they hooked up in series or parallel, etc.)
  • Network topology and IP addresses
  • Instrumentation deployment date (the “go live” date)

Infrastructure events to track

  • Changes to dataloggers, multiplexers, or datalogger programs (datalogger programs may be archived)
  • Power problems, including battery voltage
  • Enclosure temperature and humidity
  • Platform maintenance (e.g., tower inspection, tramline leveling, etc.)
  • Sampling protocol changes (e.g., timing, routine changing or upgrading of sensor parts, instrument change or replacement)
  • RF/network performance degradation (prevents some/all data from being transmitted; track health/status of IP network devices using SNMP streams to Nagios, etc.)

Site level events to track

  • Site disturbance (e.g., animal, human, weather caused)
  • Site visits (presence of people may change measurements)
  • Site maintenance (e.g., cutting brush, cutting trees, etc.)
  • Changes to sensor network design, including additions or deletions of sensors

Sub-component events to track

Here, we include components like individual telemetry, power systems, instruments, sensor components, etc. While each component doesn’t affect the whole system, they still may influence the interpretation of the measurements. To track individual components a system of IDs may be developed for all components and supported by Barcodes, Geo-Location Tags and Microchip Encoded Sensors.

  • Sensor failures
  • Sensor calibrations
  • Sensor removal
  • Sub-sensor addition, removal, or change (pluggable sub-sensor positions within the main sensor need to be noted and kept consistent)
  • Sensor installation (replacement)
  • Sensor maintenance (cleaning, change of parts)
  • Sensor firmware upgrades
  • Enclosure temperature and humidity
  • Repositioning of sensor (e.g., move up during winter to be above snowline
  • Normal (non extreme) disturbances as they are noted and removed (e.g., sticks in weirs)
  • Methodology changes (e.g., temperature radiation shield change)


  • Middleware Capabilities
    • Workflow Management
    • Proprietary Software
    • Data Workflow
      • Processing Procedures: Current Protocol, Legacy Operations
      • Quality Filters: Application/Log
      • Quality Flags: Application/Log
        • Quality Control Actions
          • Automated
          • Human-Reviewed
          • Log