Difference between revisions of "Sensor Data Acquisition"

From Earth Science Information Partners (ESIP)
Line 2: Line 2:
  
 
==Overview==
 
==Overview==
Traditionally, environmental sensor data from remote field sites were manually retrieved during infrequent site visits. However, with today's technology, these data can now be acquired in real-time. Indeed, there are several methods of automating data acquisition from remote sites, but there is insufficient knowledge among the environmental sensor community about their availability and functionality. Moreover, there are several factors that should be taken into consideration when choosing a remote data acquisition method, including [[#Real-time|how immediately the data are needed]], [[#Bandwidth|bandwidth]], [[#Protocols|hardware and network protocols]], [[#Line-of-sight|line-of-sight]], [[#Power|power consumption]], [[#Security|security]], [[#Reliability and Redunancy|reliability and redundancy]], [[Expertise|expertise]], and [[#Budget|budget]]. Here, we provide an overview of these methods and recommend best practices for their implementation.
+
Traditionally, environmental sensor data from remote field sites were manually retrieved during infrequent site visits. However, with today's technology, these data can now be acquired in real-time. Indeed, there are several methods of automating data acquisition from remote sites, but there is insufficient knowledge among the environmental sensor community about their availability and functionality. Moreover, there are several factors that should be taken into consideration when choosing a remote data acquisition method, including [[#Collection Frequency|desired data collection frequency]], [[#Bandwidth|bandwidth requirements]], [[#Protocols|hardware and network protocols]], [[#Line-of-sight|line-of-sight]], [[#Power|power consumption]], [[#Security|security]], [[#Reliability and Redunancy|reliability and redundancy]], [[Expertise|expertise]], and [[#Budget|budget]]. Here, we provide an overview of these methods and recommend best practices for their implementation.
  
 
==Introduction==
 
==Introduction==
Line 14: Line 14:
 
The decision of which sensor data acquisition method to use at a given site requires the careful consideration of many factors, for which we provide an overview here.  
 
The decision of which sensor data acquisition method to use at a given site requires the careful consideration of many factors, for which we provide an overview here.  
  
====Site Selection====
+
====Collection Frequency====
Automated data acquisition and transmission methods require many of the same site selection considerations discussed in [[Sensor Site and Platform Selection|Sensor Site and Platform Selection]].
+
What is the desired collection frequency? How important is real-time accessibility? For instance, the data could be retrieved in near real-time (every few minutes to every few hours) or just once or twice per day. High frequency datasets or images should be collected more frequently.
 
 
====Collection Interval====
 
At what time scale should the data be collected? For instance, the data could be retrieved in near real-time (every few minutes to every few hours) or just once or twice per day.
 
  
 
====Bandwidth====
 
====Bandwidth====
 
Bandwidth can be an important consideration, particularly when high frequency data are being collected.   
 
Bandwidth can be an important consideration, particularly when high frequency data are being collected.   
Will cameras be utilized at the site? Where is broadband point of presence (POP) located? Does equipment work with required bandwidth? collection/access interval
+
Will cameras be utilized at the site? Where is broadband point of presence (POP) located? Does equipment work with required bandwidth? More frequent collection intervals require less bandwidth per transmission are are recommended for high frequency datasets or for images.
  
 
====Protocols====
 
====Protocols====
IP: private vs public networks
+
=====Hardware=====
Serial:
+
Many dataloggers only have serial (RS232) ports, therefore requiring a serial-to-ethernet converter to interface with automated acquisition instrumentation. USB.
Many field instrumentation only comes with serial ports, therefore a Serial-to-Ethernet (e.g. Campbell Scientific NLxxx series)  converter is required to interface with transmission instrumentation, which often comes with ethernet port.    
+
=====Network=====
 +
Private vs public IP networks.
  
 
====Line-of-sight====
 
====Line-of-sight====
 
Evaluation of environment, topography, and vegetation.  Can be initially determined using LOS calculators, which use DEM models, but must be ground truthed. Often requires a repeater infrastructure.
 
Evaluation of environment, topography, and vegetation.  Can be initially determined using LOS calculators, which use DEM models, but must be ground truthed. Often requires a repeater infrastructure.
 
Choosing repeater locations involves many of the same considerations for choosing site selection.  Distance to repeater is a factor.
 
Choosing repeater locations involves many of the same considerations for choosing site selection.  Distance to repeater is a factor.
 +
Automated sensor data acquisition methods require many of the same site selection considerations discussed in [[Sensor Site and Platform Selection|Sensor Site and Platform Selection]], particularly when selecting repeater sites.
  
 
====Power====
 
====Power====
Line 58: Line 57:
  
 
==Best Practices==
 
==Best Practices==
 +
collection interval for high frequency data and images.
  
 
==Case Studies==
 
==Case Studies==

Revision as of 11:19, April 22, 2014

back to EnviroSensing Cluster main page

Overview

Traditionally, environmental sensor data from remote field sites were manually retrieved during infrequent site visits. However, with today's technology, these data can now be acquired in real-time. Indeed, there are several methods of automating data acquisition from remote sites, but there is insufficient knowledge among the environmental sensor community about their availability and functionality. Moreover, there are several factors that should be taken into consideration when choosing a remote data acquisition method, including desired data collection frequency, bandwidth requirements, hardware and network protocols, line-of-sight, power consumption, security, reliability and redundancy, expertise, and budget. Here, we provide an overview of these methods and recommend best practices for their implementation.

Introduction

The classic method of acquiring environmental sensor data from remote field sites involves routine technician site visits, in which s/he connects a laptop to a datalogger, an electronic device that records sensor data over time, and manually downloads data recorded since the last site visit. Once the technician returns to the lab, s/he is then responsible for manually uploading these data to a server for later processing and archival.

While manual acquisition methods are generally effective, there are many reasons to automate environmental sensor data acquisition. For instance, if the site is not visited frequently enough, the datalogger memory can become full and depending on how the datalogger is programmed, sensor data will either overwrite itself or stop recording entirely. This situation often occurs at remote sites that become periodically inaccessible due to environmental conditions, such as heavy winter snow pack. Second, the burden of responsibility for not only the successful retrieval of the sensor data, but also the subsequent upload to a server for safekeeping, lies solely on the technician. Moreover, with any instrumented site, there is the inherent potential for sensor or power failure. Automated data acquisition systems allow technicians to learn of such issues prior to visiting the field site, reducing the potential for data loss. Finally, automated data acquisition methods save hundreds of person hours and vehicle miles that would have otherwise been spent manually acquiring data or troubleshooting unanticipated problems, thus improving the overall quality of the data.

Bidirectional communication methods have the additional advantages of allowing technicians to remotely change system settings, test configurations, and troubleshoot problems. These methods also open the field to a wide variety of devices that may be deployed at a remote field site, such as controllable cameras, on-site wireless hotspots, and IP-enabled control or automation equipment.

Considerations

The decision of which sensor data acquisition method to use at a given site requires the careful consideration of many factors, for which we provide an overview here.

Collection Frequency

What is the desired collection frequency? How important is real-time accessibility? For instance, the data could be retrieved in near real-time (every few minutes to every few hours) or just once or twice per day. High frequency datasets or images should be collected more frequently.

Bandwidth

Bandwidth can be an important consideration, particularly when high frequency data are being collected. Will cameras be utilized at the site? Where is broadband point of presence (POP) located? Does equipment work with required bandwidth? More frequent collection intervals require less bandwidth per transmission are are recommended for high frequency datasets or for images.

Protocols

Hardware

Many dataloggers only have serial (RS232) ports, therefore requiring a serial-to-ethernet converter to interface with automated acquisition instrumentation. USB.

Network

Private vs public IP networks.

Line-of-sight

Evaluation of environment, topography, and vegetation. Can be initially determined using LOS calculators, which use DEM models, but must be ground truthed. Often requires a repeater infrastructure. Choosing repeater locations involves many of the same considerations for choosing site selection. Distance to repeater is a factor. Automated sensor data acquisition methods require many of the same site selection considerations discussed in Sensor Site and Platform Selection, particularly when selecting repeater sites.

Power

How important is real-time accessibility? (e.g., what is desired collection frequency?). What are the transmission type power requirements, onsite buffer size. Redundancy is preferred, especially in very remote sites. If power is disrupted, will system resume operations?

Security

Physical Security

see Sensor, Site, and Platform Selection

Network Security

Encryption keys, VPN

Reliability and Redundancy

of transmission mode and of equipment

Expertise

Plug-n-play vs.

Budget

Costs of implementing a data acquisition and transmission method depend on existing infrastructure, initial setup costs including personnel, personnel costs, specifically technician maintenance, and recurring costs, such as monthly recurring costs with cellular transmission.

Methods

There are three main categories of remote data acquisition methods: manual, unidirectional telemetry, and bidirectional telemetry. Each has advantages and disadvantages in terms of infrastructure, cost, reliability, required expertise, and power consumption.

Manual

This method involves scheduled visits to the site by a field technician, who uses a serial-to-computer connection and/or flash memory transfer of environmental sensor data to their laptop or similar device. Upon returning from the field, the technician is responsible for manually uploading these data to a server. This acquisition method is simple and may be the only option when site instrumentation generates large data files. However, this method provides no real-time data access and therefore, no knowledge of instrumentation failures. Moreover, the reliability of this method is completely dependent on the technician.

Unidirectional

Bidirectional

Best Practices

collection interval for high frequency data and images.

Case Studies

Resources

References