Difference between revisions of "EnviroSensing Monthly telecons"
(3 intermediate revisions by the same user not shown) | |||
Line 9: | Line 9: | ||
===Notes and recordings from past telecons=== | ===Notes and recordings from past telecons=== | ||
To listen to recordings from past telecons, click [https://esip.sharefile.com/login.aspx?cmd=route&id=%252Fapp%252F%2523home%252Fshared%252Ffo871c56-e32f-42e4-a08b-5d8988e0f041 here]. To log in, use the username guest@esipfed.org, and the password Earth111. | To listen to recordings from past telecons, click [https://esip.sharefile.com/login.aspx?cmd=route&id=%252Fapp%252F%2523home%252Fshared%252Ffo871c56-e32f-42e4-a08b-5d8988e0f041 here]. To log in, use the username guest@esipfed.org, and the password Earth111. | ||
+ | |||
+ | == September 5, 2023 == | ||
+ | [https://drive.google.com/file/d/1JUvaZtQpLizAuHn4mPJECXthHEpGhaDI/view?usp=drive_link Link to Recording] | ||
+ | |||
+ | Topic: Summer Recap Cluster Strategic Planning | ||
+ | |||
+ | Date of meeting: Sep 5, 2023 | ||
+ | |||
+ | Participants: | ||
+ | |||
+ | FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details | ||
+ | |||
+ | James (Jake) Gearon - IU Bloomington, ESIP Fellow | ||
+ | |||
+ | Joseph Bell - USGS | ||
+ | |||
+ | Scotty Strachan - Nevada System of Higher Education, NevadaNet | ||
+ | |||
+ | John Porter - UVA | ||
+ | |||
+ | Daniel Fuka - Virginia Tech | ||
+ | |||
+ | David Coyle - USGS | ||
+ | |||
+ | Megan Carter - ESIP | ||
+ | |||
+ | James Gallagher | ||
+ | |||
+ | AGENDA: | ||
+ | |||
+ | Joseph has equipment to be sent out. Some minutes directed for virtual workshop | ||
+ | |||
+ | Virtual Workshop Take 2 | ||
+ | |||
+ | Stated Desires: | ||
+ | |||
+ | Some of the equipment makes it back to Scotty | ||
+ | |||
+ | USDA Equipment that goes offline will likely need to be returned to USDA | ||
+ | |||
+ | Number of Kits x Number of Cellular uplinks y | ||
+ | |||
+ | 2 - John Porter - Grad Student? | ||
+ | |||
+ | 2 - James/Martha | ||
+ | |||
+ | 2 - Jake | ||
+ | |||
+ | 2 - Joseph | ||
+ | |||
+ | 2 - Renee Brown | ||
+ | |||
+ | <nowiki>https://edirepository.org/</nowiki> | ||
+ | |||
+ | <nowiki>https://knb.ecoinformatics.org/</nowiki> | ||
+ | |||
+ | <nowiki>https://developers.planet.com/docs/planetschool/introduction-to-stac-part-1-an-overview-of-the-specification/</nowiki> | ||
+ | |||
+ | <nowiki>https://www.dataone.org/network/</nowiki> | ||
+ | |||
+ | Ezeml | ||
+ | |||
+ | CUASI | ||
+ | |||
+ | Announcements: | ||
+ | |||
+ | Time Stamps (ET): | ||
+ | |||
+ | 17:06 - introductions, etc. coming off of the meeting, we have a chance to restructure. Martha Apple is transitioning out as is Scotty. Sent invites to some folks to discuss co chairing. Idea of hands-on sensor building, cluster focused toward that end (interactive building). | ||
+ | |||
+ | 17:08 - summer meeting recap. Even though it was a back up it went very well, was useful for figuring out the scale of tinkering that is possible given the time constraints. | ||
+ | |||
+ | 17:11 - John P. more show and tell than planned. Good show and tell, though! Good range of people, a relative novice to experienced users. A question to throw out. What is the audience were trying to reach and what do we want them to learn? | ||
+ | |||
+ | 17:12 - Joe, looking ahead, have use cases etc. We can have a suite of parameters (temp, pressure, etc) HydroTech, Grad Student, Researcher, environmental consultant. When i collect this data, how am i collecting this data to share? Use cases Joe would seek are: good data quality, collection, and delivery. Do we get this out through a GS database or something like coords. | ||
+ | |||
+ | 17:14 - James G. What do we think of as graduate students and their capabilities? Where do they start? Have they had python experience? Only informal? | ||
+ | |||
+ | 17:16 - Dan Fuka, are we not trying to hit a slightly lower layer on the technology stack? Actually getting or creating or having a device they can work through to familiarize themselves with data flow. I.e. how do you pass info between devices, etc. Thats the hardest layer to get people started themselves. As we know, each of these layers is fairly complex, programming the arduino etc, using amazon cloud services or other data repositories. How do we get value off of a specific sensor? A workshop towards that initial introduction, not focused on a new set of people, but those focused on ESIP returners and the faithful. Allows folks to start on the same page (those that attend the meetings with frequency) | ||
+ | |||
+ | 17:18 - Scotty: there is the "intro" crowd, i.e., people that know something about using software and basic scripting/programming....and then there is a "technical" crowd that has some combination of DC electronics, mechanical/fabrication, database, software backend, wireless, etc. experience | ||
+ | |||
+ | 17:19 - John P. many permutations of sensor data/communication. Lot of options out there for networking, what is great is to see some working solutions and say “hey thats like my problem”. John took a course that had standardized hardware etc, was all planned out. Towards the notion of training within ourselves. Having some sessions where folks go into some depth about their own solutions or works in progress. Could be more of an office hours. | ||
+ | |||
+ | 17:22 - Joe: needs for use cases. The organization chooses a single source/product is great. This is where to build off the prev session. Using the USDA board. The winter meeting could be use cases, talks, urban vs forest etc. Each session would have a focus of honing the use case, and working with the core group. | ||
+ | |||
+ | 17:25 - John P. would love to play around with daniels board. Important to have one that works and is accessible to people. But its easier to build with success, aka using the usda board. | ||
+ | |||
+ | 17:25 - As scotty steps back, cluster has the opportunity to move forward etc. as such, we need to discuss the audience. There is the internal audience (highly skilled, technical people) then there is the external audience (all others, stakeholders etc). They have questions regarding using/deploying sensors. Do we do a demonstration with those external audience members. Can we touch both audiences? | ||
+ | |||
+ | 17:30 - Megan C. you have to start with who is showing up now, what do they need. Who is not here that we would like to see here. The faithful etc. But there is the grad student/general ESIP population. Could work across committees. | ||
+ | |||
+ | 17:32 Joe, what does the cluster provide now? Site selection, planning data routing, the specific use cases. The grad student that Joe thinks of is one who is focused on sensors etc. maybe we can be a place. Simple build experiment and provide the instruction. And where do I send my data? | ||
+ | |||
+ | 17:34 Dan Fuka: wrapping thoughts together, maybe we need some grad students to pull into this. | ||
+ | |||
+ | 17:36 - James G: don’t really think the problems associated with the boards and the sensors are not unique to what were looking at. There are 1000s of links associated with that sort of stuff. We have always looked at the big picture. There is another technical problem though, you have sensors in the field, how do you get those data out of the sensors and into a place that is visible? This is the part of the use case where we can go where others cant go. | ||
+ | |||
+ | 17:39 - Joe: Q to James, what would we then orient to? 2 ways: go to device and get the data (doesnt really address the problem) Coords and thingspeak/ iot things might be more amenable. Data management, provenance, archiving, etc. are there repositories for those info? Becomes a complicated problem. As the sensor networks we deploy grow larger, the data processing and managing gets more difficult. | ||
+ | |||
+ | 17:41 - Dan Fuka - after our group has common knowledge we can utilize, then we involve others (other esip clusters etc). Lets get our problem to the point where the data is transmitting. Then we can decide on how to move forward. Maybe DataVerse? | ||
+ | |||
+ | 17:42 - Scotty: Thinking out loud here......gonna be difficult to address the full spectrum of problems across the workflow....in a monthly 1-hr cluster....the 2014 assembly of best practices was high-level and it still took forever..... | ||
+ | |||
+ | You may be more successful with some deliverables by really being focused on 1-2 parts of the workflow....and saving the wider discussion for an annual event? | ||
+ | |||
+ | 17:43 - James G. responding to Dan, not sure we have the time for that. May have to accept its gonna be messy and go forward on all fronts. | ||
+ | |||
+ | 17:44 - Joe, what if we took an more “ESIP” view. We can to connect with this cluster (hazards life cycle) set that as the scope for the sensor we build. Could let them choose the data handoff. That approach may provide a filter we need. | ||
+ | |||
+ | 17:47 - Scotty: Joseph needs to recruit folks to put time in etc. | ||
+ | |||
+ | 17:50 - Megan: this makes a strong case for a broader event (annual event) sticky spots etc. talking more broadly, where were going to focus in the near term, but starting with a broader enticement to ESIP. | ||
+ | |||
+ | 17:52 - Joe, we have 2 different vectors. | ||
+ | |||
+ | 17:53 - formalizing a build, then purchasing the build and what practices etc. | ||
+ | |||
+ | 17:54 - Dan Fuka, there are methods out there for how to sense many many things and get the data, start collecting, etc. how to get it into a few repositories. Predominantly commercial. The only problem I have is how to get that into an open data repository. Have had problems with this for years. Where do we put the data? How do we get it to the world? | ||
+ | |||
+ | 17:55 Megan I need to go now. Sounds like there are a lot of good ideas. Just remember everything will take longer than you think in terms of planning. Be kind to yourself in setting milestones. Make sure to build in what you each need to get out of this group and the ability to pivot and make adjustments along the way. | ||
+ | |||
+ | 17:56 - John P. there are solutions (lightweight ones) in the ecological space. EDI (<nowiki>https://edirepository.org/</nowiki>) is where LTER is sending its data these days. But they are open to others, DataOne is similar and related. Etc. <nowiki>https://knb.ecoinformatics.org/</nowiki> | ||
+ | |||
+ | 17:59 - this is clearly a problem and some people are clearly addressing it. The use case is connecting up the sensors to cloud data archives. Come prepared to talk on these topics. | ||
+ | |||
+ | 18:01 - Here is the list of member nodes for dataone. Some of these might work (it includes KNB and EDI): <nowiki>https://www.dataone.org/network/</nowiki> | ||
+ | |||
+ | Do we have people come and talk about that? | ||
+ | |||
+ | 18:03 - See also: Globus: <nowiki>https://docs.globus.org/api/</nowiki> (USGS has a license) - David Coyle | ||
+ | |||
+ | 18:03 - EasyEML | ||
+ | |||
+ | 18:04 - STAC is cool but complex. | ||
+ | |||
+ | 18:05 - Globus is an interesting conversation....started out as a transfer tool.... | ||
+ | |||
+ | James g. We want to know what is good enough, | ||
+ | |||
+ | == June 6, 2023 == | ||
+ | [https://drive.google.com/file/d/1RLGGXC474w_gqFAQ_AUd6QhxSfyTopPl/view?usp=drive_link Link to Recording] | ||
+ | |||
+ | Topic: Jamie Shanley USGS | ||
+ | |||
+ | Date of meeting: June 6, 2023 | ||
+ | |||
+ | Participants: | ||
+ | |||
+ | FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details | ||
+ | |||
+ | James (Jake) Gearon - IU Bloomington, ESIP Fellow | ||
+ | |||
+ | Martha Apple - Montana Tech | ||
+ | |||
+ | Joseph Bell - USGS | ||
+ | |||
+ | Scotty Strachan - Nevada System of Higher Education, NevadaNet | ||
+ | |||
+ | Daniel Fuka - VTech | ||
+ | |||
+ | James Gallagher - OPeNDAP | ||
+ | |||
+ | Bryce Nance - NEON Aquatic Instrumentation | ||
+ | |||
+ | Sean Kacur - Subsurface Insights | ||
+ | |||
+ | Announcements: | ||
+ | |||
+ | * Preparing for July ESIP meeting | ||
+ | * If you want more ESIP news: <nowiki>https://esipfed.org/subscribe</nowiki> | ||
+ | |||
+ | Time Stamps (ET): | ||
+ | |||
+ | 17:10 - Research watershed in NE Vermont, sleepers watershed | ||
+ | |||
+ | 17:11 - Northern VT, near Quebec | ||
+ | |||
+ | 17:13 - instream is fluorescence, turbidity, CO2, meteorology, snow measurements | ||
+ | |||
+ | 17:14 - “Snow station” sleepers, lot of snow instrumentation. Dont get a deep snow pack, (<1m) make up for it in persistence. 500 m elevation | ||
+ | |||
+ | 17:14 - 111 km^2 do most work in the headwater region, about 100 acres | ||
+ | |||
+ | 17:15 - lot of snowmelt monitoring, DOC measurements as that is mobilized. | ||
+ | |||
+ | 17:15 - absorbance and fluorescence for carbon quality | ||
+ | |||
+ | 17:16 - use it to detect organic carbon molecules (only 2-3% fluoresce, but the measurement is so precise that it is a useful proxy. | ||
+ | |||
+ | 17:17 - fluorometer and turbidity sensor + CO2 sensor. In the water. | ||
+ | |||
+ | 17:18 - FDOM (proxy for DOC from fluorescence) shows that DOC is highly flow dependent. | ||
+ | |||
+ | 17:19 - mirrors discharge quite well (DOC), however nitrate peaks with peak flow, and then relaxes, once per year or so | ||
+ | |||
+ | 17:21 - Turbidity is an SSC proxy | ||
+ | |||
+ | 17:22 - Consistent relationship between % of DOC that is part of SSC | ||
+ | |||
+ | 17:23 - turbidity corrects FDOM signal as the signal is dragged down by high turbidity (sensor is only seeing half of the FDOM) | ||
+ | |||
+ | 17:24 - DOM dissolved organic (material?) is colored dark orange, the question is does it go to the stream | ||
+ | |||
+ | 17:27 - UVM Snow project, Whiteface to Mt. Washington: New York, Vermont, New Hampshire | ||
+ | |||
+ | 17:28 - 22 sites across the network, 3 at sleepers river | ||
+ | |||
+ | 17:29 - cadillac of the operation is the Somers Snowpack Analyzer, uses the dielectric constant to calculate the impedance of the snowpack, determining the amount of ice, air and water in the snow pack which add up to 100%. Pretty sweet. Can go up to 3-4 m plus. Has an angled sensor that integrates over the depth of the snowpack. Can determine when the snowpack might release. The Somers Snow Scale is also used (maybe more familiar with the snow pillow), the scale is only in the very center of a 2.5 m^2 unit. | ||
+ | |||
+ | 17:32 - Water Quality monitoring, Lake Champlain | ||
+ | |||
+ | 17:33 - Buoys are lowering down FDOM and turbidity sensors, plus temp, conductivity. Attempting to track the thermocline | ||
+ | |||
+ | 17:34 - Burlington is on the shores of lake champlain | ||
+ | |||
+ | 17:34 - Question from James about Somers Snow Sensor, sounds like how moisture sensors are sensed. | ||
+ | |||
+ | 17:36 - Martha: could be useful for late into summer ice packs (like in the rockies) | ||
+ | |||
+ | 17:37 - SNO(W)TEL (sp?) lot of universities are lobbying congress to fund a version in the east | ||
+ | |||
+ | 17:40 - Dan asks about neutron probes–for soil moisture, or for any moisture | ||
+ | |||
+ | == May 2, 2023 == | ||
+ | [https://drive.google.com/file/d/1YqSRwpTn_lBhcKVpQ2Tgw_t2o9vW_rLP/view?usp=share_link Link to Recording] | ||
+ | |||
+ | Topic: Athena Clarke USGS | ||
+ | |||
+ | Date of meeting: May 2, 2023 | ||
+ | |||
+ | Participants: | ||
+ | |||
+ | FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details | ||
+ | |||
+ | James (Jake) Gearon - IU Bloomington, ESIP Fellow | ||
+ | |||
+ | Martha Apple - Montana Tech | ||
+ | |||
+ | Joseph Bell - USGS | ||
+ | |||
+ | Scotty Strachan - Nevada System of Higher Education, NevadaNet | ||
+ | |||
+ | David Coyle - USGS | ||
+ | |||
+ | John Porter - UVA & VCR LTER | ||
+ | |||
+ | Announcements: | ||
+ | |||
+ | Time Stamps (ET): | ||
+ | |||
+ | * Registration opens next week for summer meeting in Burlington, VT in July. | ||
+ | * Session proposals: <nowiki>https://esipfed.org/july23-sessions</nowiki> | ||
+ | |||
+ | Notes: | ||
+ | |||
+ | 17:06 - introductions | ||
+ | |||
+ | 17:17 - Presentation begins | ||
+ | |||
+ | Passion project that was developed within the USGS. Will be presenting at association of floodplain managers next week | ||
+ | |||
+ | 17:18 map is alternative way of delivering USGS stream gage data. An integrative approach | ||
+ | |||
+ | 17:19 idea came as an end user of USGS stream gage data. Stage height data is not very intuitive. | ||
+ | |||
+ | 17:21 - gage height in relationship to nearby structures or locations, bridges, parking lots, etc. | ||
+ | |||
+ | 17:22 required to survey them once every 3 years. So would be easy enough to measure elevation up to close. Flood impact locations are shown with icons that indicate flooding. | ||
+ | |||
+ | 17:25: can receive alerts if nearby locations are flooding, can also look thru historical data. | ||
+ | |||
+ | 17:27 Accessible to the end user. | ||
+ | |||
+ | 17:34 Intuitive interface | ||
+ | |||
+ | 17:50 would be useful to get remote sensing data. | ||
+ | |||
+ | == April 4, 2023 == | ||
+ | [https://drive.google.com/file/d/1IHr76_EbMyf8ol8L8MuuawFWvD2dnUSF/view?usp=share_link Link to Recording] | ||
+ | |||
+ | Topic: Dan Fuka + SparkFun | ||
+ | |||
+ | Participants: | ||
+ | |||
+ | FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details | ||
+ | |||
+ | James (Jake) Gearon - IU Bloomington, ESIP Fellow | ||
+ | |||
+ | Martha Apple - Montana Tech | ||
+ | |||
+ | Joseph Bell - USGS | ||
+ | |||
+ | Daniel Fuka - Virginia Tech/Ronin | ||
+ | |||
+ | Andrew Wiedlea - ESnet/LBNL | ||
+ | |||
+ | Scotty Strachan - Nevada System of Higher Education | ||
+ | |||
+ | Jennifer Mullins - SparkFun | ||
+ | |||
+ | Jim Garrison - SparkFun | ||
+ | |||
+ | Dryw Wade - SparkFun | ||
+ | |||
+ | Kade Shelton - USDA ARS AHERU | ||
+ | |||
+ | David Coyle, USGS, Reston, VA | ||
+ | |||
+ | Bhuwan Thapa, Appalachian State, NC | ||
+ | |||
+ | John Porter - University of Virginia | ||
+ | |||
+ | Announcements: | ||
+ | |||
+ | Time Stamps (ET): | ||
+ | |||
+ | * Registration opens next week for summer meeting in Burlington, VT in July. | ||
+ | * Session proposals: <nowiki>https://esipfed.org/july23-sessions</nowiki> | ||
+ | |||
+ | Notes: | ||
+ | |||
+ | Dan Fuka | ||
+ | |||
+ | ARS established partnerships for data innovation in 2019 to shape FAIR habits, partner ship between ag research and private sector tech companies. Microsoft and ESRI joined forces to provide ARS a platform, | ||
+ | |||
+ | ARS provides 90 locations and 137 worksites around USA | ||
+ | |||
+ | USDA-ARS PDI @ Hydraulic Engineering Research Unit (HERU) | ||
+ | |||
+ | $25 Multi Science Station, fun small project for sensing purposes to determine where horses go to the bathroom | ||
+ | |||
+ | Getting into CS screw terminals is a pain | ||
+ | |||
+ | Almost ideal embedded MCU +LoRa low price and saves dev time (saves 16 solder joints) | ||
+ | |||
+ | With NDA Build America, Buy America Act | ||
+ | |||
+ | >60% manufactured in the US. | ||
+ | |||
+ | IoT is great learning tool, several groups have designed IoT based traditional WX stations (3D-PAWS) | ||
+ | |||
+ | Traditional but takes a while to build and install. | ||
+ | |||
+ | Field Techs and Researches have been programming data loggers since the beginning of epoch time!, it seems ok for them. However, mouse driven workflows are a different story. | ||
+ | |||
+ | Kade and Dan then went to SparkFun | ||
+ | |||
+ | Jennifer Mullins: | ||
+ | |||
+ | SparkFun has a good system for a la carte board designs, | ||
+ | |||
+ | SparkFun wants to accelerate prototyping, initiatives, experimenting, etc. | ||
+ | |||
+ | Make technology easily accessible to as many people as possible and strive to find creative solutions to meet users needs. Happy to develop relationships w partners | ||
+ | |||
+ | SparkFun develops original product release once a month, also can do resell which are specifically targeted to support SF initiatives. | ||
+ | |||
+ | Launched a services branch 5 years ago to find creative solutions for customers. | ||
+ | |||
+ | Services: | ||
+ | |||
+ | -Custom Kitting | ||
+ | |||
+ | - Hundreds every year, lot of universities, end use solutions for business partners as well. | ||
+ | |||
+ | -Custom Design (USDA) | ||
+ | |||
+ | -Custom Design, however, constantly busy at this time, but continuing to evolve and expand at this time | ||
+ | |||
+ | -Logistics | ||
+ | |||
+ | -Logistics, will support needs of customers, shipping, inventory management etc. | ||
+ | |||
+ | -Marketing Development | ||
+ | |||
+ | Marketing has been able to support really incredible projects through custom landing pages, creating customized forms, product documentation and tutorials, custom product pages | ||
+ | |||
+ | ALC DESIGN TOOL | ||
+ | |||
+ | -Custom design/ideation | ||
+ | |||
+ | -SparkFun authorized reseller/manufacturer | ||
+ | |||
+ | -US-based manufacturing | ||
+ | |||
+ | ALC provided a jumping off point for engineering. | ||
+ | |||
+ | Design process: | ||
+ | |||
+ | USDA project | ||
+ | |||
+ | -Board development/design with our Engineering team. Constant communication with USDA on spec, adjustments, etc. Final design reviewed and approved to move to prototype then full production. | ||
+ | |||
+ | Production/Manufacturing | ||
+ | |||
+ | Procurement-PCB’s and components are ordered; lead times vary | ||
+ | |||
+ | Scheduling - Build is scheduled | ||
+ | |||
+ | Production - Time depends on complexity and schedule | ||
+ | |||
+ | Testing - we test everything | ||
+ | |||
+ | Inventory & Shipping -we ship globally | ||
+ | |||
+ | The Final Product: | ||
+ | |||
+ | * Dan: | ||
+ | |||
+ | Very simple to interface with SF, had few meetings but very quickly a design was returned. | ||
+ | |||
+ | * Dryw | ||
+ | * Built it, tested, sent it to Kade and Dan for testing, | ||
+ | * Completely open source project | ||
+ | * USDA Watershed Monitor | ||
+ | |||
+ | Dan | ||
+ | |||
+ | * GPS | ||
+ | * Can go almost a year on the battery, some 16k chirps | ||
+ | * Supplemental power through solar | ||
+ | * Looped chirping of gps and WX gives ~12k chirps | ||
+ | * LowPower - 180uA regardless | ||
+ | * How to ID these things? For public IoT based sensing, what about getting a mac address block? | ||
+ | |||
+ | Get hands in the units of ESIPers | ||
+ | |||
+ | Is 10 distributed dataloggers the maximum? Say at 10 min intervals | ||
+ | |||
+ | LoRa can go from tip of SA to canada with a yagi (sp?) antenna | ||
+ | |||
+ | Talk to megan re conference details | ||
+ | |||
+ | How much time will this take? | ||
+ | |||
+ | * Possibility of starting before lunch so that we took lunches out into the field. | ||
+ | * Telemetry, LoRa or cell chip? | ||
+ | * Distributed stations to gateway to internet | ||
+ | * Several stations (5) and at each station there is a different component of research, by the end, they have a sensor chirping out 5 different data streams. | ||
+ | * SparkFun is looking to attend ESIP | ||
+ | |||
+ | Chat: | ||
+ | |||
+ | == March 7, 2023 == | ||
+ | [[Link to Recordihttps://drive.google.com/file/d/1PdLtfx8qWNlSMT9K6UKPwcu4HmE5Y39w/view?usp=share linkng|Link to Recording]] | ||
+ | |||
+ | Topic: Round Table for Summer Session | ||
+ | |||
+ | Participants: | ||
+ | |||
+ | FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details | ||
+ | |||
+ | James (Jake) Gearon - IU Bloomington, ESIP Fellow | ||
+ | |||
+ | Martha Apple - Montana Tech | ||
+ | |||
+ | Joseph Bell - USGS | ||
+ | |||
+ | David Coyle - USGS | ||
+ | |||
+ | Daniel Fuka - Virginia Tech/Ronin | ||
+ | |||
+ | Andrew Wiedlea - LBNL/ESnet | ||
+ | |||
+ | Announcements: | ||
+ | |||
+ | Time Stamps (ET): | ||
+ | |||
+ | 17: - Envirosensing slack channel question | ||
+ | |||
+ | * Raskin Scholarship | ||
+ | * Field trip related activities | ||
+ | * Shriever (sp?) Watershed with in depth temperature records plus others, east side so it would take a little bit to organize a trip out there. Also connecting with folks at a local college that could put on a field trip. Pushing from GS side. Could go from there and launch into ideas. - Joe | ||
+ | |||
+ | 17:12 - not a lot of joint monitoring in vermont. There is monitoring by private industry but the USGS footprint in VT in small. | ||
+ | |||
+ | 17:13 - Jameson from climate change canada. Sentinel lake. | ||
+ | |||
+ | 17:14 - Right next to a wooden glen | ||
+ | |||
+ | 17:16 Megan is going up soon so we can give her questions to ask | ||
+ | |||
+ | 17:17 Joseph says he is willing to help scout the meeting as well | ||
+ | |||
+ | 17:18 Maybe there is potential for two workshops | ||
+ | |||
+ | 17:20 Concept of spanning over the lunch time. | ||
+ | |||
+ | 17:21 What about starting from morning and going through lunch. | ||
+ | |||
+ | 17:22 Last week we talked about a hands on sensor making session where we have arduino kits or something. Would be a good thing to do. | ||
+ | |||
+ | 17:23 - Dan, soldering takes time, design he has makes connecting new sensors easy w/ ethernet. | ||
+ | |||
+ | 17:24 - Could do drones and sensors together, full workflow would be good. So what do we need to go there | ||
+ | |||
+ | * | ||
+ | |||
+ | 17:26 just need proposal with requested timeslot and paragraph - a combined session | ||
+ | |||
+ | 17:27 can call the tower close by and ask them for permission. If they okay it we can fly drones | ||
+ | |||
+ | 17:30 workshop on making sensors, then same or different day get deploy sensors | ||
+ | |||
+ | 17:31 University of VT has LoRa WAN access can see if we have coverage: sensor folks, what would we look to transmit the data over? | ||
+ | |||
+ | 17:32 LoRa 2 LoRa single channel LoRa. We can leave this open. | ||
+ | |||
+ | Bluetooth low energy as a replacement for LoRa. James, just bring a few leaf nodes. How is the way we get data? Joseph can have a LoRa gateway there easily from USGS. The Echo Lake Science center. Classrooms and learning spac could be another resource | ||
+ | |||
+ | We can plan for a specific LoRa , 4 or 5 different frequences that can have uplink. Also open call for folks to bring what they like. Bring an uplink of use ours. We can provide authentication information. Cellular device would you need to check coverage on the nearby towers? – another exciting thing is the CBRS ESNet presentation from andrew wiedlea, but i dont know if we can get a station ready. But would be cool. | ||
+ | |||
+ | Could also use HAM | ||
+ | |||
+ | David Coyle - solid state circuit breakers and other things are becoming more available at moderate power. Could be able to monitor energy of devices. | ||
+ | |||
+ | We need to figure out what we need to provide, kits would be ideal | ||
+ | |||
+ | Suppose we had a workshop but you have a lab fee, popular arduino kit, can pick ones that dont require soldering. Difficulty is you wont get an IP68 case. You could stick those things in the woods. Could also use little salad containers. | ||
+ | |||
+ | Dan - ESP32 could be a good option. Has many useful features. You can strip an ethernet cable and connect it! Also LoRa, Qwiic, BME680, Load Cell. SparkFun Boards could be cool. Whats cost per board? Would need to find funding. | ||
+ | |||
+ | Could take the ferry from Burlington to New York, there are three now according to Joseph. It takes about 45 minutes. | ||
+ | |||
+ | Field Trip - Remote .. maybe | ||
+ | |||
+ | Field Trip - Local .. looking good, centennial woods, Leahy center for lake champlain (ECHO) | ||
+ | |||
+ | Could also extend it into the evening, have a bar evening or the like. | ||
+ | |||
+ | Megan - Ask to visit Centennial Woods and possibly the Leahy Center, also ask about funding costs, we need to ascertain whether we are doing 1 kit/board per person or groups of people. | ||
+ | |||
+ | Workshop fee? Possibility. 30 dollars for the workshop / 100 for total. | ||
+ | |||
+ | Some of the arduinos come with multimeter and have a bunch of gizmos with em. Servo motors etc, which might be cool for the people. | ||
+ | |||
+ | For every 10 units comes a cellular uplink! | ||
+ | |||
+ | Thing worth throwing out there: cellular uplink could push us in a different direction. Cellular point would be useful… | ||
+ | |||
+ | Dan has an ESRI geoserver, if not esri, there are a lot of IoT things. Its REST based webserver. Also has Coords and ESRI | ||
+ | |||
+ | We should consider the accessibility of the situation as well. | ||
+ | |||
+ | By next meeting in April, we should have a kit refined. Ideally we are demoing it in the april meeting. That will be training for the workshop. | ||
+ | |||
+ | Notes: | ||
+ | |||
+ | Chat: | ||
+ | |||
+ | == Feb 7, 2023 == | ||
+ | [https://drive.google.com/file/d/1n9TdKYXSMmpvuryhsJ7JG1y75MW-9E7q/view?usp=share_link Link to Recording] | ||
+ | |||
+ | Topic: Round Table | ||
+ | |||
+ | Participants: | ||
+ | |||
+ | FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details | ||
+ | |||
+ | [[Mailto:drf28@cornell.edu|Daniel Fuka]]| Virginia Tech | ||
+ | |||
+ | James (Jake) Gearon - IU Bloomington, ESIP Fellow | ||
+ | |||
+ | Martha Apple - Montana Tech | ||
+ | |||
+ | Joseph Bell - USGS | ||
+ | |||
+ | John Porter - UVA/VCR LTER | ||
+ | |||
+ | Bryan Hoff- SparkFun | ||
+ | |||
+ | Announcements: | ||
+ | |||
+ | Time Stamps (ET): | ||
+ | |||
+ | 17: - | ||
+ | |||
+ | Notes: | ||
+ | |||
+ | General: | ||
+ | |||
+ | Review of winter meeting | ||
+ | |||
+ | Allison Mills 5:09 PM | ||
+ | |||
+ | Thanks, folks! Here is the IT&I info: <nowiki>https://esipfed.org/ITI</nowiki> | ||
+ | |||
+ | Registration for the Highlights Webinar: <nowiki>https://www.esipfed.org/get-involved/community-calendar</nowiki> | ||
+ | |||
+ | Sched - all session notes, recordings, and resources live here: <nowiki>https://2023januaryesipmeeting.sched.com/</nowiki> | ||
+ | |||
+ | Not a lot of EnviroSensing at winter meeting…. Opportunities for future meetings | ||
+ | |||
+ | Sessions on New satellites (ops for link to ground data) | ||
+ | |||
+ | In Situ data through cloud providers | ||
+ | |||
+ | Session on fires – sensor applications (possibly with new wildfire cluster) | ||
+ | |||
+ | There might be potential session collaborations with other Clusters for the July Meeting (James just named Cloud Computing and Air Quality and also the new Wildfires) | ||
+ | |||
+ | Fusion of in-situ and remotely sensed data is no-brainer especially if it can help in real time calibration. Can improve way satellite data is processed - radiation needs to move through the atmosphere twice to reach satellite. Small number of ground sensors can be spread out. | ||
+ | |||
+ | Allow more depth in field studies. Shift from pulling to pushing - more cost effective sensors. | ||
+ | |||
+ | Data publication: making it accessible to others – big part of IoT | ||
+ | |||
+ | Price is coming down on sensors, can move toward distributed networks. Dan opines on the cost of sensors and economies of scale. Daniel shifting to more costly sensors - see more next month | ||
+ | |||
+ | Switch to where we’re going: | ||
+ | |||
+ | Discussion: | ||
+ | |||
+ | Around the Room: | ||
+ | |||
+ | Joseph - USGS focus on R&D upcoming. Observing Systems Division involved. Training Certificate being discussed for link to Academia (practical, records processing, fouling and calibration drift adjustments). Everything from entry level to partial diff equations. | ||
+ | |||
+ | Q: any travel grants for summer meeting? ESIP will cover travel for 1 or two speakers… but not USGS. | ||
+ | |||
+ | GS wont sponsor travel, but funding for trips sometimes happen. | ||
+ | |||
+ | Dan - had to spend lot of $$$ in 2 weeks, developed sensor w james and martha, slightly more expensive– $166. ESP32 low power capability, LoRa, BME680 (temp, pressure, rel. Humidity, volatile organic carbon), 9DoF (Wind velocity), Loadcell, Weighing lysimeter | ||
+ | |||
+ | Cellular note carrier (lifetime supply of data Qwick Cellular Notecarrier 500MB), goal is 10 or more sensors which can communicate | ||
+ | |||
+ | All US weather stations combined are ~ 1GB (including history) | ||
+ | |||
+ | USDA funded development of sensor via SparkFun, fed agencies cant buy chinese parts (rip) | ||
+ | |||
+ | ESIP Summer meeting ideas: potential workshop outside, putting together session / tutorial. Possibly a multi-day event. Couple hours plus a lunch w/ workshop afterwards. Outdoor + drones group would be great. Flying around collecting data | ||
+ | |||
+ | Bryan, SparkFun engineering manager. Custom designed unit but started with building blocks–full custom board solution for Dan. Background in implementing wifi connection + bluetooth in small devices like printers. SparkFun is open source and open hardware with a focused on learning/teaching. | ||
+ | |||
+ | All local manufacturing and sourcing within SparkFun | ||
+ | |||
+ | John - Wrestling LoRa, has LoRa gateway set up on eastern shore, want to see what kind of antennas he would need to communicate ~10-14km. Wants to do experimenting. Has GPS unit on a boat to determine where signal will be. Directional antennas also an option. Open to suggestions, thinking about ChirpStack, where he is, LoRaWAN may not be as critical. | ||
+ | |||
+ | James - mentioned LoRa mesh protocols as possible solution | ||
+ | |||
+ | John - big issue, lagoons are not good locations for solar panels, must be inconspicuous, plus storms and wave action. Primary battery may be key since solar not good in lagoon. | ||
+ | |||
+ | James - also considering bluetooth, considering study based on different frequencies. | ||
+ | |||
+ | Dan- recommend starting simpler, back to base LoRa instead of LoRa WAN. probably double the distance you can reach because you dont need acknowledgement needed from LoRa WAN, also easier to program. | ||
+ | |||
+ | Martha - DIRT project, digital integrated research technology. Demonstrated need for an ability to get sensor data from inaccessible field sites to the user. As well as unobtrusive sensors! | ||
+ | |||
+ | Another project - TOMST soil moisture and temperature sensors at a mine waste area. Looking to see if metals in soil are correlated with plant spacing patterns. | ||
+ | |||
+ | Joe - used tile markers, has lots of ideas re salt marshes. | ||
+ | |||
+ | Jake- working on proposals! Remote sensing, ISAT spaceborne altimetry | ||
+ | |||
+ | == 12/06/22 == | ||
+ | Topic: Chris Gazoorian, USGS | ||
+ | |||
+ | Date of meeting: Dec. 6, 2022 | ||
+ | [https://drive.google.com/file/d/1GMdbRBcWGgr9uBd6GmFjKkMwlDTT43rQ/view?usp=share_link Link to Recording] | ||
+ | Participants: | ||
+ | |||
+ | FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details | ||
+ | |||
+ | Scotty Strachan - Nevada System of Higher Education | System Computing Services | ||
+ | |||
+ | James (Jake) Gearon - IU Bloomington, ESIP Fellow | ||
+ | |||
+ | Martha Apple - Montana Tech | ||
+ | |||
+ | Joseph Bell - USGS | ||
+ | |||
+ | Allison Mills - ESIP | ||
+ | |||
+ | Chris Gozoorian - USGS | ||
+ | |||
+ | Andrew Wiedlea - ESNet | ||
+ | |||
+ | David Coyle - USGS | ||
+ | |||
+ | Gabrielle Boisrame - DRI | ||
+ | |||
+ | Mark Swinton - RPI | ||
+ | |||
+ | Nora Catolico - NEON | ||
+ | |||
+ | Xiongwen Chen | ||
+ | |||
+ | Announcements: | ||
+ | |||
+ | Time Stamps (ET): | ||
+ | |||
+ | 17:12 - | ||
+ | |||
+ | Notes: | ||
+ | |||
+ | General: ESIP at AGU this year, Ignite and Data Help Desk | ||
+ | |||
+ | Chris Gazoorian presentation | ||
+ | |||
+ | * Upper Neversink Innovation Test Bed, NY upstate | ||
+ | * All New York City water supply | ||
+ | * Legacy monitoring: traditional stream monitoring + a few met stations + one ET station + deposition collection site | ||
+ | * large suite of envirosensing devices and applications to figure out which works best | ||
+ | * Basically anything you could ever want re: enviro data. Monitored as it supplies water. | ||
+ | * Fish population research dovetails with hydrology, especially as local environment is spring fed | ||
+ | * Paired water and air measurements at-a-station, 51 sites with HOBO TidBit sensors where radon and depth to bedrock is also collected. | ||
+ | * Helps to identify how much the water responds to the local air temperatures. | ||
+ | * Upper pigeon brook subbasin, many ground water springs destroy the cohesion of the granular sediment and cause debris flows. | ||
+ | * Aquifer research in the neversink is a large component of the groundwater storage in the basin, trying to figure out the total volume/ | ||
+ | * Trail cameras check for stream ephemerality. | ||
+ | * Water-air temperature sensitivity changes down the catchment | ||
+ | * ML model for depth to bedrock, then can observe where there are large disparities in temperature equilibration timescales | ||
+ | * Monitoring for radon to determine groundwater residence times. | ||
+ | * O18 isotopes were also collected, show a highly seasonal dependence. The autumn isotopic signature is distinct from the spring signature | ||
+ | * HVSR tremino sensors, require shear velocities for analysis of depth to bedrock | ||
+ | * Thermal imagery can help attribute water sources by looking for colder groundwater seeping into active channels | ||
+ | * Site recon with FO-DTS to map subsurface seepages. Can also use handheld thermal cameras. | ||
+ | * Cables can be 1km long for FO-DTS, looking for preferential groundwater recharge zones, which can also be directly monitored. | ||
+ | * Gauges run off of AC power, though some of the more remote sensors are run by solar, use high gain antennas as there is no cell service | ||
+ | * UAS orthoimagery and thermal imagery | ||
+ | * Fixed mount thermal went through some iterations, settled on FLIR A700, can stream the data with different protocols, allowing for customization. | ||
+ | * Also have real time RGB imagery | ||
+ | * Appears to be differntial groundwater dynamics between the alluvial floodplain and the hillslope seeps. | ||
+ | * Rivers, if unconnected to groundwater, are quite more variable in temperature | ||
+ | * Ideally scale this to the national level for groundwater inventory | ||
+ | * <nowiki>https://www.usgs.gov/apps/ecosheds/fpe/#/</nowiki> trailcam water flow explorer software application | ||
+ | |||
+ | Discussion: | ||
+ | |||
+ | Chat: | ||
+ | 00:20:22 Joseph: NGWOS = The Next Generation Water Observing System | ||
+ | 00:57:26 Scotty Strachan: if anyone has a question, feel free to "raise hand" in Zoom | ||
+ | |||
+ | 01:05:16 Chris Gazoorian USGS: <nowiki>https://www.usgs.gov/apps/ecosheds/fpe/#/</nowiki> | ||
+ | |||
+ | 01:10:28 Joseph: <nowiki>https://water.usgs.gov/fisp/</nowiki> | ||
+ | |||
+ | == 11/01/2022 == | ||
+ | Topic: Mahta Moghaddam | ||
+ | |||
+ | [https://drive.google.com/file/d/1y5O6MgXLznIEP_DW1PYRMMYa9wBl7jNP/view?usp=share_link Link to Recording] | ||
+ | |||
+ | FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details | ||
+ | |||
+ | Scotty Strachan - Nevada System of Higher Education | System Computing Services | ||
+ | |||
+ | James (Jake) Gearon - IU Bloomington, ESIP Fellow | ||
+ | |||
+ | Martha Apple - Montana Tech | ||
+ | |||
+ | Joseph Bell - USGS | ||
+ | |||
+ | Allison Mills - ESIP | ||
+ | |||
+ | John Porter, UVA VCR/LTER | ||
+ | |||
+ | Todd Caldwell - USGS | ||
+ | |||
+ | Mahta Moghaddam - USC | ||
+ | |||
+ | Gabrielle Boisrame - DRI | ||
+ | |||
+ | Xiongwen Chen, Alabama A & M University | ||
+ | |||
+ | Stijn Wielandt, Lawrence Berkeley National Laboratory | ||
+ | |||
+ | Bryce Nance - National Ecological Observatory Network (NEON) / Battelle | ||
+ | |||
+ | Announcements: | ||
+ | |||
+ | * LAST DAY to submit! Ignite at AGU coming up- sci comm oriented talks: <nowiki>https://igniteagu.io/</nowiki> | ||
+ | |||
+ | * ESIP: Subscribe to ESIP Update: sign up! esipfed.org/subscribe | ||
+ | * ESIP Meeting Session due next week - Fri Nov 11 esipfed.org/meetings | ||
+ | * Envirosensing session at summer 2023 ESIP meeting? | ||
+ | ** Trying to gauge interest | ||
+ | |||
+ | Time Stamps (ET): | ||
+ | |||
+ | 17:12 - Start of presentation: Mahta Moghadam | ||
+ | |||
+ | Notes: | ||
+ | |||
+ | * Remote sensing of water with microwaves, it is primarily sensitive to the water signal. Ultimate strategy must use all types of remote sensing, including in-situ sensor networks. Want to sense water at a variety of scales. | ||
+ | * At low frequency, you can get dielectric profiles from subsurface!!! | ||
+ | * Led to design of NASA AirMOSS radar, P band frequency was chosen to ensure soil water content below veg and surface | ||
+ | * Collected data from all across the US, including 9 different Biomes | ||
+ | * Can get down to -75 cm into the subsurface… | ||
+ | * Inversion algorithm finds best set of scattering model parameters… | ||
+ | * With 1000 flight hours, errors are reasonable with max error at 0.45 m | ||
+ | * What happens with permafrost and P-band radar? | ||
+ | * No direct active layer property measurements | ||
+ | * P-band radar can resolve the depth to active layer remotely!!!! | ||
+ | ** Can generate time series with this methodology | ||
+ | * Many variables that go into active layer occurrence over time, need more data to confirm. | ||
+ | * UAS may be able to gap fill more expensive airborne acquisitions. | ||
+ | * Software defined radar uses a low cost hardware platform that can be remotely configured for operating frequency, bandwidth, imaging mode, and power | ||
+ | * Good estimation of water-table depths down to 12 FEET! | ||
+ | * Can identify the snow surface and ground surface! | ||
+ | * Trees make things harder | ||
+ | * Custom hardware and software for WSN and UAV operations | ||
+ | * Everything is autonomous re: the drone. | ||
+ | * Good SNR below trees, can convert to soil moisture | ||
+ | * UAS-based systems will be able to gap-fill larger more $$ datasets | ||
+ | * Volume scattering at higher frequencies, even at P-band, we are sensitive to presence of roots. Larger rocks in the subsurface | ||
+ | * NEON flies sites every 3 years lidar, curious about bathymetry applications? | ||
+ | * Hoping to start a project with this drone based system, need lower frequencies. How to separate reflections from the top and bottom. | ||
+ | * Scotty: cm scale environment at plot or landscape for surface characteristics as well as moisture constant. Can you get improve accuracy with the system for cm-scale characterization? | ||
+ | ** A: a qualified yes: cm-scale or sub-cm surface roughness, probably a yes. They have the EM wave roughness scale as a variable. We assume somewhere between .5 cm to cm roughness. Cm-scale resolution is probably not possible with current limit of bandwidth. However, the next best thing is using an analytical profile model. This makes it more possible | ||
+ | * | ||
+ | |||
+ | Discussion: | ||
+ | |||
+ | Chat: | ||
+ | |||
+ | 00:17:21 Room 2: If you’re just joining us, here is the sign in sheet! <nowiki>https://docs.google.com/document/d/12QVf-WwBnoW1mlADsoa6CZ_hkXzaFozR1TJ3oW-PCi0/edit</nowiki> | ||
+ | |||
+ | 00:18:06 Room 2: If you’d like to sign up for the list serve, here is the link! <nowiki>https://lists.esipfed.org/mailman/listinfo/esip-envirosensing</nowiki> | ||
+ | |||
+ | 00:23:26 Allison Mills | ESIP: Welcome, Gabrielle! | ||
+ | |||
+ | 00:24:09 Allison Mills | ESIP: Welcome to you as well, Xiongwen! | ||
+ | |||
+ | 00:49:38 Room 2: Wow! | ||
+ | |||
+ | 00:50:04 Room 2: Consider ICESAT-2 for ground surface validation, it can get bare earth transects in vegetated terrain | ||
+ | |||
+ | 01:14:19 James Gallagher: Thanks! Great Talk. | ||
+ | |||
+ | 01:14:42 Mahta Moghaddam - USC: Thank you very much for letting me speak at this wonderful forum! | ||
== 10/04/2022 == | == 10/04/2022 == |
Latest revision as of 10:47, November 1, 2023
back to EnviroSensing Cluster main page
Telecons are on the first Monday of every month. Click on 'join'
Schedule
- Next telecon: June 1, 2021 at 5:00pm EST
Notes and recordings from past telecons
To listen to recordings from past telecons, click here. To log in, use the username guest@esipfed.org, and the password Earth111.
September 5, 2023
Topic: Summer Recap Cluster Strategic Planning
Date of meeting: Sep 5, 2023
Participants:
FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details
James (Jake) Gearon - IU Bloomington, ESIP Fellow
Joseph Bell - USGS
Scotty Strachan - Nevada System of Higher Education, NevadaNet
John Porter - UVA
Daniel Fuka - Virginia Tech
David Coyle - USGS
Megan Carter - ESIP
James Gallagher
AGENDA:
Joseph has equipment to be sent out. Some minutes directed for virtual workshop
Virtual Workshop Take 2
Stated Desires:
Some of the equipment makes it back to Scotty
USDA Equipment that goes offline will likely need to be returned to USDA
Number of Kits x Number of Cellular uplinks y
2 - John Porter - Grad Student?
2 - James/Martha
2 - Jake
2 - Joseph
2 - Renee Brown
https://edirepository.org/
https://knb.ecoinformatics.org/
https://developers.planet.com/docs/planetschool/introduction-to-stac-part-1-an-overview-of-the-specification/
https://www.dataone.org/network/
Ezeml
CUASI
Announcements:
Time Stamps (ET):
17:06 - introductions, etc. coming off of the meeting, we have a chance to restructure. Martha Apple is transitioning out as is Scotty. Sent invites to some folks to discuss co chairing. Idea of hands-on sensor building, cluster focused toward that end (interactive building).
17:08 - summer meeting recap. Even though it was a back up it went very well, was useful for figuring out the scale of tinkering that is possible given the time constraints.
17:11 - John P. more show and tell than planned. Good show and tell, though! Good range of people, a relative novice to experienced users. A question to throw out. What is the audience were trying to reach and what do we want them to learn?
17:12 - Joe, looking ahead, have use cases etc. We can have a suite of parameters (temp, pressure, etc) HydroTech, Grad Student, Researcher, environmental consultant. When i collect this data, how am i collecting this data to share? Use cases Joe would seek are: good data quality, collection, and delivery. Do we get this out through a GS database or something like coords.
17:14 - James G. What do we think of as graduate students and their capabilities? Where do they start? Have they had python experience? Only informal?
17:16 - Dan Fuka, are we not trying to hit a slightly lower layer on the technology stack? Actually getting or creating or having a device they can work through to familiarize themselves with data flow. I.e. how do you pass info between devices, etc. Thats the hardest layer to get people started themselves. As we know, each of these layers is fairly complex, programming the arduino etc, using amazon cloud services or other data repositories. How do we get value off of a specific sensor? A workshop towards that initial introduction, not focused on a new set of people, but those focused on ESIP returners and the faithful. Allows folks to start on the same page (those that attend the meetings with frequency)
17:18 - Scotty: there is the "intro" crowd, i.e., people that know something about using software and basic scripting/programming....and then there is a "technical" crowd that has some combination of DC electronics, mechanical/fabrication, database, software backend, wireless, etc. experience
17:19 - John P. many permutations of sensor data/communication. Lot of options out there for networking, what is great is to see some working solutions and say “hey thats like my problem”. John took a course that had standardized hardware etc, was all planned out. Towards the notion of training within ourselves. Having some sessions where folks go into some depth about their own solutions or works in progress. Could be more of an office hours.
17:22 - Joe: needs for use cases. The organization chooses a single source/product is great. This is where to build off the prev session. Using the USDA board. The winter meeting could be use cases, talks, urban vs forest etc. Each session would have a focus of honing the use case, and working with the core group.
17:25 - John P. would love to play around with daniels board. Important to have one that works and is accessible to people. But its easier to build with success, aka using the usda board.
17:25 - As scotty steps back, cluster has the opportunity to move forward etc. as such, we need to discuss the audience. There is the internal audience (highly skilled, technical people) then there is the external audience (all others, stakeholders etc). They have questions regarding using/deploying sensors. Do we do a demonstration with those external audience members. Can we touch both audiences?
17:30 - Megan C. you have to start with who is showing up now, what do they need. Who is not here that we would like to see here. The faithful etc. But there is the grad student/general ESIP population. Could work across committees.
17:32 Joe, what does the cluster provide now? Site selection, planning data routing, the specific use cases. The grad student that Joe thinks of is one who is focused on sensors etc. maybe we can be a place. Simple build experiment and provide the instruction. And where do I send my data?
17:34 Dan Fuka: wrapping thoughts together, maybe we need some grad students to pull into this.
17:36 - James G: don’t really think the problems associated with the boards and the sensors are not unique to what were looking at. There are 1000s of links associated with that sort of stuff. We have always looked at the big picture. There is another technical problem though, you have sensors in the field, how do you get those data out of the sensors and into a place that is visible? This is the part of the use case where we can go where others cant go.
17:39 - Joe: Q to James, what would we then orient to? 2 ways: go to device and get the data (doesnt really address the problem) Coords and thingspeak/ iot things might be more amenable. Data management, provenance, archiving, etc. are there repositories for those info? Becomes a complicated problem. As the sensor networks we deploy grow larger, the data processing and managing gets more difficult.
17:41 - Dan Fuka - after our group has common knowledge we can utilize, then we involve others (other esip clusters etc). Lets get our problem to the point where the data is transmitting. Then we can decide on how to move forward. Maybe DataVerse?
17:42 - Scotty: Thinking out loud here......gonna be difficult to address the full spectrum of problems across the workflow....in a monthly 1-hr cluster....the 2014 assembly of best practices was high-level and it still took forever.....
You may be more successful with some deliverables by really being focused on 1-2 parts of the workflow....and saving the wider discussion for an annual event?
17:43 - James G. responding to Dan, not sure we have the time for that. May have to accept its gonna be messy and go forward on all fronts.
17:44 - Joe, what if we took an more “ESIP” view. We can to connect with this cluster (hazards life cycle) set that as the scope for the sensor we build. Could let them choose the data handoff. That approach may provide a filter we need.
17:47 - Scotty: Joseph needs to recruit folks to put time in etc.
17:50 - Megan: this makes a strong case for a broader event (annual event) sticky spots etc. talking more broadly, where were going to focus in the near term, but starting with a broader enticement to ESIP.
17:52 - Joe, we have 2 different vectors.
17:53 - formalizing a build, then purchasing the build and what practices etc.
17:54 - Dan Fuka, there are methods out there for how to sense many many things and get the data, start collecting, etc. how to get it into a few repositories. Predominantly commercial. The only problem I have is how to get that into an open data repository. Have had problems with this for years. Where do we put the data? How do we get it to the world?
17:55 Megan I need to go now. Sounds like there are a lot of good ideas. Just remember everything will take longer than you think in terms of planning. Be kind to yourself in setting milestones. Make sure to build in what you each need to get out of this group and the ability to pivot and make adjustments along the way.
17:56 - John P. there are solutions (lightweight ones) in the ecological space. EDI (https://edirepository.org/) is where LTER is sending its data these days. But they are open to others, DataOne is similar and related. Etc. https://knb.ecoinformatics.org/
17:59 - this is clearly a problem and some people are clearly addressing it. The use case is connecting up the sensors to cloud data archives. Come prepared to talk on these topics.
18:01 - Here is the list of member nodes for dataone. Some of these might work (it includes KNB and EDI): https://www.dataone.org/network/
Do we have people come and talk about that?
18:03 - See also: Globus: https://docs.globus.org/api/ (USGS has a license) - David Coyle
18:03 - EasyEML
18:04 - STAC is cool but complex.
18:05 - Globus is an interesting conversation....started out as a transfer tool....
James g. We want to know what is good enough,
June 6, 2023
Topic: Jamie Shanley USGS
Date of meeting: June 6, 2023
Participants:
FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details
James (Jake) Gearon - IU Bloomington, ESIP Fellow
Martha Apple - Montana Tech
Joseph Bell - USGS
Scotty Strachan - Nevada System of Higher Education, NevadaNet
Daniel Fuka - VTech
James Gallagher - OPeNDAP
Bryce Nance - NEON Aquatic Instrumentation
Sean Kacur - Subsurface Insights
Announcements:
- Preparing for July ESIP meeting
- If you want more ESIP news: https://esipfed.org/subscribe
Time Stamps (ET):
17:10 - Research watershed in NE Vermont, sleepers watershed
17:11 - Northern VT, near Quebec
17:13 - instream is fluorescence, turbidity, CO2, meteorology, snow measurements
17:14 - “Snow station” sleepers, lot of snow instrumentation. Dont get a deep snow pack, (<1m) make up for it in persistence. 500 m elevation
17:14 - 111 km^2 do most work in the headwater region, about 100 acres
17:15 - lot of snowmelt monitoring, DOC measurements as that is mobilized.
17:15 - absorbance and fluorescence for carbon quality
17:16 - use it to detect organic carbon molecules (only 2-3% fluoresce, but the measurement is so precise that it is a useful proxy.
17:17 - fluorometer and turbidity sensor + CO2 sensor. In the water.
17:18 - FDOM (proxy for DOC from fluorescence) shows that DOC is highly flow dependent.
17:19 - mirrors discharge quite well (DOC), however nitrate peaks with peak flow, and then relaxes, once per year or so
17:21 - Turbidity is an SSC proxy
17:22 - Consistent relationship between % of DOC that is part of SSC
17:23 - turbidity corrects FDOM signal as the signal is dragged down by high turbidity (sensor is only seeing half of the FDOM)
17:24 - DOM dissolved organic (material?) is colored dark orange, the question is does it go to the stream
17:27 - UVM Snow project, Whiteface to Mt. Washington: New York, Vermont, New Hampshire
17:28 - 22 sites across the network, 3 at sleepers river
17:29 - cadillac of the operation is the Somers Snowpack Analyzer, uses the dielectric constant to calculate the impedance of the snowpack, determining the amount of ice, air and water in the snow pack which add up to 100%. Pretty sweet. Can go up to 3-4 m plus. Has an angled sensor that integrates over the depth of the snowpack. Can determine when the snowpack might release. The Somers Snow Scale is also used (maybe more familiar with the snow pillow), the scale is only in the very center of a 2.5 m^2 unit.
17:32 - Water Quality monitoring, Lake Champlain
17:33 - Buoys are lowering down FDOM and turbidity sensors, plus temp, conductivity. Attempting to track the thermocline
17:34 - Burlington is on the shores of lake champlain
17:34 - Question from James about Somers Snow Sensor, sounds like how moisture sensors are sensed.
17:36 - Martha: could be useful for late into summer ice packs (like in the rockies)
17:37 - SNO(W)TEL (sp?) lot of universities are lobbying congress to fund a version in the east
17:40 - Dan asks about neutron probes–for soil moisture, or for any moisture
May 2, 2023
Topic: Athena Clarke USGS
Date of meeting: May 2, 2023
Participants:
FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details
James (Jake) Gearon - IU Bloomington, ESIP Fellow
Martha Apple - Montana Tech
Joseph Bell - USGS
Scotty Strachan - Nevada System of Higher Education, NevadaNet
David Coyle - USGS
John Porter - UVA & VCR LTER
Announcements:
Time Stamps (ET):
- Registration opens next week for summer meeting in Burlington, VT in July.
- Session proposals: https://esipfed.org/july23-sessions
Notes:
17:06 - introductions
17:17 - Presentation begins
Passion project that was developed within the USGS. Will be presenting at association of floodplain managers next week
17:18 map is alternative way of delivering USGS stream gage data. An integrative approach
17:19 idea came as an end user of USGS stream gage data. Stage height data is not very intuitive.
17:21 - gage height in relationship to nearby structures or locations, bridges, parking lots, etc.
17:22 required to survey them once every 3 years. So would be easy enough to measure elevation up to close. Flood impact locations are shown with icons that indicate flooding.
17:25: can receive alerts if nearby locations are flooding, can also look thru historical data.
17:27 Accessible to the end user.
17:34 Intuitive interface
17:50 would be useful to get remote sensing data.
April 4, 2023
Topic: Dan Fuka + SparkFun
Participants:
FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details
James (Jake) Gearon - IU Bloomington, ESIP Fellow
Martha Apple - Montana Tech
Joseph Bell - USGS
Daniel Fuka - Virginia Tech/Ronin
Andrew Wiedlea - ESnet/LBNL
Scotty Strachan - Nevada System of Higher Education
Jennifer Mullins - SparkFun
Jim Garrison - SparkFun
Dryw Wade - SparkFun
Kade Shelton - USDA ARS AHERU
David Coyle, USGS, Reston, VA
Bhuwan Thapa, Appalachian State, NC
John Porter - University of Virginia
Announcements:
Time Stamps (ET):
- Registration opens next week for summer meeting in Burlington, VT in July.
- Session proposals: https://esipfed.org/july23-sessions
Notes:
Dan Fuka
ARS established partnerships for data innovation in 2019 to shape FAIR habits, partner ship between ag research and private sector tech companies. Microsoft and ESRI joined forces to provide ARS a platform,
ARS provides 90 locations and 137 worksites around USA
USDA-ARS PDI @ Hydraulic Engineering Research Unit (HERU)
$25 Multi Science Station, fun small project for sensing purposes to determine where horses go to the bathroom
Getting into CS screw terminals is a pain
Almost ideal embedded MCU +LoRa low price and saves dev time (saves 16 solder joints)
With NDA Build America, Buy America Act
>60% manufactured in the US.
IoT is great learning tool, several groups have designed IoT based traditional WX stations (3D-PAWS)
Traditional but takes a while to build and install.
Field Techs and Researches have been programming data loggers since the beginning of epoch time!, it seems ok for them. However, mouse driven workflows are a different story.
Kade and Dan then went to SparkFun
Jennifer Mullins:
SparkFun has a good system for a la carte board designs,
SparkFun wants to accelerate prototyping, initiatives, experimenting, etc.
Make technology easily accessible to as many people as possible and strive to find creative solutions to meet users needs. Happy to develop relationships w partners
SparkFun develops original product release once a month, also can do resell which are specifically targeted to support SF initiatives.
Launched a services branch 5 years ago to find creative solutions for customers.
Services:
-Custom Kitting
- Hundreds every year, lot of universities, end use solutions for business partners as well.
-Custom Design (USDA)
-Custom Design, however, constantly busy at this time, but continuing to evolve and expand at this time
-Logistics
-Logistics, will support needs of customers, shipping, inventory management etc.
-Marketing Development
Marketing has been able to support really incredible projects through custom landing pages, creating customized forms, product documentation and tutorials, custom product pages
ALC DESIGN TOOL
-Custom design/ideation
-SparkFun authorized reseller/manufacturer
-US-based manufacturing
ALC provided a jumping off point for engineering.
Design process:
USDA project
-Board development/design with our Engineering team. Constant communication with USDA on spec, adjustments, etc. Final design reviewed and approved to move to prototype then full production.
Production/Manufacturing
Procurement-PCB’s and components are ordered; lead times vary
Scheduling - Build is scheduled
Production - Time depends on complexity and schedule
Testing - we test everything
Inventory & Shipping -we ship globally
The Final Product:
- Dan:
Very simple to interface with SF, had few meetings but very quickly a design was returned.
- Dryw
- Built it, tested, sent it to Kade and Dan for testing,
- Completely open source project
- USDA Watershed Monitor
Dan
- GPS
- Can go almost a year on the battery, some 16k chirps
- Supplemental power through solar
- Looped chirping of gps and WX gives ~12k chirps
- LowPower - 180uA regardless
- How to ID these things? For public IoT based sensing, what about getting a mac address block?
Get hands in the units of ESIPers
Is 10 distributed dataloggers the maximum? Say at 10 min intervals
LoRa can go from tip of SA to canada with a yagi (sp?) antenna
Talk to megan re conference details
How much time will this take?
- Possibility of starting before lunch so that we took lunches out into the field.
- Telemetry, LoRa or cell chip?
- Distributed stations to gateway to internet
- Several stations (5) and at each station there is a different component of research, by the end, they have a sensor chirping out 5 different data streams.
- SparkFun is looking to attend ESIP
Chat:
March 7, 2023
Topic: Round Table for Summer Session
Participants:
FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details
James (Jake) Gearon - IU Bloomington, ESIP Fellow
Martha Apple - Montana Tech
Joseph Bell - USGS
David Coyle - USGS
Daniel Fuka - Virginia Tech/Ronin
Andrew Wiedlea - LBNL/ESnet
Announcements:
Time Stamps (ET):
17: - Envirosensing slack channel question
- Raskin Scholarship
- Field trip related activities
- Shriever (sp?) Watershed with in depth temperature records plus others, east side so it would take a little bit to organize a trip out there. Also connecting with folks at a local college that could put on a field trip. Pushing from GS side. Could go from there and launch into ideas. - Joe
17:12 - not a lot of joint monitoring in vermont. There is monitoring by private industry but the USGS footprint in VT in small.
17:13 - Jameson from climate change canada. Sentinel lake.
17:14 - Right next to a wooden glen
17:16 Megan is going up soon so we can give her questions to ask
17:17 Joseph says he is willing to help scout the meeting as well
17:18 Maybe there is potential for two workshops
17:20 Concept of spanning over the lunch time.
17:21 What about starting from morning and going through lunch.
17:22 Last week we talked about a hands on sensor making session where we have arduino kits or something. Would be a good thing to do.
17:23 - Dan, soldering takes time, design he has makes connecting new sensors easy w/ ethernet.
17:24 - Could do drones and sensors together, full workflow would be good. So what do we need to go there
17:26 just need proposal with requested timeslot and paragraph - a combined session
17:27 can call the tower close by and ask them for permission. If they okay it we can fly drones
17:30 workshop on making sensors, then same or different day get deploy sensors
17:31 University of VT has LoRa WAN access can see if we have coverage: sensor folks, what would we look to transmit the data over?
17:32 LoRa 2 LoRa single channel LoRa. We can leave this open.
Bluetooth low energy as a replacement for LoRa. James, just bring a few leaf nodes. How is the way we get data? Joseph can have a LoRa gateway there easily from USGS. The Echo Lake Science center. Classrooms and learning spac could be another resource
We can plan for a specific LoRa , 4 or 5 different frequences that can have uplink. Also open call for folks to bring what they like. Bring an uplink of use ours. We can provide authentication information. Cellular device would you need to check coverage on the nearby towers? – another exciting thing is the CBRS ESNet presentation from andrew wiedlea, but i dont know if we can get a station ready. But would be cool.
Could also use HAM
David Coyle - solid state circuit breakers and other things are becoming more available at moderate power. Could be able to monitor energy of devices.
We need to figure out what we need to provide, kits would be ideal
Suppose we had a workshop but you have a lab fee, popular arduino kit, can pick ones that dont require soldering. Difficulty is you wont get an IP68 case. You could stick those things in the woods. Could also use little salad containers.
Dan - ESP32 could be a good option. Has many useful features. You can strip an ethernet cable and connect it! Also LoRa, Qwiic, BME680, Load Cell. SparkFun Boards could be cool. Whats cost per board? Would need to find funding.
Could take the ferry from Burlington to New York, there are three now according to Joseph. It takes about 45 minutes.
Field Trip - Remote .. maybe
Field Trip - Local .. looking good, centennial woods, Leahy center for lake champlain (ECHO)
Could also extend it into the evening, have a bar evening or the like.
Megan - Ask to visit Centennial Woods and possibly the Leahy Center, also ask about funding costs, we need to ascertain whether we are doing 1 kit/board per person or groups of people.
Workshop fee? Possibility. 30 dollars for the workshop / 100 for total.
Some of the arduinos come with multimeter and have a bunch of gizmos with em. Servo motors etc, which might be cool for the people.
For every 10 units comes a cellular uplink!
Thing worth throwing out there: cellular uplink could push us in a different direction. Cellular point would be useful…
Dan has an ESRI geoserver, if not esri, there are a lot of IoT things. Its REST based webserver. Also has Coords and ESRI
We should consider the accessibility of the situation as well.
By next meeting in April, we should have a kit refined. Ideally we are demoing it in the april meeting. That will be training for the workshop.
Notes:
Chat:
Feb 7, 2023
Topic: Round Table
Participants:
FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details
[Fuka]| Virginia Tech
James (Jake) Gearon - IU Bloomington, ESIP Fellow
Martha Apple - Montana Tech
Joseph Bell - USGS
John Porter - UVA/VCR LTER
Bryan Hoff- SparkFun
Announcements:
Time Stamps (ET):
17: -
Notes:
General:
Review of winter meeting
Allison Mills 5:09 PM
Thanks, folks! Here is the IT&I info: https://esipfed.org/ITI
Registration for the Highlights Webinar: https://www.esipfed.org/get-involved/community-calendar
Sched - all session notes, recordings, and resources live here: https://2023januaryesipmeeting.sched.com/
Not a lot of EnviroSensing at winter meeting…. Opportunities for future meetings
Sessions on New satellites (ops for link to ground data)
In Situ data through cloud providers
Session on fires – sensor applications (possibly with new wildfire cluster)
There might be potential session collaborations with other Clusters for the July Meeting (James just named Cloud Computing and Air Quality and also the new Wildfires)
Fusion of in-situ and remotely sensed data is no-brainer especially if it can help in real time calibration. Can improve way satellite data is processed - radiation needs to move through the atmosphere twice to reach satellite. Small number of ground sensors can be spread out.
Allow more depth in field studies. Shift from pulling to pushing - more cost effective sensors.
Data publication: making it accessible to others – big part of IoT
Price is coming down on sensors, can move toward distributed networks. Dan opines on the cost of sensors and economies of scale. Daniel shifting to more costly sensors - see more next month
Switch to where we’re going:
Discussion:
Around the Room:
Joseph - USGS focus on R&D upcoming. Observing Systems Division involved. Training Certificate being discussed for link to Academia (practical, records processing, fouling and calibration drift adjustments). Everything from entry level to partial diff equations.
Q: any travel grants for summer meeting? ESIP will cover travel for 1 or two speakers… but not USGS.
GS wont sponsor travel, but funding for trips sometimes happen.
Dan - had to spend lot of $$$ in 2 weeks, developed sensor w james and martha, slightly more expensive– $166. ESP32 low power capability, LoRa, BME680 (temp, pressure, rel. Humidity, volatile organic carbon), 9DoF (Wind velocity), Loadcell, Weighing lysimeter
Cellular note carrier (lifetime supply of data Qwick Cellular Notecarrier 500MB), goal is 10 or more sensors which can communicate
All US weather stations combined are ~ 1GB (including history)
USDA funded development of sensor via SparkFun, fed agencies cant buy chinese parts (rip)
ESIP Summer meeting ideas: potential workshop outside, putting together session / tutorial. Possibly a multi-day event. Couple hours plus a lunch w/ workshop afterwards. Outdoor + drones group would be great. Flying around collecting data
Bryan, SparkFun engineering manager. Custom designed unit but started with building blocks–full custom board solution for Dan. Background in implementing wifi connection + bluetooth in small devices like printers. SparkFun is open source and open hardware with a focused on learning/teaching.
All local manufacturing and sourcing within SparkFun
John - Wrestling LoRa, has LoRa gateway set up on eastern shore, want to see what kind of antennas he would need to communicate ~10-14km. Wants to do experimenting. Has GPS unit on a boat to determine where signal will be. Directional antennas also an option. Open to suggestions, thinking about ChirpStack, where he is, LoRaWAN may not be as critical.
James - mentioned LoRa mesh protocols as possible solution
John - big issue, lagoons are not good locations for solar panels, must be inconspicuous, plus storms and wave action. Primary battery may be key since solar not good in lagoon.
James - also considering bluetooth, considering study based on different frequencies.
Dan- recommend starting simpler, back to base LoRa instead of LoRa WAN. probably double the distance you can reach because you dont need acknowledgement needed from LoRa WAN, also easier to program.
Martha - DIRT project, digital integrated research technology. Demonstrated need for an ability to get sensor data from inaccessible field sites to the user. As well as unobtrusive sensors!
Another project - TOMST soil moisture and temperature sensors at a mine waste area. Looking to see if metals in soil are correlated with plant spacing patterns.
Joe - used tile markers, has lots of ideas re salt marshes.
Jake- working on proposals! Remote sensing, ISAT spaceborne altimetry
12/06/22
Topic: Chris Gazoorian, USGS
Date of meeting: Dec. 6, 2022 Link to Recording Participants:
FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details
Scotty Strachan - Nevada System of Higher Education | System Computing Services
James (Jake) Gearon - IU Bloomington, ESIP Fellow
Martha Apple - Montana Tech
Joseph Bell - USGS
Allison Mills - ESIP
Chris Gozoorian - USGS
Andrew Wiedlea - ESNet
David Coyle - USGS
Gabrielle Boisrame - DRI
Mark Swinton - RPI
Nora Catolico - NEON
Xiongwen Chen
Announcements:
Time Stamps (ET):
17:12 -
Notes:
General: ESIP at AGU this year, Ignite and Data Help Desk
Chris Gazoorian presentation
- Upper Neversink Innovation Test Bed, NY upstate
- All New York City water supply
- Legacy monitoring: traditional stream monitoring + a few met stations + one ET station + deposition collection site
- large suite of envirosensing devices and applications to figure out which works best
- Basically anything you could ever want re: enviro data. Monitored as it supplies water.
- Fish population research dovetails with hydrology, especially as local environment is spring fed
- Paired water and air measurements at-a-station, 51 sites with HOBO TidBit sensors where radon and depth to bedrock is also collected.
- Helps to identify how much the water responds to the local air temperatures.
- Upper pigeon brook subbasin, many ground water springs destroy the cohesion of the granular sediment and cause debris flows.
- Aquifer research in the neversink is a large component of the groundwater storage in the basin, trying to figure out the total volume/
- Trail cameras check for stream ephemerality.
- Water-air temperature sensitivity changes down the catchment
- ML model for depth to bedrock, then can observe where there are large disparities in temperature equilibration timescales
- Monitoring for radon to determine groundwater residence times.
- O18 isotopes were also collected, show a highly seasonal dependence. The autumn isotopic signature is distinct from the spring signature
- HVSR tremino sensors, require shear velocities for analysis of depth to bedrock
- Thermal imagery can help attribute water sources by looking for colder groundwater seeping into active channels
- Site recon with FO-DTS to map subsurface seepages. Can also use handheld thermal cameras.
- Cables can be 1km long for FO-DTS, looking for preferential groundwater recharge zones, which can also be directly monitored.
- Gauges run off of AC power, though some of the more remote sensors are run by solar, use high gain antennas as there is no cell service
- UAS orthoimagery and thermal imagery
- Fixed mount thermal went through some iterations, settled on FLIR A700, can stream the data with different protocols, allowing for customization.
- Also have real time RGB imagery
- Appears to be differntial groundwater dynamics between the alluvial floodplain and the hillslope seeps.
- Rivers, if unconnected to groundwater, are quite more variable in temperature
- Ideally scale this to the national level for groundwater inventory
- https://www.usgs.gov/apps/ecosheds/fpe/#/ trailcam water flow explorer software application
Discussion:
Chat: 00:20:22 Joseph: NGWOS = The Next Generation Water Observing System 00:57:26 Scotty Strachan: if anyone has a question, feel free to "raise hand" in Zoom
01:05:16 Chris Gazoorian USGS: https://www.usgs.gov/apps/ecosheds/fpe/#/
01:10:28 Joseph: https://water.usgs.gov/fisp/
11/01/2022
Topic: Mahta Moghaddam
FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details
Scotty Strachan - Nevada System of Higher Education | System Computing Services
James (Jake) Gearon - IU Bloomington, ESIP Fellow
Martha Apple - Montana Tech
Joseph Bell - USGS
Allison Mills - ESIP
John Porter, UVA VCR/LTER
Todd Caldwell - USGS
Mahta Moghaddam - USC
Gabrielle Boisrame - DRI
Xiongwen Chen, Alabama A & M University
Stijn Wielandt, Lawrence Berkeley National Laboratory
Bryce Nance - National Ecological Observatory Network (NEON) / Battelle
Announcements:
- LAST DAY to submit! Ignite at AGU coming up- sci comm oriented talks: https://igniteagu.io/
- ESIP: Subscribe to ESIP Update: sign up! esipfed.org/subscribe
- ESIP Meeting Session due next week - Fri Nov 11 esipfed.org/meetings
- Envirosensing session at summer 2023 ESIP meeting?
- Trying to gauge interest
Time Stamps (ET):
17:12 - Start of presentation: Mahta Moghadam
Notes:
- Remote sensing of water with microwaves, it is primarily sensitive to the water signal. Ultimate strategy must use all types of remote sensing, including in-situ sensor networks. Want to sense water at a variety of scales.
- At low frequency, you can get dielectric profiles from subsurface!!!
- Led to design of NASA AirMOSS radar, P band frequency was chosen to ensure soil water content below veg and surface
- Collected data from all across the US, including 9 different Biomes
- Can get down to -75 cm into the subsurface…
- Inversion algorithm finds best set of scattering model parameters…
- With 1000 flight hours, errors are reasonable with max error at 0.45 m
- What happens with permafrost and P-band radar?
- No direct active layer property measurements
- P-band radar can resolve the depth to active layer remotely!!!!
- Can generate time series with this methodology
- Many variables that go into active layer occurrence over time, need more data to confirm.
- UAS may be able to gap fill more expensive airborne acquisitions.
- Software defined radar uses a low cost hardware platform that can be remotely configured for operating frequency, bandwidth, imaging mode, and power
- Good estimation of water-table depths down to 12 FEET!
- Can identify the snow surface and ground surface!
- Trees make things harder
- Custom hardware and software for WSN and UAV operations
- Everything is autonomous re: the drone.
- Good SNR below trees, can convert to soil moisture
- UAS-based systems will be able to gap-fill larger more $$ datasets
- Volume scattering at higher frequencies, even at P-band, we are sensitive to presence of roots. Larger rocks in the subsurface
- NEON flies sites every 3 years lidar, curious about bathymetry applications?
- Hoping to start a project with this drone based system, need lower frequencies. How to separate reflections from the top and bottom.
- Scotty: cm scale environment at plot or landscape for surface characteristics as well as moisture constant. Can you get improve accuracy with the system for cm-scale characterization?
- A: a qualified yes: cm-scale or sub-cm surface roughness, probably a yes. They have the EM wave roughness scale as a variable. We assume somewhere between .5 cm to cm roughness. Cm-scale resolution is probably not possible with current limit of bandwidth. However, the next best thing is using an analytical profile model. This makes it more possible
Discussion:
Chat:
00:17:21 Room 2: If you’re just joining us, here is the sign in sheet! https://docs.google.com/document/d/12QVf-WwBnoW1mlADsoa6CZ_hkXzaFozR1TJ3oW-PCi0/edit
00:18:06 Room 2: If you’d like to sign up for the list serve, here is the link! https://lists.esipfed.org/mailman/listinfo/esip-envirosensing
00:23:26 Allison Mills | ESIP: Welcome, Gabrielle!
00:24:09 Allison Mills | ESIP: Welcome to you as well, Xiongwen!
00:49:38 Room 2: Wow!
00:50:04 Room 2: Consider ICESAT-2 for ground surface validation, it can get bare earth transects in vegetated terrain
01:14:19 James Gallagher: Thanks! Great Talk.
01:14:42 Mahta Moghaddam - USC: Thank you very much for letting me speak at this wonderful forum!
10/04/2022
Topic: Sander Denham, IU
Date of meeting: Oct. 4, 2022 [Link to Recording] Participants:
FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details
Scotty Strachan - Nevada System of Higher Education | System Computing Services
James (Jake) Gearon - IU Bloomington, ESIP Fellow
Martha Apple - Montana Tech
Joseph Bell - USGS
Felimon Gayanilo -TAMUCC
John Porter, VCRLTER, Univ. Virginia
James Gallagher - OPeNDAP
Allison Mills - ESIP
Daniel Fuka - Virginia Tech
Announcements:
- Partnership award is up and live! https://www.esipfed.org/get-involved/peer-recognition/esip-of-the-year-partner-award
- Martha Maiden award for earth science informatics: https://www.esipfed.org/get-involved/peer-recognition/martha-maiden-award
- Ignite at AGU coming up- sci comm oriented talks: https://igniteagu.io/
- ESIP: New way to sign in/subscribe to ESIP Update: sign up! esipfed.org/subscribe
- Community Fellows application are open: https://www.esipfed.org/get-involved/student-opportunities
Time Stamps (ET):
17:11 Novick lab scales from leaf to whole plant.
17:12 leaf-level measurements done by leaf water potential and leaf level gas exchange
17:13 can get live read out from data in chamber, and fully controlled.
17:15 60 hz collection from Ameriflux tower in Morgan Monroe State Forest
15:18 sapflux sensors use thermopile embedded deep in the tree to measure values.
15:18 thermal dissipation is the temperature differential, lowest water flow is at highest temperature differential.
15:21 Reduce outside temperature by insulating from the outside world, bubble wrap, tape, etc. HVAC system reflective bubble wrap
15:21 hydrologic stress mechanism in water transport in plants
Soil moisture and VPD (vapor water deficit)
15:22 Different species of plants regulate their stomata in different ways, more conservative in water use strategies, or more conservative in water use strategies.
15:23 use pressure bomb to see pressure on watr column, can determine stomatal activity.
15:24 pressure on water column can cause embolism in plants, “catastrophic hydraulic failure” during drought.
15:25 Convert sapflux data (transpiration) is converted to stomatal conductance
15:26 different common indiana trees have different conductance to VPD ratios,
15:27 tulip poplar has quite different conductance during wet vs dry conditions, sugar maple as well. White Oak, however, has a more risky behavior, has high rates of canopy conductance in the face of dry conditions.
15:28 Applied this methodology to wet and dry places, missouri, NC, and indiana. Not all species represented but similar species. In wet sites, still seeing large reductions in stomatal conductance that is driven from VPD.
15:30 In drier sites, soil moisture reductions become more important relative to vapor pressure deficits. Moving into different climate scenarios, regardless of soil heterogeneity, VPD is expected to increase everywhere with increasing temperatures, driving changes across species.
15:33 Species-specific response to change, generally done on the ground which can be subjective…current work is on how to identify specific species canopies in the phenocam to have more objective outlook on shifts in season based on climate change.
15:37 Drought forecasting using phenology and leaf on time could be useful.
15:39 Build everything in lab, question from John Porter: Can you comment on how long the sap sensors are able to run between servicing/replacement? Also, how many replicate trees do you typically require to characterize a species sap flow? If no critters or storms, can go for around a month! Never use less than four for replicate trees.
15:43 Line power or solar? Usually line power, but they can be installed with solar power as well. Marine battery connected solar panels.
15:44 Dan Fuka question: 18 trees monitored, can model one tree accurately, but 18 trees is an amazing dataset which you could go to the NSF with for anything mentioning “scale”
15:45 Have you considered doing IR? Interested in IR. this book is great: https://www.abebooks.com/servlet/SearchResults?isbn=0387902287&cm_sp=mbc-_-ISBN-_-all
15:47 dan set corn on fire with heaters
15:48 UVM and NY doing sap sensors in sleeper watershed. ESIP summer 2023 is in burlington
15:50 Have you looked at leaf anatomy? Oak leaves might be more drought resistant from a thickened cuticle…Have not looked at leaf anatomy. But have a growth chamber..colleague of Sander’s is looking at white oak and sugar maple. Stay tuned to lab. Specific leaf area would be useful as well. Stomatal density could be playing a role. Pathways of water vapor
15:53 They used CS16’s to measure, what sort of calibration stuff do you have to do with those? Question from James Gallagher.
- https://www.campbellsci.com/cs616-reflectometer
15:55 Soil water potential might be better than soil moisture for plant water availability.
15:57 Check data weekly to make sure sensors are off. Limitation is built sensors.
15:58 Ken Connor at Campbell Scientific is a good contact [[1]]
15:59 Bullet style camera, code developed out of canadian lab.
Notes:
Discussion:
Chat:
09/06/2022
Topic: Green Stream Technologies Presentation
Date of meeting: Sep. 6, 2022 [Link to Recording] Participants:
FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details
Scotty Strachan - Nevada System of Higher Education | System Computing Services
James (Jake) Gearon - IU Bloomington, ESIP Fellow
Martha Apple - Montana Tech
Vasu Kilaru - US EPA. interested in sensors and data standards
Allison Mills - ESIP, communications manager, [[2]]
David Coyle - USGS
Joseph Bell - USGS
Bryce Nance - NEON
Steven Pyle - Director iFLOWS, VA Dept of Emergency Management
Karen Lindquist - Green Stream COO
Jim Gray - Green Stream CEO
Jim Gallagher - Chief Engineer, NH Dept of Environmental Sciences
Matt Gyves - USGS Pennsylvania Water Science Center
Announcements:
- ESIP: New way to sign in/subscribe to ESIP Update: sign up! esipfed.org/subscribe
- Also sign up for webinars here: www.esipfed.org/webinars.
- Community Fellows application are open: https://www.esipfed.org/get-involved/student-opportunities
Time Stamps (ET):
17:13 - Green Stream Technologies www.greenstream.com is an enviro tech company based in Wake Forest NC as well as ops in VA and hampton roads area.
17:14 - overarching goal is to make envirosensing easy and affordable for customers. This is needed due to imminent climate change threats. Also modernization of legacy equipment, doing more with less data-wise
17:15 - this can be costly, legacy systems themselves and maintenance is expensive. W.r.t GS, end -to end solution provider for customers. Design, develop and deploy those solutions
17:16 - 4 columns to solution, sensor network, wireless comms, cloud services, real-time data. LoRa WAN, satellites in the form of GOES. cloud based program.
17:17 - more customers are adopting cloud providers, (Azure, AWS, Google), attempt to make data real-time or close as possible, multiple ways for customers to consume, dashboards are built and provided to customers. Dashboards are adaptable on a per-client basis.
17:18 - API’s are provided for customers for get and post reqs. Additionally, sensors can talk directly to customers platforms as well.
17:19 - In house toolset: Electronics (R & D), systems eng. Computer science (programming at the embedded level up to the cloud level), mech eng. (install), business law and contracting, service operations (maintenance of products afterwards).
17:21 - started off with a low cost low footprint sensor with mainly off the shelf parts and packaging. Easy to install and low maintenance req.
17:22 - govt customers as well as private citizens are buyers.
17:22 - solar powered, internal batteries, no external infrastructure for deployment, work on LTE and LoRa WAN
17:23 - reports every six minutes (adjustable). Streaming real time into system and pulled in by customers. Provide real time dashboards with associated APIs
17:24 - all the sensors deployed in NC fed into FIMAN https://fiman.nc.gov system (flood inundation mapping and alert network) and VIMS StormSense and onto NWS.
17:25 - city state and county level, entities will purchase and then GS connects and deploys.
17:25 - overcoming the 3 V’s, vegetation, varmints, and vandals. Have deployed filtering algos to minimize noise associated with 3 V’s. Bugs and spiders like the sensors (the cones).
17:27 - sometimes people rob the sensor :(
17:27 - GS also does larger deployments, sensors with larger footprints. Deployed on piers in the OBX which required specific solutions and installed by GS.
17:29 - CAD done in house, fusion 360. Everything is modeled for visualization as well as working out the kinks beforehand. The model is then sent out for fabrication. Built and assembled in house then installed.
17:30 - GS can also deploy 3rd party devices
17:32 - case study upgrading 3G devices for VA Beach to LTE.
17:33 - large - scale deployment of radar, 71 radar sensors put in for the state of NC. LTE and GOES satellite comms. All campbell scientific, customer had concept for enclosure but that was it. GS did the mount solutions. Fabricated bespoke boxes to make them 100p watertight. In house test jigs were built. Given access to network management system for NC, called contrail, configured that for the state. Helped with encoding and decoding from GOES.
17:37 - coordinated lane closure and traffic control for deployments. Currently, GS is working with USGS for a pilot project in Frankford Creek, request from USGS, boxes need to talk to AWS and support multiple wireless communications. Designed proprietary electronics to support LoRaWAN, LTE, bluetooth for multiple forms of communication. This is important for the sensor to be able to leverage different comm channels for different things: events vs. updates etc. Wifi also allows wireless configuration via set up mode. Acts as a web server for updates, then can re boot and change comm configuration. Communicating with AWS IoT core, mqtt and https. Jobs capability with AWS which can help for updates. Boxes report every 6 min with LTE followup with LoRaWAN. All brand new hardware.
17:41 - building battery packs with solar charging, can run off 6 V solar panels and recharge itself.
17:43 - in house building of electronics helps to save costs and improve performance.
17:44 - Also building a suite of interface cards and libraries around those, concept is to make this modular for end user.
17:45 - Network Management: event publishing, device config, job processing, file download, application management.
17:46 - Make dev kits moving to industrial, multiple comms, modular interface cards, firmware libraries, common dev tools.
17:48 - Needs: mobile apps, circuit board layout, firmware coding, building libraries, building network management, also, decision needs to be made to be OS or proprietary. Some risk comes with either option.
Scotty - big fan of publishing mature code base and fork it off for proprietary purposes etc. Contact matt bartos for intern help https://www.caee.utexas.edu/people/faculty/faculty-directory/bartos
scotty: question for at-scale deployment. Pushing updates is different than set and forget. Interesting category of hardware emerging. Question about pushing OTA updates one by one or across the board.
Answer: stairstepping, in house testing first, but will push based on customer reqs as opposed to all at once or by fleet.
17:57 - Joseph, enjoys the CAD model creation, which is helpful for the end user (like the USGS).
Answer: CAD models gives flexibility to look at a myriad of solutions rather than a simple diagram. Models have been VERY worth it in long run. Models of circuit boards are also created.
18:00 - Micropython question from James Gallagher, how useful was it vs C++.
Answer Using pycom, performance has been very good, little devices can be multi-threaded !!! and secure SSL communication. Maker community is also very good and useful. Overall pleasantly surprised with it. It can do async ops as well.
18:02 - GOES-R communication and iridium. What's the operation difference between those two?
Answer: analysis done in proposal, are sites where u neet sat comms. GOES is one-way trip. It’s robust and it works, footprint is large relatively, message size is robust, but only once an hour default transmission, free for govt entity which is a big one. Window is small (170 char window). Iridium is ~50 character window. Iridium is cheap up front but has higher comms cost. Antenna size is tough with GOES, IoT via satellite is lower cost for data and hardware with smaller antenna.
18:06 - Joseph, some frustration with iridium at usgs. Iridium pricing :C.
18:07 - Matt Gyves, will deploy combo LTE/LoRa/Wifi Green Stream gauges on USGS pilot at frankford creek in Philadelphia.
Notes:
Discussion:
Chat:
08/02/2022
Topic: Member Recap / Roundtable
Date of meeting: Aug 2, 2022
Round Table [Link to Recording] Participants:
FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details
Scotty Strachan - Nevada System of Higher Education | System Computing Services
James (Jake) Gearon - IU Bloomington, ESIP Fellow
Allison Mills - ESIP, comms manager
John Porter - University of Virginia, Virginia Coast Reserve Long-Term Ecological Research
James Gallagher - OPeNDAP
Martha Apple - Montana Tech
Mark Lucius - RPI/The Jefferson Project
Bhuwan Thapa, Dept. of Geography & Planning, Appalachian State Univ, NC
Announcements:
- ESIP: New way to sign in/subscribe to ESIP Update: sign up! esipfed.org/subscribe
- Also sign up for webinars here: www.esipfed.org/webinars. Next one is the ESIP Meeting Highlights Webinar: Wed Aug 17 at 3 PM EDT
- Data for All t-short fundraiser closes Aug 10: https://www.customink.com/fundraising/esip2022
Time Stamps (ET):
17:05 - Intros
17:06 - OPeNDAP, James G. and Martha A. Dirt Project: sensor web deployed in alpine envs, uses LoRA tech to take measurements from a group of distributed sensors and sends it to a central node, which uses iridium tech to send the info back to server. Master node is proofed, as well as leaf nodes, of which there will be multiple talking back to master node. Leaf node was installed a year ago in a research plot to test. Data has been plotted and sent back to the node (1h interval). Batteries are good / healthy with good voltage. Projected 10 years of battery life: microprocessor based, uses microamps (20) while sleeping. Wakes up once an hour and does ~3s of work. Probably do NOT need to boost. Tested in Goat Flat (impractical to have solar panels or external/separate power sources). Also installed soil moisture probes but difficult in the rockies.
17:15 - Scotty S. new project: 5 year NSF harnessing data revolution for fire science. Deploying experimental sensors in UAVs, fly over pre and post fire plots as well as during. Experiment with high resolution sub-centimeter detection of fuel sources. Whole process of computer vision + sensor fusion is exciting and cutting edge! LoRA Wan gateways in tahoe region for community science. Answering questions that state wants answered around lake tahoe, as well as let community members deploy sensors after following a short tutorial. Hoping to cross-pollinate with ESIP!
17:17 John P. - robotic controller for stepper motor (https://photos.app.goo.gl/6wkYNU7ZHsTUQUsz8) that can be used in middle of bay. Essentially goes inside a stainless steel tube. Security sensor reed switches can go inside the box and you can turn things on and off without opening the tube. Sensor is designed to counteract tidal action to get consistent data. For work in shallow lagoons, solar panels are not good. They don’t float, they attract thieves and others. Really looking to get into LoRA out there…asks scotty if they need to set up their own LoRA nodes. How do you deal with LoRAWAN gateway pricing on state grants?
17:21 Allison M: run farm, having trouble making thermometers work cold settings vis a vis bluetooth. Any help is desired! Otherwise helps with social media and communications.
17:22 Mark L. Jefferson Project: operates a sensor project on lake george, 5 vertical profiling platforms. YSI ones were great so they built their own. Also deployed on other lakes around. Deployed for harmful algal bloom research. Kevin Rose is director, sensors are focused on spatial resolution, 26 locations on Lake George alone. New to group, hoping to learn. Also looking to improve QA/QC as it is currently time intensive.
17:26: Bhuwan T. USDA funding assessing role of wind breaks, getting data from remote sensing as well as machine learning. Wants to learn if can deploy sensors to look at wind damage with UAV, also teaching a course on urban planning, wants to bring in sensors into that class.
17:29 Dan F: horse-tail mounted sensors!!!! Different ways to engage ag industry and funders. Playing with sensors that are designed to be cheap ~ 25$. USDA is tied to ESRI! ASR6502, can cut power to a trickle, and burst when data is back. Fits in a nice chinese lightbulb. USDA is federal, but its all china based…and so thats a no go. Now Dan has found a laser cutter! Which makes a nice little housing unit. Teamed up to design replacement for microprocessor and other replacements. Has to get data into ESRI software. Campbell Scientific doesnt wanna talk to ESRI and vice versa. Big roadblock. Most of what Dan will be doing next year will be walking that line.
17:39 Joseph B. agrees with china prohibition makes hard to acquire parts. Joseph is trying to establish a liaison between the USGS and ESIP—academics, industry, etc. also tasked with transitioning sensor networks for next gen. How do we have networks without borders etc? Key focus this year, trying to get DCP, also LoRA WAN surveying for LoRA WAN working in dense urban environments. Also connected with California Water Science Center
17:47 Dominant use of GOES @ USGS, existing data collection workflow.
Scotty S. Q: LoRA gateway protocol for not standard sensors that get jammed into transmission which unload it in goes gateway and from that get disseminated into things network automatically? - Joseph says yes! Can we keep the existing pipeline via retrofitting ? 30-45k to start a gauge, 15k to maintain gauge down the line…if a node can stand up that can provide another gauge, would be helpful.
GOES window. Window of time to send sensor data back and forth. This would be a total alternative to using Iridium. Given a modem, you have a certain bandwidth you can press into the timewindow you’re given. If you’re only transmitting data from one sensor, you’re wasting bandwidth. It’s a very robust global delivery system due to the infrastructure…it is cheaper too!
NRCS uses snotel, media burst still. The footprint of the antenna is huge! 6m wavelength antenna. GOES uses decent amt. of power. Campbell Scientific modem is not cheap though.
Notes:
Discussion:
Chat:
05/03/2022
Topic: Eric Rowell presentation - wildfire science - DRI
Date of meeting: May 3, 2022
Round Table [Link to Recording] Participants:
FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details
Scotty Strachan - Nevada System of Higher Education | System Computing Services
Daniel Fuka - Virginia Tech |
Allison Mills - ESIP
Joseph Bell - USGS
Martha Apple - Montana Tech
Bryce Nance - NEON
Felimon Gayanilo - Texas A&M
James Gallagher - OPENDAP
John Porter - VCR-LTER
Karen Moe - Disaster Lifecycle cluster
Eric Rowell - DRI
Announcements:
- ESIP: New way to sign in/subscribe to ESIP Update: sign up! esipfed.org/subscribe
- Also sign up for webinars here: www.esipfed.org/webinars
- Registration is open for the July meeting! esipfed.org/meetings
- ESIP Lab RFP - funding ML tutorials focused on hydrology, seismology, and cryospheric science; proposals are due May 15: www.esipfed.org/rfp
Time Stamps (ET):
17:05 - Intros
17:16 - Beginning of Eric Rowell’s talk
Notes:
Discussion:
Chat:
04/05/2022
Topic: John Fulton Lecture - USGS
Date of meeting: Apr 5, 2022
Round Table [Link to Recording] Participants:
FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details
Joseph Bell - USGS
Jake Gearon - Indiana University, ESIP Community Fellow
Scotty Strachan - Nevada System of Higher Education | System Computing Services
Martha Apple - Montana Tech ! Butte, Montana
Guy Litt - NEON
John Porter - Virginia Coast Reserve LTER, Univ. Virginia
Allison Mills - ESIP
Karen Moe - NASA ret. ESIP Disaster Lifecycle cluster co-chair
Steve Young - USEPA (ret) & Innovate! Inc; Air Quality Cluster co-chair
Announcements:
- ESIP: New way to sign in/subscribe to ESIP Update: sign up! esipfed.org/subscribe
- Also sign up for webinars here: www.esipfed.org/webinars
- Registration is open for the July meeting! Also, proposals are open for sessions! esipfed.org/meetings
Time Stamps (ET):
17:03 - Intros
17:12 - Beginning of John Fulton’s talk: Post-wildfire Streamgaging
17:14 NonContact vs Conventional Stream gaging - these are used in high altitude streams
17:15 basic streamgage measures…stage!, which has been referenced to a datum. Discharge has a relationship with stage.
17:16 - ADCP: produces a distribution of velocity and depth which gives Q. this correlates to stage. Rating curve is generated from this.
17:17 - NonContact Monitoring in Burn Scars - ensemble vs ground based methods, when paired, can minimize false alarms.
17:20 30ft/s stream velocities can damage sensors, even no contact ones!
17:21 Dataloggers with solar panels, rain gauge buckets, soil moisture probes. This is a functional unit
17:22, stage radars and velocity radars help to monitor the stream continuously
17:23 - 1 min frequencies between data points. Have radar, rain gage. Also transmit data quality in the form of signal to noise ratio.
17:25 - Lag times are crucial, especially based on velocity which is what is fed to alert networks. Earlier response in velocity than stage!
17:27 need techniques to quantify area during floods, since the hydraulic geometry is so changed.
17:29 uses Q = phi * u_max * A
Mean vel of channel: max vel of channel
Idea is to take surface velocity value and give mean channel velocity
U_max doesnt need historical data
17:33 Terrestrial lidar can be used to assess area of channel before and after
17:37 Phi is the mean channel area divided by max velocity !
17:45 - stage area ratings are imperative!
17:49 - Daniel Fuka gives update on Summer Meeting session proposal!!!
17:50 - demonstrate all components of data workflow at ESIP summer meeting.
Notes:
Discussion:
Chat:
03/01/2022
Topic: March Monthly Meeting
Date of meeting: March 1, 2022
Round Table [Link to Recording] Participants:
FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details
Joseph Bell - USGS
Jake Gearon - Indiana University, ESIP Community Fellow
Scotty Strachan - Nevada System of Higher Education | System Computing Services
Martha Apple - Montana Tech ! Butte, Montana
Daniel Fuka - EC/AGU | Scientist | Virginia Tech | Likes NA Beer
Guy Litt - NEON
Announcements:
- ESIP: New way to sign in/subscribe to ESIP Update: sign up! esipfed.org/subscribe
- Also sign up for webinars here: www.esipfed.org/webinars
- Scotty is giving a talk! https://docs.google.com/document/d/1jxQmW8bwmQi9RJhkqMXoVK4VfnuiOGZu9nEdzp-R4iE/edit
Time Stamps (ET):
17:04 Intro Welcome from Scotty
17:07 Intros from all
17:10 Thinking about cluster theme: wildfire aligned themes for the next year or so…opportunity for questions. Floor is opened.
17:11 ESIP is pushing in-person meeting for summer and virtual for winter 2023: opportunity for hands-on workshop at summer meeting?
17:12 Virtual meetings lower accessibility barrier for meetings…
17:13 Sensor workshop will be hard to do, but there are many resources available… could be from getting sensors to work to data gathering / labelling / cleaning. etc making it FAIR. - D. Fuka.
17:14 J-link debugging for microcontroller units - small demo possibly from James Gallagher
17:15 just for potential comradery: there are other clusters talking about trying to do more of a workshop/hands on session for ESIP Summer as well (Semantic Harmonization) -- so, there may be more general enthusiasm and co-support for hands on! - Bar
17:16 USDA partnerships in innovation would be good contact, possible sponsor. On its way
17:17 Two people stepping up for summer meeting! Daniel F. and James G.
17:19 Making it an outdoor session would be ideal…introduction to local host in Philadelphia could be good. - Daniel Fuka. +1 from Martha. Will need to figure out how this meshes with the meeting plan..
17:23 Necessary to plan for these insofar as when folks come unprepared its derails the workshop.
17:25 injecting problems to be debugged that we know how to debug. Details do need to be worked out a long time in advance..
17:27 just for potential comradery: there are other clusters talking about trying to do more of a workshop/hands on session for ESIP Summer as well (Semantic Harmonization) -- so, there may be more general enthusiasm and co-support for hands on! - Bar
17:30 Consider a debug-specific session too? Could help account for overflow
17:30 Switching topics now: Wildfire
17:31 Ag and Climate cluster unanimously voted to go on a hiatus…
17:32 Disaster Lifecycle has gamed out next three meetings worth of content re: wildfire. Seems like energy is headed that direction.
17:33 Envirosensing is technical cluster and in a good position to contribute, but glad to continue in an advisory role. ESIP can make impacts by connecting stakeholders.
17:34 When opportunity for cross over, let Envirosensing cluster know.
17:35 Guy is moving on in his role as NEON participant, thanks all and says someone new will rotate into the role.
17:37 USGS and NOAA facing same problems as NEON re data networks for sensor deployment. Cyber security will play a large role as sensor networks become more integral
17:39 Joseph introduces slides for poster about envirosensing cluster
Can help to connect people in agencies, academia, private sector
17:42 Proposal prep: small (<10k) and medium (>10k - 60k) concerning next gen water systems, sensor networks.
17:47 Whats obvious to Envirosensing cluster isnt to everyone. Scotty, Martha, Jake will work on poster with joseph, others feel free to help
17:47 Next few calls will be speakers, supporting hands on workshop meetings may be offline or could be fit in.
Notes:
Discussion:
Chat:
02/01/2022
Envirosensing Monthly Meeting
Topic: February Monthly Meeting
Date of meeting: Feb 1, 2022
Participants:
FirstName LastName - (ESIP Affiliation if applicable) Title | Affiliation | Other details
Joseph Bell - USGS
Jake Gearon - Indiana University, ESIP Community Fellow
Scotty Strachan - Nevada State Higher Ed System, Envirosensing CoChair
Felimon Gayanilo (Chair, Partnership Comm) Texas A&M University
Guy Litt - NEON
Allison Mills - ESIP
Brian Wee - Massive Connections
Steve Young- EPA (ret) & Innovate! Inc | he/him | Arlington VA
Martha Apple - Montana Tech ! Butte, Montana
Daniel Fuka - Virginia Tech | Albuquerque, NM
Time Stamps:
17:05 - Introductions
17:12 Recap of Meeting: discussing high-level wildfire concepts from envirosensing context. Stakeholders at meeting were important and helpful: e.g. Chief Winnacker from ESIP meeting. Interdisciplinary conversation on how to apply science-driven tech to tactical response for wildfire. Also for mitigation before and after fire events. Long term observation is bread and butter of cluster, but how can we increase the velocity of the data? How can be better operationalize data for a wider audience? We want to see more thematically driven collaboration across the clusters with longer timeframes (1-2 years) on topics like wildfire. ESIP doesn’t have to deliver products, but needs to foment conversation and connections between experts.
17:18: From USGS perspective, fast or real-time data is crucial. Even binary data can be helpful at high velocity. - Joseph
17:18 Sensor use in context: where do you put sensors in context of a wild-fire? Can we position things in place ahead of time? Chief Dave Winnacker helped to ground the discussion in what responders really need. - Martha
17:20 Air quality aspect of sensing wildfires should be considered. Additionally, how possible is it to alert those on the ground? Especially when wildfires are fast and unpredictable. Would deployable sensors be effective here? - Steve Young
17:22 Air quality and distributed sensors: chief winnacker stated that he would like to have that sort of data summarized quickly and efficiently. For air quality, you’re wading into proprietary data which can be difficult to handle. Can we combine air quality sensors with the ability to turn on sprinklers in a back-yard (from winnacker).
17:27 OpenAQ with distributed sensors. Open EPA solicitation here: https://www.epa.gov/arp/enhanced-air-quality-monitoring-funding-under-arp
17:28 Situational awareness, deployable sensors could be put into action by firecrews? - Joseph
17:30 Scotty shows slide showing possible pathways for data collection to analysis and delivery.
17:32 The Chief mentioned a sort of fire sensor "fence" proof of concept that used a perimeter of sensors/IOT model to detect a wildfire within a very finite area. - Steve Young
17:33 Josh Lieberman Open Geospatial Data Consortium. Working on a data product that is designed for stakeholders on the ground to summarize many data.
17:36 Vis a vis infrastructure, maintenance is often the bottleneck, as it will incur costs ad infinitum - Brian Wee
17:39 The government funding to improve broadband access in underserved areas might create opportunities to improve connectivity for sensors - Steve Young
17:40 Emerging technology is included in infrastructure bill to a certain degree: esp. Regarding AI/ML
17:41 Scotty shares slide: regarding R&D for wildfire, its largely piecemeal as is.
How best to merge UAS data with in situ data? Machine Learning ready training data sets will be needed and in-demand as data products.
17:47 Monthly calls will be speakers, but we will stay close beside wildfire focus of ESIP
17:48 Re: Artificial Intelligence and data-ready AI data is hyper important.
17:49 Can we use in-situ sensu deployments to perform image tagging on image datasets
17:50 We have all the sensors, ESIP, strong foundation, create a NEW federal repository through a federal unit. One agency doesn’t have a federal data repository which is USDA: project is called XX. covers all states with reps in every county. They are currently trying to build up data repository. Could we use this to build a ground-up network?
17:52 https://www.nifc.gov/fire-information from Steve Young
17:54 NIFC (National Interagency Fire Center) is one place to look, but interestingly not a NASA partner
17:55 Two people who come to mind for cloud-based architecture/infinitely scalable data repos. Colin Bode and Matt Bartos
17:58 NASA workshop upcoming. Info gathering stage https://nari.arc.nasa.gov/sites/default/files/attachments/Feb2022_SMDWorkshop_ParticipantAgenda_v2.pdf
6:03 USDA also has to host almost every kind of data which would give some
Announcements:
- ESIP: New way to sign in/subscribe to ESIP Update: sign up! esipfed.org/subscribe
- Also sign up for webinars here: www.esipfed.org/webinars
- Friday Feb. 11th @ 1pm ET Conference Highlights
12/07/2021
Envirosensing Monthly Meeting
Attendees: Joseph Bell - USGS, Jake Gearon - Indiana University, ESIP Community Fellow, Scotty Strachan - UNV Reno, Envirosensing CoChair, Allison Mills - ESIP Comm. Manager, Kristina Fauss - UC Santa Barbara, Martha Apple - Montana Tech, Co-Chair of Envirosensing Cluster, Megan Carter - ESIP Community Director, Daniel Fuka - VA Tech University, Zachary Robbins - North Carolina State University, James Gallagher - OpenDap, David LeBauer- University of Arizona, Xi Yang - ESnet, Bar - ESIP, John Porter-UVA, Andrew Wiedlea - Energy Sciences Network (ESnet), Brian Wee - Massive Connections, LLC | Co-chair ESIP Ag and Climate Cluster
Announcements
- More info on the ESIP January Meeting including registration and the call for sessions: https://2022esipjanuarymeeting.sched.com/info
- Sign up for ESIP Winter Meeting! ^
- Specific session for Ag & Climate x Envirosensing
- Links from Allison:
- ESIP Links:
- Meeting registration: https://2022esipjanuarymeeting.sched.com/info
- ESIP Update: esipfed.org/subscribe
- Ignite! If you'll be at AGU add this to your calendar: gniteagu.io/
- ESIP Links:
Notes
Call for a speaker at the joint Ag Climate & Envirosensing, reach out if so!
James Gallagher: building units and deploying/testing is difficult, want to go from star network topology to Mesh topology. Asks if anyone has experience with simulating sensor networks.
- Power budget is limited, sensors can only function a few seconds per hour. Mesh networking edge nodes are senders and receivers, transmits are expensive. Simulating power consumption is ideal.
- Mini-net but for the edge, ability to simulate power budgets instead of just network topologies.
- https://www.dryad.net/. Presentation in mesh over LoRa at the Tactical Fire Remote Sensing Advisory Committee (TFRSAC: pronounced “tiffer-sac”) meeting co-organized by the USDA Forest Service / NASA on 2021-12-01 and 2021-12-02.
- Another link https://publish.illinois.edu/ymb/ that might be helpful
- Good computer science + engineering project to determine optimal power consumption for sensor networks (?). Could start with discrete sensor simulations, reaching out to DRYAD might be helpful. It may be proprietary but should be reached out to.
- Electrical engineering simulation + sensor network simulation. NSF may be interested in funding via CSSI, framework with element pilot then submit the grant :). The proposal to develop such a software would be worth conversation with a program manager
- David LeBauer sent link: https://beta.nsf.gov/funding/opportunities/communications-circuits-and-sensing-systems-ccss-0 for possible resource concerning Electrical Engineering research.
LoRa implementation, has anyone employed Dragino sensors?
- Programming was ‘funky’ re: multiple ethernet connections and poor documentation. Linux box with YUN + arduino. Therefore cannot run much code. Some units arent rugged enough to be outdoors
- Daniel Fuka: much of Dragino is simple via an OpenWRT interface. However, they do not allow multichannel LoRa unless you purchase a much more expensive unit. Not great about sharing source code. Sensors are efficient. They can be hardwired for more efficiency but a good place to start.
- Wondering about Dragino uses for meteorological applications. The unit with the combined MCU + LoRa. Cannot code to be only single channel.
- LoRa experimentation is usually one-off (individual sensors, gateways) How to deploy gateways at scale? Containerized + automated codestack. 10-20 that can all be updated in a uniform way. Any thought of software management automation with at-scale gateway deployments of LoRa?
- Daniel Fuka mentions he has backed out of LoRa at scale because you need reliable two way communication, range is degraded.
- Can you manage networked gateways easily?
- Cheap Dragino router has auto-update since its OpenWRT. Can establish connection and confirm updates remotely. On a 1x1 with autoupdates happening every few weeks. However, they do die sometimes!
- Final call for LoRa discussion: moving on to networking and network management from routing perspective
Question: are people running networks using one network point or multiple? If multiple, how do you allow sensors to utilize multiple network connections.
- John Porter: one wireless backhaul network with ethernet. Sometimes have radio serial links but are right on the net. Don’t have to deal with those issues of different routing.
- Would multiple paths out still be useful? Wanted? Or not needed at all?
- John Porter, if main node is down, network is down and is sometimes inaccessible. Have considered setting a cellular modem but routing questions make this difficult. Would consider doing something that is switched, RaspPi might check if primary network is up and then switching to a secondary if it isnt. Interested in LoRa but havent set up yet.
- Local Area networks which are tied back into a wider research network. Consider a classic network topology (WAN) that has multiple ways to get back to the network, cellular, p2p network, fiber, etc. routing on that uses classic IPV4 wide area routing protocols. Value in enterprise type routing with these networks. Multiple paths out will be feasible soon
There is a role for network engineering and organized sensor networks. How can we bring network engineering to scientific sensing networks? How to do this is a good question!
- Cross-training w/ graduate students could be useful, even just a little networking.
- Networking has to be a little simpler, though. Wired/Wireless with cell back up works but reverse engineering ad-hoc systems is still difficult. Simpler might be better.
- To what extent are folks thinking about two-way needs? Sensor retasking or algo changes.
- Starlink is cool but is large and requires a lot of energy. Might be too large for the usecase
- new versions of sensors will come out soon that are more efficient? Could be a space to watch. Some people are thinking of provisioning 5G directly from satellites to handsets (not possible now but maybe 5-10 years down the line)
Field has moved from specific science question sensor networks to more general. How can you change the velocity of data and make a network more responsive? Differential timescales of need depending on question.
- Want to be able to deploy systems capable of multiple tasks. Pushing automation might be more valuable
Daniel Fuka suggests FI dog collar (touches on multiple networks. Prefers bluetooth, then wifi, then cellular). GPS dog collar. Nice general target for networking.
- Cow collars for tracking/fencing collars in ag. V cool.
Any applications for drones? LoRa instruments cheap drones. Throw a package onto and network back through LoRa. get 9 km.
- Intel RealSense LiDAR unit. Have to fly at night. Connects to a raspberry pi which spits back data through wifi or possible LoRa with less data. Radar is wallobot (?) with api package and 15 different frequencies.
10/05/2021
Envirosensing Monthly Meeting
Attendees: Joseph Bell, John Porter, Scotty Strachan, Ethan McMahon, Allison Mills, Kristina Fauss, Martha Apple, David Coyle, Guy Lit, Megan Carter, Daniel Fuka
Announcements:
Cluster is exploring collaboration with the Agriculture and Climate cluster relating to wildfire.
Notes:
- Possible collaborations with Agriculture and Climate Cluster relating to wildfire.
* Ag Climate and Envirosensing cluster share a transition zone: in situ monitoring and observation collection for analysis.
* New technology emerging in wildfire monitoring. Where are the knowledge gaps for both researchers and response teams, and how can new sensor technology address them?
* Emerging sensor technology with a wildfire focus: Wildfire detection using cameras placed on a tower with landscape view, smoke detection triggers camera to zoom in and run other algorithms detecting fire/ignition.
* UC San Diego is generating real time fire spread model maps based on local meteorology and ignition points.
- Discussion on spatial locations and the sensors available to assess what we have and where it may work:
* Feasibility to air drop (consumable) sensors into wildfire zones.
* "Sit and wait" - type sensors that chirp periodically and ping on detection of an event.
* Citizen science in the deployment of small sensor networks (dormant, pre-fire).
* LORA logistics and feasibility: canopy penetration, functional in ravines or complex terrain?
* 3D printed weather stations are being produced, build on this idea.
* Immediate Post fire data collection is also important - soil hydrophobicity is important to know immediately after the fire due to direct impact on flood risk and ecosystem recovery; typically measured manually but may be an opportunity for drone deployed sensors.
** Building on above: With drones, instead of using imagery, equip the drone with multispectral radar - before and after precipitation event to capture state change at the surface - however the inexpensive L515 Intel Lidar ($300) sensor is being discontinued
- What sensing would support teams fighting fire?
* Tracking people - including location, but also time since deployed or most recent rest.
* Smart home sprinklers - which would allow the firefighting team (or automatic heat detection) to turn on sprinklers in the community. However this is dependent on electricity & internet (vulnerable during wildfire), a problem of network resilience ~ LoRa, Starnet
* Telemetry challenges: LoRaWAN is a multifrequency gateway, requires 2 way communication, acknowledgement, and often authentication - less capable for the above uses?
* USGS Geohazards program is looking into this, absent from charter is a connection with local infrastructure and municipalities.
09/07/2021
Envirosensing Monthly Meeting
Guest Speaker Amber S. Jones: Techniques for increased automation of aquatic sensor post processing in python.
Attendees: Joseph Bell, John Porter, Scotty Strachan, Kristina Fauss, David Coyle, Guy Lit, Megan Carter, Daniel Fuka, Amber Jones, Jeffery Horsburgh, Bill Teng, Felimon Gayanilo, Matt Bartos, James Gallagher, Adam Kennedy, Renee Brown, Brian Wee
Announcements:
- ESIP Sponsored Bystander Training: September 27, 2021
- Envirosensing and Ag & Climate Cluster may be partnering on sessions in the summer/winter meetings; Envirosensing members are invited to join Ag & Climate Cluster Monthly meetings.
Timestamped Notes:
12:30 Guest presentation begins:
14:00 Challenges of post processing: Sensor Quality Control
14:25 Challenges of sensor quality control: large volume of data, tool accessibility, tedious, large discrepancy of corrections made by technicians - need for reproducibility and standards.
15:30 PY Hydro QC: Python package for quality control available on github, open source.
16:15 Package workflow:
* Rules Based preprocessing - anomalies are identified using basic rules, such as values recorded out of the possible range of a sensor.
* Calibration preprocessing - drift of aquatic sensors disturbs calibration, but this can be fixed in this stage.
* Regression models - ARIMA and LSTM models explored; model estimates and residuals help identify anomalies.
* Anomaly detection and post processing - thresholds determine whether a point is classified as an anomaly; a widening function may be used to classify anomalous events from points; an aggregating function may incorporate detections from multiple models.
** 21:10 Detection Results: In comparison of algorithm detections to labels applied by technicians the ARIMA model performed best; oversensitivity is preferred over undersensitivity - too many detections can be narrowed down by a technician.
* Correction - ARIMA models are used to generate a forward cast from pre-event data and a backcast from post-event data, which are blended to create the model correction; the algorithm is able to forecast diurnal cycles.
25:17 Conclusions and Recommendations:
* Rules based detection is an important component and could be made more complex.
* Drift correction and calibration can be the most time consuming step in manual QC and therefore automation should be increased, which should also increase consistency.
* The model presented was effective and its sensitivity was dependent on threshold settings.
* The correction algorithm was able to approximate diurnal patterns and each event does require review.
* The package can be used in script or notebooks but does require some python experience.
* Package design is modular and is therefore able to accommodate future additions to the QC process.
* Technician review is required for points noted by the model, but far fewer than purely manual QC; technician review is required for corrections, but the model is able to provide corrections a human could not.
27:45 Presentation concludes; Q&A:
Updates on work: Package is now published on the python package index, so it may be installed into a user’s python environment, examples are also available online.
A couple of the models tested were multivariate, but generally did not outperform univariate alternatives.
Calibration events - model assumes technician will supply a date of calibration for drift correction.
Addressing codependence? This could be implemented as rules defining specific “impossible” conditions among multiple variables (at the “Rules” Step).
47:20 New Discussion Topic: Ag & Climate Cluster will be discussing the best ways to observe pre, during, and post wildfire processes; such as merging remotely sensed and in-situ data.
* Ag & Climate Cluster will be hosting wildfire management and mitigation stakeholders over the next three months - what do they need and where can technology fill the gaps?
06/01/2021
Envirosensing Monthly Meeting
Attendees: Matt Bartos, Scotty Strachan, Ethan McMahon, Kristina Fauss, Renee Brown, David Coyle, James Gallagher, Guy Lit, Megan Carter, Daniel Fuka, John Porter
Announcements:
Summer meeting session: 3 people confirmed… a few more in the works
Notes:
- Vendor Discussion
How to bring in vendors to the Cluster?
* Megan: connected with someone from FieldKit —> will send that info to Cluster leadership team
* Some concerns about bringing in vendors, don’t want it to be a sales pitch
* James: what about Sparkfun or adafruit?
* Scotty: likes James’ idea of bringing in open hardware vendors since we’re already working in this space.
* Megan: With respect to FieldKit, I’d be curious to ask them to talk about their vision for the wiki of open-source sensor information they are planning to build. And what they plan to include and who do you want to contribute/most collaborate with?
* John: ask vendors to tell a story.. e.g., what is the story behind a successful sensor? What are the trends you’re seeing?
* Megan: harness the ESIP theme of this year- innovation.. Tell us a story about that.
- Virtual office hours
* Scotty: what if (in lieu of virtual microtutorial project, for now because people are still overloaded with commitments due to covid, etc.) Cluster offered “virtual office hours” that could be broadcast to email list and beyond?
* Ethan: +1 re office hours. It's similar to data stewards. Our cluster would serve as a resource and learn and document what we hear.
* Megan: Agreed on office hours also - will take some effort to recruit people to come ask questions, though, so good to develop a communication plan re: the opportunity. Hardest part is getting people to show up. Recommend keeping it during the regular meeting time to simplify workload
* Renée: like the idea of doing this 1-2x year during telecon.
* Ethan: I agree. We don't know if there are others in the ESIP realm who are interested; if they are interested then why haven't they attended these meetings
* Megan: reach out to your networks
* Ethan: how about we do this within our own Cluster? This could be a place where cluster members could ask each other questions about their problem.
* Megan: That’s a really good point - these calls should benefit you all - not just be a service to others.
* Matt: Setting up a new lab, has to go through digikey (can’t purchase through adafruit or sparkfun)... has questions about what to buy. Second example, for office hours- opened a discord server with the intent of having students talk to each other- but they ended up having questions for him… a perpetual office hours. For office hours, definitely need one or more persons to lead it to help with organizational structure
* Scotty: we also have a Slack channel… we aren’t using it.
* Megan: can use Data Help Desk model… use a form to ask people to submit questions
* Megan: can there be a connection between Virtual Office Hours and microtutorials?
* Scotty: yes
- Summer meeting session:
* 3 people confirmed… a few more in the works
* How much time should we set aside for discussion?
* Daniel has a topic he could talk on if there is a slot available re: PCBs
** He also suggested Nate, founder of SparkFun
** James: @Dan Who do you use for PCBs and population?
** Dan: Makerfabs is our last batch, but using their open design. https://www.makerfabs.com/
- ESIP Lab:
* https://www.esipfed.org/lab/funding-opportunities/2021-spring-esip-lab-request-for-proposals
* Daniel is putting something in
- Daniel Fuka to Everyone (3:59 PM)
- Here is a fun LoRa based soil moisture sensor we are testing and I am adapting to add GPS
- https://www.makerfabs.com/wiki/index.php?title=Lora_Soil_Moisture_Sensor#FAQ
05/04/2021
Envirosensing Monthly Meeting
Attendees: Matt Bartos, Joseph Bell, Scotty Strachan, Ethan McMahon, Kristina Fauss, Renee Brown, Adam Kennedy, Martha Apple, David Coyle, James Gallagher, Guy Lit, Megan Carter, Daniel Fuka
Announcements:
Summer meeting Flash Talks proposal accepted - very short, what are your recent successes? Can be “half-baked” - there will be an email going out soon for presenters, topics can be broadly related from physical sensors to data quality control (but EnviroSensing generally not focused on satellites sensors)
Please submit presenters for the ESIP Summer Meeting EnviroSensing Cluster session!
Two complementary speakers who have not been to ESIP before (Megan)
Notes:
- General Discussion
* Shifts in funding are coming from the federal level with the new administration, expect new govt budget on climate change, covid, and the economy and air water budget changes - opportunities to fund sensors to be placed outside of typical natural pristine areas, like metropolitan areas.
* Public summaries of budget changes published, Ethann McMahon will send for distribution to cluster/ESIP listserve or slack: Biden’s budget request (https://www.whitehouse.gov/wp-content/uploads/2021/04/FY2022-Discretionary-Request.pdf) (58 page version). 3 page summary (https://www.whitehouse.gov/wp-content/uploads/2021/04/FY2022-Discretionary-Request-Press-Release.pdf)
- Telemetry Discussion
* Mixed wireless and wired networking (for research) needs help to scale up
* New opportunities opening in the fields of telemetry and end-to-end data flow for our community to engage on these topics and contribute expertise; there is funding available for broadband connectivity for education, hard to reach communities, and science - these opportunities tend to be missing people with expertise like that of Envirosensing members
* Adam shared updates about Andrews wireless: Wired Internet has been down since last year’s wildfire, was previously T1 line, but has upgraded to Ubiquiti AirFiber line with 4’ dishes (wireless communication), but now fighting to defend unlicensed bands, it’s exciting that 6ghz is being added to unlicensed channels. How do we remain on a clean frequency on the unlicensed band? How do we generate revenue to support the service (which is expensive to maintain)? Is using low-power Ubiquiti radios (2.4 Ghz + 900 Mhz) for internal network connections that are only turned on when needed to save power, especially in the winter, feasible? Options for up sites with to 75 people during the summer, all of our telemetry passes through the system?
* David: Ondas.com https://www.ondas.com/
* Joe: GOES channels are going private (GOES is a satellite-based telemetry)
* Renee: There is an extensive wifi network at Sevilleta through Ubiquiti, there is also good line of sight which allows good wireless links and good bandwidth at the field station. The water quality monitor sites are in remote places with no cell service - impractical to set up a wifi network there. In Antarctica, Iridium is the only thing that works. Using Ubiquiti mostly now, standard wifi, radios, recently converted 2g to 5g - Are you using standard wifi vs proprietary for ubiquiti? Anyone have experience with this?
* Adam: Just bought some AC’s Airview M2 nanostation , we don't have any concern with anything Ubiquiti might change to - all is compatible within Ubiquiti, biggest benefit is to go down to 5200 frequency as it is cleaner => recommendation get a couple and see how they work
* Renee/Adam: Wired connection is vulnerable to physical damage (fire, cut). Wifi is a good backup but we need better than 1g for today’s data expectations
* Scotty: Starlink has advantages, particularly if Starlink starts including mobile applications
* David Coyle: Starlink (unlike Iridium) does not have many fail safes/redundancy
* Scotty: Starlink goes down sometimes, but when it’s up it works really well, could work well as a secondary/tertiary internet connection depending on the application
* Dan: Lora is 250 bytes per second
04/06/2021
Envirosensing Monthly Meeting
Attendees: Scotty Strachan, Kristina Fauss, Daniel Fuka, James Gallagher, John Porter, Felimon Gayanilo, Ethan McMan, David Coyle, Joseph Bell, Martha Apple, Megan Carter, Renee Brown, Jay Pearlman
Announcements:
There will be a data help desk - watch ESIP channels, all help is welcome.”Curious about the Data Help Desk at EGU21? Check out http://bit.ly/DataHelpEGU21.”
Summer session proposals by April 16 “ESIP Summer Meeting details: https://2021esipsummermeeting.sched.com/info (registration open now and sessions accepted now through 4/16)”
No envirosensing AGU session this year - wait till next year/AGU is back in person, but feel free to reach out if you have questions or would like support in your own AGU activities
Notes:
- AGU Discussion
* AGU accessibility does not compare online vs in person
* Unclear if AGU will be fully virtual or not
* MacGyver session at AGU might be a good place for Envirosensing member topics, Envirosensing cluster will not be organizing a session
- Video Microtutorials
* How to continue knowledge transfer/teaching/discovery, building on momentum from the world moving virtual
* Are there existing tools/templates/guidance to produce?
* Work towards developing a task matrix
* Important to include in microtutorials: Order of operations, plan or strategy relating to the instruments, transferring tacit knowledge, knowledge of hazards
* Address the problem: saturation of poorly organized material, not easy to get what you need (accurate, detailed quality) on a quick pass
* How to classify, order, tag, summarize, so that people can decide if the tutorial suits their needs quickly
* What’s been done in the space? Resources on instructions/workflow to produce microtutorials, let’s find these and pool into a cluster resource
* Renee - Virtual Field Project: OBFS organization for biological field stations, has NSF grant for virtual field trips for their field stations and sites to expose students to different ecosystems
* Consider ways to include accountability for intellectual property, aspects of authorship
{in the chat: So are we talking about a curated list? Or our own videos? -> both are being considered}
* How to motivate researchers/technical experts -> CV items, elements of peer review? Items deserving of recognition, producing digital products within the scientific community
** Megan - “Credit for non traditional contributions and roles”
* Youtube videos have commercials - downside for instructional/informational video
** Megan - Alternative platforms, means to getting a doi: https://casrai.org; CRediT - Contributor Roles Taxonomy but not a place to view a resource
* Concept of “Folksonomy” - allow people to tag their own items
* Megan - “Also consider talking to the Data Management Training Clearinghouse - maybe inviting Karl Benedict or Nancy Hoebelheinrich in. Both would probably have a lot of thoughts here.”
* John Porter - SCIVEE.tv used to be a YouTube-like site aimed specifically at scientific presentations, worked well but vanished one day when the funding ran out
- ESIP Summer Meeting, Envirosensing Session
* Open envirosensing session - what are you doing/excited about? Need speakers
* “Half baked” / Ignite concept - 15 min talks, flash - 3-5 min talks
* There will be 90 min total for the session
* Cross cluster - Pull in speaker from your "nearest neighbor" ESIP cluster; possible Ag-Climate
03/02/2021
Microtutorials and Oceans Best Practices
Attendees: Scotty Strachan, Kristina Fauss, Jay Pearlman, Matt Bartos, James Gallagher, Guy Lit, David Coyle, Renee Brown, Megan Carter, Christine Renee Gregg, Joseph Bell, Bar, Ethan McMahon
Announcements:
Joseph Bell is a new co-chair for the Envirosensing cluster.
Call for sessions for the summer meeting coming out soon.
Summer meeting will be virtual, likely July 19-23.
Notes:
- Video Microtutorials
* Determine best practices on production from youtube.
* Cluster discussion, who wants to step up to take on some of the pieces to developing microtutorials?
Community driven topic - what do you want to do next?
Test cases, best practices, to implementation.
* MicrogranPs from ESIP could support this.
- Oceans best practices - Jay Pearlman
* Presently working on a microtutorial for unbaited fish monitor - does it work, why does it work, what should you never do? - important to include "worst practices"
* Oceans Best Practices system: Way to foster quality control, consolidate best practice materials, open access
* Before recording, a promising method would have to be reviewed, accepted and deployed by multiple organizations within the community; ensures quality and consistency
Q across operations
Interoperabiltiy of data
* Data traceability and transparency
* Ocean Best Practices viewed as a repository from the beginning - under IODE, “Data Exchange” => ~1,200 best practices across oceans community, very broad from sensing to aquaculture
* The term "Best practices" can add a barrier to submitting -> “manuals” and “standard operating procedures” are more common terms
* Adding academic credibility -> peer review process and publication
* Ocean teacher global academy - training the trainer, working with established organizations, we don’t create the videos, we host them and make people aware of them
* No exclusivity, anything in the repository is open access
* Discovery of specific microtutorials may be easier on a specific site than youtube
* Expand into visual immersion techniques, CAD type designs -> giving tutorials in CAD/3d design
* Long term goal for a library of virtual training
* Concept for disk distribution in regions with low internet access
- Questions?
* Renee - How is Oceans funded?
** Jay - European commission grant Atlantis, NSF funding started the foundation, UNESCO present funder of operations and staff, tech development is done under grants; a lot of volunteer support
* Renee - Most of us do not work in marine environments, how do terrestrial systems work fit in?
** Jay - Newly acquired grant for Arctic expansion - sea ice, land, and atmosphere; could advocate a “land” best practice system as a cluster
- Winter meeting recap
* Range of subjects to address through microtutorials
* Wide range of experience, capability, and focus represented in Envirosensing cluster
* Envirosensing has collective knowledge among members on how to make these microtutorials
* Need good metadata associated with videos to improve discoverability
01/05/2021
Envirosensing Cluster meeting, Pre ESIP Winter Meeting
Attendees: Martha Apple, Daniel Fuka, Ethan McMahon, James Gallagher, Joseph Bell, Matt Bartos, Megan Carter, Mike Daniels, Steve Young, Renee Brown , Scotty Strachan, Kristina Fauss
Recording Time Stamps:
0:00 Introductions
11:00 Cluster Announcements
11:35 AGU 2020 Recap
23:20 AGU Recap blends into discussion on Sharing Content
28:45 Winter Meeting Working Session Discussion & Microtutorials
54:53 Closeout
Notes:
- Recap & Discussion of AGU Participation
* AGU was virtual this year for Covid 19 Pandemic
* ESIP Envirosensing had its first proposal for sessions in AGU this year, which was merged with a similar proposed session
* Envirosensing session included both an oral and E-lighting poster session, which worked out to be quite similar in the virtual environment
* AGU session were more Q&A in the online format
* Matt - Envirosensing cluster could run a research expo building on AGU format
* E Lightening vs standard oral presentation format at AGU: The E lightning style AGU sessions promote more interaction, more of a community feel
- Planning for winter working session
* Could be a brainstorming exercise for virtual content, Asynchronous “Micro-Tutorials”, what works? Presentation style, recording style? How to structure a demo? Best practices and parameters for virtual demos?
* Development of mini tutorial videos for low cost, open source technology
* Brainstorm in breakout groups during winter meeting: What content to prioritize, video length, target audience(s), possible applications
- Ideas for microtutorials
* Provide members a platform to share valuable materials/info in one searchable place
* Share common techniques
* Balance between broad and technical
* Audience may be ESIP community, scientific community, students, technical hobbyists
- Additional Points
* Renee - yearly webinar series coming up, could feature participants
* ESIP lab proposals due Jan 29
* Do any members have any experience in media production?
10/01/2019
Recording and Transcript of Session
ESIP Envirosensing Notes - 10/01/2019
Attendees - Scotty Strachan, Connor Scully-Allison, Megan Carter, John Porter, Joseph Bell, Felimon Gayanilo
Notes:
- Scotty on National Research Platform
* There is a lot of opportunities for funding and collaboration
- NRP Workshop
* NSF Funded * Support projects working on cloud-like technologies
- Workshop committee
* Universities, Internet2 * Usually heads of information computing * CIOs * Being driven by technology and infrastructure advisors
- Agenda
* Scotty highlighted the importance of internet of things in disrupting wireless research
- Internet of wild things: Driven workflows
* We can plug in many sensors on the ground * Scotty received questions about the Delaware River Basin Pilot * Many of these networks are very complex and coming very soon * Software side * Creating a software pipeline to scale and automate these things * 5TB/day of data
- NSF OAC
* Advanced Cyberinfrastructure Grants * Important to be driven by Scientists and Science Problems * A good opportunity to build infrastructure for small scale research and scale up existing research * Cyberinfrastructure is growing in focus across NSF * NSF has finally realized that CI is an ecosystem (not just HPC) * Maps very well to the themes of ESIP * NSF is aware that instruments and observatories include sensor networks * But they are not looking very closely at grassroots network (yet) * Science can transform how technology is being implemented * Regional networks are pretty big for research and education * Emerging people networks * The ESIP model could work for them * NSF CaRCC * They have various working groups * They want to become an ESIP-like community
08/06/2019
ESIP Envirosensing Notes - 08/06/2019
Attendees - Scotty Strachan, Connor Scully-Allison, Dawn Henshaw, Renee Brown, Felimon, David Coyle
Notes:
- Renee’s Thoughts
* Well attended * About 50 People * Very diverse group of people * Might have been one of our best showings at ESIP * Three topics of interest * Visualization Dashboards * Metadata Capture * Specifically in regards to sensor related metadata * Data Archival * Raw sensor data
- Scotty’s Thoughts
* James and Martha spoke on using LoRaWan * In alpine environments * Renee’s talk on McMurdo Dry Valleys
- General Feedback
* Felimon * Agrees with earlier statements, likes topic and thought this was relevant * Suggests a survey or ranking of vis tools * Was looking for more on long term observerties * How do they handle this data that has been captured for many years * This group may want to explore data management for long term exploratories * Don * Really good presentations * Well attended * We should include LoRaWan in this summary in some capacity * That was especially of interest to people * David Coyle * (RE: LoRa) Plans to work with CMU to get some boxes for more integration into USGS pipelines and try them out in the field. * Changes to three takeaways * Three takeaways have been altered to combine current 2 & 3 and add LoRa as #3 * Open Hardware * Felimon suggests us interacting with open/hardware community * Scotty requests some names from people who participate in these community organizations. * Citizen Science and their own small Envirosensing projects * Summer Meeting of 2020 - Session ideas * Maintain a long meeting through multiple sessions in a row * Each one organized around a theme * And one final one for internal cluster town hall * Don suggests an unconference for this last town hall
06/04/2019
ESIP Envirosensing Notes - 06/04/2019
Attendees - Scotty Strachan, Aaron Sweeney, Joseph Bell, Connor Scully-Allison, David Coyle, Ethan McMahon, John Porter, Matt Bartos, Megan Carter, Renee F. Brown, Lindsey Rustand
Notes:
- Welcomes and Introductions
- Renee and Scotty Solicit for Talks at ESIP summer
* Formal announcement going out on listserv (in next week approx)
- R&S Comment on Frontiers in Environmental Sensing Innovation workshop
* Organized by Cody Youngbull * Cody is a hardware geek/ optical physicist * Flathead lake is creating a center of excellence for sensors and aquatic research * Scotty gave a talk there (pitching ESIP and our cluster) * It was not just about lakes tons of IOT and future thinking discussions there
- Lindsey Rustad’s Talk
* Lindsey expresses appreciation for the envirosensing group * Lindsey introduces herself: * Forest Ecologist * 80s and 90s: she started with sensors * Mary Martin was invaluable for helping to start up these sensors * Enmeshed in sensors for the entirety of her career * Presentation - WaterVIZ * Making Data Available to a broader audience * Water visualization and sonification tool * How it works * Hubbard Brook * 1955 Center of Hydrological Research * Bowl-shaped valley * Data transmitted via high power radios to a base station * This data goes to visualization via charts and graphs along with artistic sonification/visualization * Xavier Cortata - Produced artistic representation of all aspects of water cycle * Data feeds the visualization * Number of particles and speed are determined by live data * “Near real time” data * Sonification aspect * Data drives a musical interpretation of the water cycle * Marty Quinn * Musician who created the sonification * Multiple years of data have been sonified * 15mins for the whole year * Last hour of music sonified can also be heard * Synchronized music and visualization * Working with neuroscientists and exploring how we can engage auditory, visual and reasoning parts of brain\ * waterviz.org * Website for all of this
05/07/2019
Agenda for May, 7 2019 (times in EST)
Attendees: Connor, Scotty, Megan, Matt, Renee, Don, Joseph Bell, David Coyle, John Porter, Felimon Gayanilo, Jerry Butch
Notes
- Introductions
- David Coyle’s Presentation
* IT Specialist, USGS * USGS NGWOS LPWAN Experiment * Wanted to experiment with LPWAN * Next generation water observing system * Looking into off-the-shelf solutions for this system * Studying the Delaware River Basin * LoRa * Has the best range of any other existing technology * Enables sparse sensor placement * Low power long range * Network features with end-to-end security * Data is encrypted at rest and in transit * How does it accomplish this? * Multiple classes of LoRaWAN * Costs * $500-300 with sensors * $75 with a temperature probe * Deployment is very easy with these solutions and inexpensive * LoRa is Open Source under MIT license * Developer kits can be purchased to build from the ground up * LoRa Gateway OS * Linux based * LoRA Gatweway bridge * Uses MQTT * Abstracts a lot of the tedious packagement code with UDP * LoRa server * Duplication and Failover of servers * Real-time failover * Risks * Risks grow as we scale to industrial scale * POC Purgatory: It never gets of the ground * It’s not about “Tech” - it’s about business transformation * Trade offs of building your own network * LoRaWAN Requirements * Avoid feature creep * Small data payloads and a small number of sensors * Heterogeneous systems are possible (cell and lorawan) * Advantages * Can run an application closer to the geographic area * Can you elaborate on how LoRaWAN enables this? * Q&A * Scotty asks about standards * David references the CayenneLPP (The Things Network) * He indicates that SensorML is maybe too big an encoding for this level of the infrastructure * Scotty asks about use for such a use case as in the Great Basin * David indicates that the architecture can work really well for the connection of granular sensors * Scotty indicates he could install a LoRaWAN gateway offer access as a service.
03/05/2019
Recording and Transcript of Session
ESIP Envirosensing Notes - 03/05/2019
Attendees - Scotty Strachan, John Porter, Matt Bartos, Connor Scully-Allison, Mike Daniels
Notes:*
- The was a paper
- What is the value of this?
- Precipitation data is becoming sparse
- Most sensors can only measure a point
- The solution
- One option is vehicles
- They have sensors which can transfer data back
- Windshield wipers as precipitation sensors
- Study area was Ann Arbor
- Map of single are trip shows a use case narrative and how it works
- Use a video to determine the ground truth
- Wiper Intensity does not significantly correlate to rainfall intensity
- However wiper performs significantly better at determining the binary rainfall state
- Produce a data product of combined radar and wiper data
- Prediction performance was better than radar alone
- Q&A:
- Mike & Connor ask questions
- Mike: How will you get data from Ford?
- Connor: What was the computational intensity of this data synthesis? And, do you anticipate that this computation will scale well?
- Matt: Its very computationally intense and may not scale well with more vehicles. It is easily parallelizable however due to the large time scales of the radar data
- Scotty talks about the potential future viability of accessing and aggregating data from connected and smart vehicles.
- Presentation session in ESIP Summer Session (Scotty and Connor will work on)
- Deadline: April 26th
02/05/2019
Agenda for February, 5 2019 (times in EST):
Introductions - 5:00 pm (5 min)
ESIP Winter Meeting Review and Report - 5:05 (approx. 10 min) (Mike and Connor)
Project Reporting - 5:15 (approx. 5 min)
Discussion of 2019 Cluster Agenda - 5:20 (Remaining Time)
Attendees: Connor, Scotty, Megan Carter, Matt Bartos, Renee, Roger Tyler, Don Henshaw, Ethan McMahon,
Notes
Megan expresses support of this cluster
- Indicates that this one most actively collaborates without her involvement
Introductions
- Scotty suggests to Ethan opportunity for a future potential meeting topic.
ESIP Winter Recap
- Connor Reviewed, Much of the envirosensing cluster was gone but well represented in the poster session with Mike and Connor
- Matt indicates that Mike had positive feedback on Sensor DAT
- A few parties are interested in the workflow and technology
Project Reporting
- Progress has been made by Matt on using machine learning for Quality Control
- He has a server that uses Graphana for visualization
- The results are very impressive
- Connor is exploring the use of source files to aid the ingestion workflow for Renee's data
Year’s Agenda
- Scotty and Renee are attending the same workshop
- Matt mentions PhD candidate collaborator in New Mexico
- Suggests him for a talk
- Scotty expresses interest
- Matt mentions that he would like to give a talk on a paper he did concerning windshield wipers and water
- Don
- Recommends talk from Scotty/Renee related to what comes out of their New Mexico workshop
- Scotty is hopeful about the opportunities for the year
9/04/2018
Attendees:
Scotty Strachan
Matt Bartos
Ethan McMahon
Mark Gahler
Mike Daniels
Renee Brown
Cody Youngbull
John Porter
Connor Scully-Allison
Don Henshaw
Martha Apple
Vasu Kilaru
Jane Wyngaard
Stephanie Schmidt
Vincent Moriarty
- Recap of summer meeting
- SensorDat update
- Presentation: Cody Youngbull
- SensorSpace: A Cutting-Edge Facility for Environmental Sensor Design, Prototyping, Production, and Deployment
- Oldest continuously operating biological field station in the US
- Montana underserved area; field station marine laboratories grants
- Sensorspace
- Four major areas: manufacturing, electronics, optics, microfluidics
- Universities often do not have adequate electronics/optics shops. Sensorspace fills this gap.
- Full-service; not just manufacturing.
- Cater almost exclusively to ecologists. Build prototypes.
- Lower costs for nonprofits/academia
- Help write grants
- Electronics: Make custom low-power boards.
- Optics: Sondes, Dissolved oxygen, dissolved CO2 sensors.
- Instrument considerations: e.g. vandalism-proof; shotgun-proof enclosures.
- World's only continuous-flow digital PCR machine.
- Digital-droplet quantification: instead of a analog detector, use multiple binary detectors.
- Digital-droplet quantification enabled by microfluidics: can create tiny droplets, then count presence/absence of analytes in each drop.
- Making environmental sensor networks summer course.
- Setting up a LoraWAN sensor network
Scotty: Ongoing trend, sensor development and sensor explosion. What are you teaching students about best practices for documenting site conditions and managing data?
Jane: Do you open-source your sensor designs? Cody: Yes
John: What is the "ideal student" for your summer course? Are you looking for advanced graduate students? Faculty? Cody: Yes, we need faculty. Contact me.
Mike: Which IoT platform are you using? Are you deploying to things like standards, controlled vocabularies and ontologies for metadata, variable naming, etc.? Cody: MySQL, node red, grafana.
Mark: When will the DO and CO2 sensors you're developing be available for purchase? How can we find out about it? Cody: In beta stage currently. Under $1000 for both logger and sensor. Beta list is open. How to do this? Set up an external company (LLC) with a research contract. Transfer 100% of profit to university.
5/01/2018
Attendees: Matt Bartos Janet Fredericks Mike Daniels Renee Brown Scotty Strachan
Scotty: NEVCAN network
- Network in the great basin, funded from 2009-2013
- Team of people (dozen co-PIs) interest in the intersection of water/plants/geography in the great basin
- I came from the paleoclimate side; came on to lead the field team
- Ten core sites, five additional sites
- Science questions: change in ecohydrologic (vegetation) processes across valley-mountain gradients
- All major great basin vegetation zones
- Standardized WX weather sensor packages (WRCC)
- Site-specific experimental deployments
- Connected by digital data network
- Weather and basic soil profile
- 3B sensor observations
- 4M camera images
- Sites in Snake Range, Sheep Range
- Sagebrush zone, pinyon-juniper zone, montane zone, subalpine zone
- Data acquisition:
- Campbell scientific dataloggers
- 3 second scan, 1/10/60 min tables
- 50+ variables
- 5-min upload, 1-yr local storage
- PTZ cameras, 20 scenes
- 30-min PhenoCam FTP upload
- 30-min tree sap flow and diameter
- Networks
- Beyond the reach of cellular
- 4-100 km terrestrial wireless
- 12-30 Mbps station links
- TCP/IP, private OSFP
- Power systems
- All standalone PV
- Significantly overbuilt
- Battery protection is key
- MPTT chargers
- Put solar panels higher than max snow height
- 60 degree shed angle
- Parallel-redundant electrical generation and storage
- Ventilation/condensation prevention using case fan
- Tiered voltage cutoffs
- Heating/cooling cuts off first, then cameras, then mission critical devices (comms/dataloggers)
- Adopted a USGS site that was going to be abandoned
- Management software
- Simplifies/records field maintenance
- Publications
- Testing the daily PRISM air temperature model on semiarid mountain slopes
- Biases in model based on source observations
- Precipitation and conifer response in semiarid mountains
- Rate at which precipitation arrives significantly affects soil moisture
- Testing the daily PRISM air temperature model on semiarid mountain slopes
- The future
- Keep sites running
- Get endowment funding
- Partner with good collabs
- Keep improving data flow
Mike: Maintenance: cameras, riming issues. How often do you have to visit the sites? Scotty: About twice a year (mid to end of June, and then in October). Cameras definitely get rimed, but we also get direct sunlight.
Mike: What does QAQC entail. Scotty: Mostly range checks, etc. No trend analysis right now, although it is becoming more important as we approach 10 years.
4/03/2018
Attendees:
Matt Bartos
Don Henshaw
Felimon Gayanilo
John Porter
Mike Daniels
Renee F. Brown
Scotty Strachan
Vincent Moriarty
Corinna Gries
Janet Fredericks
Ethan McMahon
Envirosensing Summer 2018 Meeting Planning
- ideas on what they would like to see us do
- Additional standards
- two IOT protocols
- LWMQM and Coapp
- Coapp very compressed data transmission protocol
- Additional standards
- Standards for manual user corrections of data
- corrections and or annotations
- without necessarily changing the data itself
- markup language for manual user corrections of data
- ways to annotate at the data point level
- richer way of annotating individual data points
- could be done with SQL or in NOSQL database
- Standards for manual user corrections of data
- how does this relate to theme?
- Automated quality control
- broader category of data quality
- CSSI elements to frameworks
- standardized containers for QA and metadata capture
- e.g., for iButton network
- pass on to level 1 automated QA tests
- simple, not as complex as GCE toolbox
- help prepare more reliable datasets
- Scotty working on proposal
- would sensorML be helpful for ingestion
- Mike also working on JsonLD extensions for real-time data sources
- outline region - get summary of operational sensors, sampling rates etc.
- how does this relate to theme?
- How does jsonLD compare with sensorML?
- schema.org used by google for finding related items
- want to extend to data searches
- json LD stands for Linked Data
- schema.org relatively small set of ways to link data
- would need extensions for handling datasets
- in EarthCube focuses on data centers
- working in new proposals on going to parameter level
- Janet - sensorML is more content-rich
- very broad!
- want to extend work to come up with profiles manufactures can populate
- archiving real time data
- how often to update in portals etc. - best practices
- connecting real time streams to archives
- How does jsonLD compare with sensorML?
- data descriptors for quality assurance and control
- could we have a session on that topic?
- what standard does (or doesn't) do for sensors
- what are best practices, recommendations for new users
- who would like to volunteer to talk about standards - metadata interoperability
- would be good to hear about Coapp and ISPO
- constraint application protocol - COAP
- IPSO protocol for describing connected things including actuators
- may not be mature, but has large user base...
- very limited vocabulary for sensors
- metadata capture, quality control, reporting quality control process
- existing standards or new needed?
- Smart Cities community is not yet at this point in terms of data interoperability
- interested in monetizing
- we have something to offer them
- SWEET ontology - link to sensors not clear
- Socioeconomic Value theme?
- often collect data for specific purposes
- but additional value may come from use by others
- or could lead them wrong if metadata not complete enough to explain what has been done
- getting better handle on collection of metadata
- cautionary tales - what happens if we DON'T have the right tools for metadata
- SNOTEL temperature data is a major cautionary tale in this way - Scotty
- documentation is all manual
- Smart Cities is huge socieconomic impact
- tie to events
- floods in Boulder - models underestimated water content of clouds
- Feedback on robust data corrections might be an interesting project (ties in above)
- don't have good ways to track transformations could have one session more on annotation standards and another more on QA/QC
- could be two complementary sessions
- workflows then annotations and standards
- if you are correcting data by adding offset, that is QAQC thing....
- some drone folks think existing metadata standards are too onerous - perhaps
- better to have big "bag of words"
- more flexible standards
- how signal processing techniques can uncover structure from unstructured data
- e.g., Latent semantic indexing with frequency composition
- similar idea, but for metadata
- document, key values
- could do a good presentation on that - but wear "mad scientist" hat....
- hard to fit things in in field
- there is ESIP funding for speakers in breakout sessions - not sure how to get that!
- would be good to get Kirk to one of these sessions
- have until the end of the month to do this
- being part of ESIP governance helps
- cluster can propose serial sections
- QA workflows
- metadata standards (and unstandards)
- Scotty will put together and send to email list
- would not mind 10-15 min per talk sequences with extended discussion
- short presentations are good to start with - ideally WITHOUT conclusions
- focused talks on specific standards
- Can put directly into web site - long time to edit
- data descriptors for quality assurance and control
3/06/2018
Attendees
Matt Bartos
John Porter
Aaron Botnick
Bruce Caron
Mike Daniels
Scotty Strachan
Jason Karl
Erin Robinson
Corinna Gries
Don Henshaw
Felimon Gayanilo
Vincent Moriarty
Presentation
- Virginia Coast Reserve LTER
- Data collected by individual researchers; metadata ingested; automated sensors
- Weather station: combination temperature, humidity, wind, rain.
- Several tide stations using radar. Shared in real-time with NOAA.
- Ground water monitoring station
- 900 MHz IP, 2.4 GHz WiFi
- Major stations: Broadwater tower, Machipongo station
- Archiving and publishing data:
- Level 0: raw data
- Level 1: processed data
- Level 2: value added data
- Data collection: PC with LoggerNet to Unix Computer running SAS/R
- Something missing from automater QAQC: user corrections/flagging (e.g. tipping bucket clogged).
- Right now, hand-coded php with forms for user input.
- User input entered into a database that can be queried.
- Code to implement user corrections automatically generated (e.g. using R code)
Questions
- Felimon: any documentation on how you do QC?
- John: Documentation is the actual code template used to do the post-processing.
- Felimon: Look into QARTOD for how specific variables should be QAQC'd: https://ioos.noaa.gov/project/qartod/
- Don: Campbell, John L., Rustad, Lindsey E., Porter, John H., Taylor, Jeffrey R., Dereszynski, Ethan W., Shanley, James B., Gries, Corinna, Henshaw, Donald L., Martin, Mary E., Sheldon, Wade M., Boose, Emery R (2013) Quantity is Nothing without Quality: Automated QA/QC for Streaming Environmental Sensor Data (Pub. No: 4825, Journal Article)
- Mike: Does user QAQC require people to be constantly checking the data?
- John: User input usually a result of checking on the equipment.
- Mike: Do you use controlled vocabularies, etc.?
- John: Generally not. Many stations go back to the 1980's. Some stations have these vocabularies, and measurements from other stations are tied to these.
- Corinna: @Felimon: Are you working with NCEI?
- Felimon: Yes, we do upload to NCEI.
ESIP Summer Meeting Call for Sessions:
2/06/2018
Attendees
Don Henshaw
John Porter
Scotty Strachan
Vincent Moriarty
Mike Daniels
Renee Brown
Aaron Botnick
Corinna Gries
Introduction
Recap of winter meeting
- About 20 participants, from organizations including CUAHSI, EPA, etc.
- Talked about sensor data workflow.
- Gave survey on sensor data workflow.
- Is this survey a useful product?
Comments on survey
- Renee: In favor of distributing to a broader audience (LTERs, LBFS).
- Renee: Few people using trusted repositories.
- Mike: CHORDS trying to make use of trusted repositories more accessible.
- Renee: Everybody is using different solutions. What's use in developing own solutions, as opposed to standards?
- Corinna: DataONE is not a trusted repository; it is an aggregator.
- Corinna: Coming up with a data model that works for everybody is too ambitious. Sometimes prevents people from properly using them.
- John: Why so many solutions?: scale and resources, similarity of objectives, things change very fast.
- John: As for vocabulary, scaling can be an issue. Process can become very difficult if number of terms is too large.
- Don: Are we seeing any commonality of approaches?
- Don: Were we supposed to be identifying bottlenecks in the survey?
- Scotty: Some bottlenecks include: QC flagging process. Import configuration process. Overall development of modified data model.
- John: Possibly only use a subset of the survey as a planning tool.
- Scotty: Could be useful for a team of, say, civil engineers who are trying to write a proposal and figure out what tools they need. Most will not be thinking of trusted repositories, e.g.
- Mike: In NSF community, more emphasis on taking more data and more measurements, as opposed to what is done with those measurements (are they well-documented, etc.).
- Corinna: Don't give as many choices on survey, may be to constraining. PI's won't be planning at this level of detail, e.g.
- Aaron: The more generic a tool is, the harder it is to learn.
- Conclusions: trim down diagram to get rid of specifics. Focus on general technologies used.
- ODM seems to be taking ODM in a lot of new directions---goal of end-to-end sensor data management?
- Corinna: could be too broad. Hard to get data in, hard to get data out.
- John: Standardized response blocks. Smaller number than in current survey.
- Headers instead of diagrams.
12/05/2017
Attendees
Matt Bartos
Scotty Strachan
Don Henshaw
Paul Celicourt
Renee F. Brown
Jane Wyngaard
Discussion of plans for the winter meeting
- Scheduled a working session for the cluster with John Porter
- Use this session to sit down, talk about workflows for end-to-end sensor data management
- Everyone will hopefully have a cross-section of where high-performance/gaps in performance
- Don: See what software people are using in the field, what loggers are used, how data is sent back to base station?
- Scotty: Have a system diagram available for everyone to show their workflows and see where they can use particular tools.
- Paul: Is the working session going to be interactive?
- Scotty: Depends on how many people show up. If there are 20 people, e.g., have a brief synopsis, have an exercise in which people explore different architectures and see how their workflow fits in.
- Else if there are very few people, go around and describe for each user.
- Don: Have you described that in the abstract?
- Scotty: we kept it general.
- Don: We should say "attendees should come prepared for participation"
- Scotty: Concurrent with OGC, metadata carpentry, earth data science analytics, valuables consortium session (and one more)
We we should do this next year:
- Alternating monthly phone calls:
- (i) cluster work, potential publications
- (ii) Guest speakers present 20-30 minutes on an aspect of their research
- Mature workflows
- Completely different workspaces. Preferably earth sciences, but different.
- Renee: Would like to see NEON again.
- Scotty: Celeste had sensor network destroyed by wildfires (had insurance); would be interesting to see how they rebuild it.
- Renee: What topics would you all be interested in?
- Don: Touch base with the group and see if they have any updates.
Other things:
- Jane: How does the winter session fit into the FAIR data concept.
- Scotty: Look at how well workflow fits into the FAIR concept. Large strategic impact is FAIR concept; tactical focus is how to do you manage increasing volumes of data with decreasing person-hours.
- Jane: OGC will be at this winter meeting.
- Scotty: Many groups are using a modified ODM CUAHSI data model. Similar to what Jane is using.
- Jane: Can likely find speakers in RDA groups.
- Don: Sustainable data management group has an ESIP wiki with notes on FAIR data.
Skip January telecon; reconvene in February.
11/07/2017
Attendees
Ethan McMahon
Felimon Gayanilo
Janet Fredericks
Renee F Brown
Colin Smith
Corinna Gries
Don Henshaw
Matt Bartos
- Preliminary remarks
- senior project on CHORDS at University Nevada Reno
- Continuation of discussion on cluster goals: summary
- Time to start summarizing key software tools, seeing how they can fit together and where the gaps are
- If someone has to spin up a new sensor network from scratch, how could they use a set of tools to streamline the process
- How to have information be interoperable?
- Submitted a working session to the ESIP Winter Meeting
- To plan for taking first step forward over the next couple years
- Co-organizers are Scotty and John Porter
- Renee stepping up to be a new co-chair for the Envirosensing Group
- What would you suggest to a brand-new network manager on what they should do, what tools should they use?
- Literature review for each of the main boxes
- Corinna: most interesting part to talk about are the arrows.
- Define what the arrows represent
- How many arrows could sensorML satisfy?
- More MLs than sensorML:
- WaterML, EML
- Data Turbine as a vehicle?
- Where would it fit in, does it fit in?
- Hasn't been used, because Campbell provides all the services in the management interface.
- Has anyone done a study on how long it takes to set up and configure a sensor data management system?
- There are some aspects of QC that can slow down transfer of streaming sensor data to the database
- OGC SOS and OpenSensorHub handles many of the things on the diagram
- Ethan McMahon: Is there a general diagram?
Take homes:
- Scotty: the Cluster needs to define a topic objective with specific goals for the next 1-2 years. We have been spending most of our effort looking at software tools and workflows for sensor network scientists and managers. We could detail some of this, enhancing the resources already available in the Best Practices. Our goal should be to reduce IM time spent on managing data, to increase scalability and focus more on quality. Streamlining the setup/config/monitoring processes is key, while including community standards and external connectivity.
- Updating a workflow model graphic would give helpful guidance, modify Scotty's?
- Matt: a literature review of each sector of the workflow (tools, practices) would be very helpful, and we could add those to new Guide documents.
- Corinna: the challenge is in defining what the flows between software elements consist of (formats/contents/sizes/standards). The group could spend worthwhile time on that.
- Matt: SensorML as a primary communications medium between softwares - how much does it already cover, and could it use improvement?
- We could take a "use case approach", where as an exercise we have IM's fill in the blanks on a model graphic with their solutions, highlight the bottlenecks, and make suggestions or wish lists. - Felimon: we should have a presentation by 52north, they may have solved many of these problems or have good component solutions?
- Other resources to explore as we set up this topic: OpenSensorHub, SensorML, EML, WaterML, OGC-SOS (standards)
10/03/2017
Attendees
(Incomplete list)
Don Henshaw
Scotty Strachan
Matt Bartos
Janet Fredericks
Ethan McMahon
Jane Wyngaard
John Porter
Felimon Gayanilo
Vasu Kilaru
- Intro - Scotty
- software framework for data management
- working on proposal
- but could have community projects
- diagram shared by Scotty
- where does FRAMEWORK fit in - dashed line
- into structured database a la CUASHI
- GCE toolbox example of stand alone software
- description of diagram
- interfaces to sensor systems
- database structure built along community standards
- metadata and data proceed in parallel
- harkens to FAIR principles for data management
- discussion
- definitely worthy of groups consideration
- interested in relation with OGC and sensorWeb
- Corinna input?
- I was hoping Jeff would jump in, he's done most of this from the CUASHI perspective
- EDI can't do all of what is in diagram
- more interested in matchmaking
- want as modular as possible so we can insert expertise into particular areas
- Jeff Horsburgh
- knowing what we've worked on
- many pieces worked on as open source
- utility is variable
- community to work on them would make it better
- discussion
- still a gap , hard to spin up CUASHI software
- database structure & standards - should follow what has already been done
- worthy goal to pursue?
- is this thinking too big for the cluster attention?
- Two questions
- some providers take the lions share of IOT platform - Azure, Google, IOT etc. role in providing interface for embedded devices
- what external connections would you like to integrate with
- open question
- could use a wider array of definitions for inputs
- brining in metadata from a community should only require setting up once
- include built-in test cases - automated continuous development process
- software framework for data management
might be best to make end-to-end system work with most popular systems real-time applications
- probably not - in external connections is near real time
- lessen work for IMs etc.
- or focus on practices
- GCE toolbox actually covers a LOT of this ground
- but built on top of Matlab
- specialized metadata
- can use some "lessons learned" from it
- as Corinna said, would like to see modular with well-documented interfaces, perhaps as web services
- could go "box by box" for discussions of what is needed and what is already working
- develop some use cases
- develop cluster goals
- need to focus on interface/APIs between boxes
- that allows others to build code on either end independently
- Like modular approach - talking about process boxes
- AND uses diverse software as processing
- Loggernet to GCE Toolbox to CSV to DB
- could fill out - broaden input
- identify and move forward on generic pieces
- XDOMES at Winter Meeting - will this topic be discussed there
- want outreach to make a larger group available
- would have no problem proposing session at winter meeting
- we know how to do this as individuals - but tools for new person don't exist yet
- would be good to leave some blank boxes in framework and let others fill it out
- modularity sounds good - importance of pieces varies between groups
- would like a plug-and-play framework
- overlaying pieces on framework would identify holes and what plays well together
- Would like both higher and lower level documents
- need to communicate with managers and programmers
- Next call
- run through ideas on how data flows from place to place
- identify places where other boxes are needed
09/05/2017
Attendees
Don Henshaw
Scotty Strachan
Matt Bartos
Renee Brown
Cove Sturtevant
Cody Flagg
Janet Fredericks
Ethan McMahon
John Porter
Felimon Gayanilo
Recap of summer meeting
- Janet's session
- Wade's session
- Funding Friday Prize
Cove Sturtevant (NEON) - Mobile Applications for maintenance and Field Data
- Instrumented Systems
- IS Science Data Quality Monitoring - automated flagging of data
- Adding separate system for sensor health monitoring
- Quality monitoring application
- Science review
- Rolling analyses
- Maintenance records
- Q: Where do you put in biases and offsets for individual sensors?
- A: This is done in calibration lab.
- Q: Are standard QAQC measures used? (ioos.noaa.gov/project/qartod)
- A: Not yet.
- Q: Are technicians viewing this in field on their phones? Do they need access to cell/wifi?
- A: Generally use ruggedized tablets. Can pre-download data if there is no connectivity.
- Q: Where do you get the barcodes from?
- A: We make them.
- Q: How long does it take to develop apps?
- A: Single applications can be developed in less than a day. Some applications are drag and drop and can be made in a couple hours.
- Q: What is the cost of fulcrum
- A: Standard, Business, Professional. Standard is $18 per device/user per month.
06/06/2017
Attendees
Don Henshaw
Matt Bartos
Scotty Strachan
Renee Brown
Wade Sheldon
Announcements
Renee - McMurdough LTER Data Manager
Discussion of Summer Meeting Sessions
- No response from DataONE yet
- Four speakers right now: Matt Bartos, Mike Daniels, Mike Botts, Wade Sheldon
Poster draft
- Cluster emphasis
- End to end sensor data management
- Best practices for deployment are fairly mature
Questions
- 2 & 3 Priority for envirosensing cluster
- Raw vs. curated / snapshot vs. linking / strategies for metadata / sensorML vs. tabular / provenance -- how to link back to earlier versions, which data to keep
- Skip July meeting; skip August; reconvene in September
05/02/2017
Attendees
Don Henshaw
Amber Jones
Matt Bartos
Felimon Gayanilo
Janet Fredericks
Martha Apple
Jason Downing
Paul Celicourt
Introduction
Discussion of Envirosensing Panel
Panelists
- Confirmed
- LTER
- Data ONE
- NCEI
- Possible
- EPA/NASA
- ARS (agricultral research stations)
- Earthcube
- National snow and ice data center
Questions for panel
- Data submission protocols
- What are the proper protocols and standards for submitting data to repositories?
- e.g. should data be sent as snapshots or streams?
- Integration with real-time streaming sensor networks
- What APIs are available for automatically pushing data to a repository?
- How can repositories encourage participation from small research labs that maintain their own sensor networks?
- Data Quality
- Should repositories have a role in assuring data quality?
- What type of quality control should be performed before submission to repositories?
- Should repositories provide checks for data quality?
- How should storage of metadata associated with data quality be handled?
- Data curation
- Who should be responsible for data curation: submitter or publisher?
- Would it be helpful to have an external rating system for data quality/usefulness?
- Data duplication
- What is the proper way to deal with syncing and duplication of datasets across repositories?
04/04/2017
Attendees
Matt Bartos
Scotty Strachan
Mike Daniels
Jason P. Downing
Renee F Brown
Don Henshaw
Bruce Caron
Mike Botts
Wade Sheldon
Amber Jones
Ethan McMahon
Janet Fredericks
Introduction
CHORDS: Cloud-hosted real-time data services for the geosciences
- Real-time data is of critical and growing importance in the geosciences
- Necessary for hazards like floods, earthquakes, etc.; but also field experiments
- Enhances rates of data transfer from the field will improve data quality and research outcomes
- Organizations like NCAR have great real-time visualization tools, but the data are not easily accessible
- Small research teams are taking valuable measurements that could also be of broad benefit
- However, these data often aren't accessible to the broader community
- Case studies:
- Studying evaporation in the great lakes
- Using infrasound to detect severe weather
- Volcano monitoring in Tanzania
- Crowdsourced real-time data helping to measure and predict earthquakes
- Web of sensor data can be challenging to manage
- Varying spatial scales, flags, metadata
- Most scientists don't want to spend time reading standards
- Enter CHORDS
- Chords emphasizes simple ingest and access to real-time data
- Meant for scientists who want to spend time doing science rather than managing data.
- Can be set up using Amazon Web Services by a lay-user.
- Data is pushed using simple HTTP GET requests.
- Live demo of portal
- Implementation details
- SensorML used to register each site.
- Data fetch via geojson, csv, etc.
- Data stored in influxdb, MYSQL used for metadata
- Connects to grafana for visualization.
- Version 1.0 scheduled for October 2017
- Automatic DOIs for data
- Implementing OGC standards
- Event triggers
- CHORDS architecture
- Portals operated by individuals feed into processing, translation, mapping services.
- Workflows to integrate with archiving services.
- URL
- Discussion on plans for the summer
- Decided on a breakout session, along with a panel.
- Breakout group will focus on end-to-end sensor systems.
03/06/2017
Attendees:
Don Henshaw
Matt Bartos
Andrew Rettig
Ethan McMahon
Janet Fredericks
John Porter
Mary Martin
Paul Celicourt
Scotty Strachan
Vincent Moriarty
Martha Apple
John Andersen
Notes
Paul Celicourt - An end to end automated environmental data collection system
- Objective: develop an integrated data acquisition system.
- Incorporates sensing, data management, publication and analysis into the same package.
- Secondary objectives:
- Self-organized sensor network.
- Platform-independent and protocol-agnostic.
- Software application to encode and decode sensors and sensor platform descriptions.
- Hardware supports most popular data interfaces.
- Data publication in different formats and unit systems.
- Hardware costs less than $200.
- System operation and network organization
- TEDS are used as a mechanism to provide metadata to each station prior to deployment.
- Use CUAHSI ODM data format and Django Web Framework for automated data management.
- Field deployment in Brooklyn.
- Uses Zigbee protocol.
- Software tools: PyTED. Sensor description using IEEE 1451 standards.
- Software tools: HydroUnits. Dimensional analysis in Hydrologic computing systems using sensor-based standards.
- Summary
- System is capable of handling data collection, transmission, management and publication.
- Effective in reducing field data acquisition workload and reducing human errors.
- Currently developing an online configuration and programming tool.
Janet Fredericks - Update on XDOMES project
- SensorML editor working, but need to develop more manufacturer-friendly vocabularies.
- Want to encourage sensor manufacturers to create content.
- Sensor manufacturer suggested not only creating document, but also an ID for each sensor that can be used by data managers.
- Link to sensor registry spreadsheet added to wiki page.
- Schedule time for showing sensorML editor at upcoming telecon.
02/06/2017
Attendees:
Scotty Strachan
Matt Bartos
Don Henshaw
Janet Fredericks
John Porter
Paul Celicourt
Vasu Kilaru
Notes
Matt Bartos - Wireless sensor networks for smart water infrastructure
- Overview of research efforts underway at the University of Michigan Real-Time Water Systems Lab.
- Description of wireless sensor node hardware and data backend.
- Two ongoing applications:
- Using wireless sensor networks for real-time flash flood monitoring in the Dallas--Fort Worth Metroplex.
- Using wireless sensors to optimize stormwater quality via automated control infrastructure in Ann Arbor, MI.
Janet Fredericks - Update on XDOMES
- Current efforts
- Overhaul so users can create re-usable models of sensors with rulesets.
- Release of sensorML editor ongoing.
- Goals for cluster
- Create vocabularies that manufacturers can reference (e.g. sensor types, observable properties).
- Sensor vocabularies should be domain-driven rather than manufacturer driven.
- Remove ambiguities in vocabularies (e.g. beam strength vs. sensor strength).
- Identify some example cross-domain sensors.
- Create vocabularies that manufacturers can reference (e.g. sensor types, observable properties).
- Immediate goals
- By March meeting: Have sensor types and observable properties.
- Make spreadsheet (SensorType and ObservableProperties) a google doc that is linked to from envirosensing page to invite more community participation.
Plans for next telecon
- Doodle poll to set time for upcoming meetings (effective April).
12/05/2016
Attendees:
Don Henshaw
Janet Fredericks
Alison Adams
Scotty Strachan
Andrew Rettig (started work with ES cluster several years ago, stopped about a year ago; teaching ES networking at U of Dayton)
Erin Robinson
Janet Fredericks
Vasu Kilaru
Felimon Gayanilo
Eric Fritzinger (works with Scotty)
Matthew Mayernik
Notes:
Discussion of what is needed from community to enable semantic and syntactic interoperability
- Sensor manufacturer responsible for making OEM model description
- creates a unique identifier for each sensor
- SensorML file is created and belongs to sensor owner
- Data managers describe processing that's done
- Documents reference terms that have meaning --> reference ontologies
- Hoping to get community to help with development of ontologies, esp in communities Janet isn't a part of (i.e. not in oceanography comm'ty) -- then Janet can work with sensor manufacturers to reference those terms
- Recorded presentation includes example ontology form; Wiki reference links should include date accessed
- MMI ontology registry has a big vocabulary database
- Can create new vocabularies and map to other known ontologies
- Watch the video of the telecon for more information about how this works
Scheduling an informal lunch/coffee meeting at AGU in San Francisco (all)
- Scotty added Janet's presentation schedule to his schedule for the meeting -- contact either of them if you want to meet up with them
- Janet will send out this message to everyone
Next meeting (2/6/17)
- Vasu and Janet talk about engaging sensor manufacturers
11/07/2016
Attendees:
Don Henshaw
Janet Fredericks
Alison Adams
Scotty Strachan
Mike Botts
Mike Dye
Carlos Rueda
Paul Lemieux
Renee Brown
Ethan McMahon
Notes:
Change in leadership
- Scotty Strachan to begin co-leading EnviroSensing group with Janet Fredericks
- good time to change leadership, Don is retiring soon, time to think about new directions (XDOMES? other projects?)
- Matt Bartos (U Michigan) will take over as student fellow (Alison is done at the end of the year)
Discussion of new directions
- Don - short recap of history of EnviroSensing cluster and original mission
- this is review from previous telecons--see past notes
- Janet - update on past year of work on XDOMES
- ES group mission and goals is well-aligned with her work, but she needs people to participate rather than just be an audience for the work they're doing
- e.g. looking for vocabularies... need this before can go to sensor manufacturers
- having difficulty getting people involved -- need folks to participate!
- no ES explicit presence (i.e. no specific sessions) at winter meeting, but intend to have a session at summer meeting
- Don suggested more explicitly crafting this as a clear opportunity/desire to get people involved
- good to tie to specific time frame, task, or event
- Send out info prior to telecon, have people look at it and prep something specific, then discuss on telecon
- maybe try to target specific people - perhaps folks can help ID who these people would be
- follow-up meeting next month to try to get volunteers to describe sensors (spreadsheet?)
- plan to have an informal ESIP/EnviroSensing get-together at AGU?
- would be great to have a few volunteers, work up to people doing talks at ESIP Summer Meeting?
- spend some time together talking about what this is, catching up on the cluster, etc.
- Scotty - some thoughts on future directions
- excited to step in and help out with the group, since it's been very helpful to expand his data management world
- completed PhD over the summer, things are currently a little hectic--transitions, etc.
- focus on field designs, esp. on mountain environments
- could do multiple projects at once--would be good to stay focused and help the XDOMES project
06/06/2016
Attendees:
Don Henshaw
Janet Fredericks
Alison Adams
Jane Wyngaard
Scotty Strachan
Vasu Kilaru
Wade Sheldon
Mike Botts
Ethan McMahon
Notes:
Janet Fredericks on Q2O web service
Using SWE to bind metadata to observational data - enabling dynamic data quality assessment
Background: NOAA-sponsored project to address data quality in sensor web enablement frameworks
- Janet walked participants through the Q2O (QARTOD-to-OGC) web service implementation to demonstrate how web services are used to describe processing, select and describe data and offer it to a user as a service.
OpenSensorHub
- open sensor access so that people can discover sensor data, etc.
- geolocatable/geospatially aware, fully described data, sensors, and processing
- free open source software on github
SmartCity Air Challenge
- Ethan: EPA is putting out a challenge called the SmartCityAir challenge: ask communities to tell them how they would deploy a large team of air sensors
- provide seed money, etc.
- groups have to describe how they would monitor and manage the data, deal with sensors throughout their lifecycle, do this sustainably…
- gov’t learning what practices work best
- will be announced formally next week; Ethan will send a less formal paragraph out to this group
- interested in ideas for how to get word out about this, how to encourage people to use best data management practices that exist
- focus is really on the data management side
With XDOMES and OpenSensorHub → trying to get all of that kind of knowledge (how it was used, description of data, etc.) so that you could have people throwing sensors out and have access to that data
Summer meeting:
- Janet will be sending out an email about preparing for the workshop at the summer meeting → people can let Janet know that they’re coming
- At Thursday session hopefully Janet, Mike, and Vasu will all be presenting
- Maybe talk about whether we want to reach out to a broader audience? Janet would like to know how many people plan to attend the workshop
- Poster? Don says we did one last year and could just substitute some of the stuff on there with summary points of what we’ve done this year on our calls -- Don will produce the poster if Janet sends summary slides from some of the talks to him
NO call in July -- we’ll just see each other at ESIP! Next call will be in August.
05/02/2016
Attendees:
Don Henshaw (Forest Service in Oregon)
Alison Adams (EnviroSensing Student Fellow)
Annie Burgess (ESIP)
Lindsay Barbieri (Student Fellow)
Carlos Rueda (software engineer at Monterrey, part of XDOMES team)
Corinna Gries (LTER)
Felimon Gayanilo (Texas A&M, XDOMES)
Janet Fredericks (WHOI, XDOMES)
Wade Sheldon (LTER)
Mark Bushnell (oceanographer, XDOMES)
Jane Wyngaard (JPL)
Notes:
Mark Bushnell – Quality Assurance and Quality Control of Real-Time Ocean Data (QARTOD):
- QARTOD manuals: focus on real time, usually coastal (http://www.ioos.noaa.gov/qartod)
- Includes quality control tests and quality assurance of sensors (in an appendix)
- Discussion of operational vs. scientific quality control and different needs/contexts for each
- Board meets quarterly to review progress and identify next variables (if you have ideas for variables, let Mark know!)
- Each manual takes 6-8 months and each one is a living document that is updated
- 26 core variables
- next up: phytoplankton species!
- Discussed an example test from the waves manual
- Five “states” for data qc flags (pass, not evaluated, suspect or of high interest, fail, missing data)
Discussion & Questions for Mark:
- How to handle flags that represent a mix of semantic notions? Hard for data consumers to understand (what’s the actual problem?)
- What about showing (or not showing) data that doesn’t meet a certain standard?
- If you’re looking for extreme events, for example, you might want to see all the data…
- Helpful to have the option to see all the data (if you’re an operator, say, you might look at failed tests for an instrument)
- Best to let the data user select what level of quality they’re interested in
- For EnviroSensing, is there a place to save/share code for QC tests?
- Not at the moment
- Would be good to start tracking/storing code for tests that people do somewhere and have a DOI that describes the processing
- python codes for the implementation of the QARTOD recommendations is at https://github.com/ioos/qartod (also look here --> https://github.com/asascience-open/QARTOD)
- Core link in email from Janet (body copied below) -- many other pages and additional information can be found from that link
- Background: The U.S. Integrated Ocean Observing System® (IOOS) Quality Assurance/Quality Control of Real-Time Oceanographic Data (QARTOD) project has published nine data quality-control manuals since 2012. The manuals are based on U.S. IOOS-selected parameters (or core variables) that are of high importance to ocean observations. The purpose of the manuals is to establish real-time data quality control (QC) procedures for data collection for core variables, such as water levels, currents, waves, and dissolved nutrients. The QC procedures provide guidance to eleven U.S. IOOS Regional Associations and other ocean observing entities, helping data providers and operators to ensure the most accurate real-time data possible. It began as a grass-roots organization over a decade ago - the background can be found on the IOOS QARTOD Project website: http://www.ioos.noaa.gov/qartod/welcome.html
- Links from Mark
- The link for the flags crosswalk note is http://odv.awi.de/fileadmin/user_upload/odv/misc/ODV4_QualityFlagSets.pdf.
- There's similar work at http://www.iode.org/index.php?option=com_oe&task=viewDocumentRecord&docID=10762
- Tad Slawecki (tslawecki@limno.com) heads the informal QARTOD Implementation Working Group and posts minutes at https://docs.google.com/document/d/128xPGjTMBP9FC-SEg9vGe8bD145LGdapH21WV5smzn4/edit
- In those minutes, he has a USGS github link at https://github.com/USGS-R/sensorQC
04/04/2016
Attendees (small group):
Don Henshaw
Mark
Corinna
Scotty Strachan
Janet Fredericks
Minutes:
Best practices work:
- Post a citation suggestion on the introduction page of best practices? Maybe also a snapshot PDF that folks could download with a citation?
- People DO go to this page, says Scotty
- Mountain Research Institute (MRI) is doing some work trying to write best practices, would be good to try to get them to use ours rather than create something entirely new…
Future of EnviroSensing cluster:
- Just need to have one vision at a time--for now we can stay focused on the XDOMES work
- Scotty wants to continue to stay involved, promote cluster and its work a bit more to folks after finishing up his PhD (soon!); would be interested in taking lead in cluster after that, too
Future meetings:
- Mark will present material on real-time quality control that was planned for this month on the next call instead, due to low attendance on this call
- Would be great to reiterate that we didn’t start as ONLY LTER--the best practices doc had input from other folks too--and that that isn’t where we have to stay, either--we can continue to incorporate other things/groups
- Janet will talk about summer meeting workshop plans on June call: registering vocabulary
- will be a good workshop for beginners/people who need to be introduced to the concept
- Summer meeting (July): have a 1.5-hour session for EnviroSensing--should email asking for folks to present
03/07/2016
Attendees:
Alison Adams
Don Henshaw
Janet Fredericks
John Porter
Felimon Gayanilo
Krzysztof Janowicz
Carlos Rueda
Peter Murdoch
Vasu Kilaru
Scotty Strachan
Corinna Gries
Ethan McMahon
Minutes:
1. Future meeting ideas (Don)
- Revisit best practices--have folks present the chapters they did and rekindle interest; Scotty said he’d be willing to do this for his chapter
- Email Alison at alison.adams@uvm.edu if you have ideas for future telecons!
2. Summer meeting -- session/workshop ideas? (proposals due at beginning of April)
- XDOMES workshop: connected to EnviroSensing cluster?
- sensor provenance EnviroSensing breakout session?
- Janet to lead more hands-on workshop on Semantic Web, etc.?
- Let Alison, Don, or Janet know if you have additional ideas
3. Update on work plan draft (Don & Janet)
- Conversation with Erin last week
- If you might like to be the next leader, let Don know--thinking of having an “on deck” leader position
- Interest in AGU Geospace blog? Lots on data use, etc. This could be a place to put out info about our cluster
4. Rotating chair position for Products & Services (Alison)
- Right now, Products & Services does three things: (1) FUNding Friday, (2) the P&S Testbed, and (3) tech evaluation process with NASA AIST to evaluate AIST-funded projects. Also working on an evaluation process for the projects funded through the testbed, and would hopefully provide this to the Earth science community at large eventually.
- P&S wants to have a rotating co-chair position; would last for three months and would be a rep from a different committee. The first co-chair (starting in April) wouldn’t be involved in proposal evaluation/selection, but would be involved in ideas for student/PI matchmaking and mentoring for FUNding Friday and the evaluation and testbed activities. It would be a great opportunity to learn more about P&S and have them more about us. It wouldn’t prevent you from submitting a proposal to the testbed.
- If you’re interested, email Soren at sorenscott@gmail.com; you can sign up for the rotating co-chair position here
5. XDOMES team on Semantic Web (Krzysztof Janowicz)
- Full recording with slides available in recorded meeting
- Semantic technologies can improve semantic interoperability
- More intelligent metadata, more efficient discovery, use, reproducibility of data; reduce misinterpretation and meet data requirements of journals, etc.
- Large community in science, gov’t, industry; many open source and commercial tolls and infrastructures; there are existing ontologies and huge amounts of interlinked data
- Move the logic OUT of software and INTO the data
- Semantic technologies support horizontal as well as vertical workflows
- Semantic interoperability can only be measured after the fact bc meaning is an emergent property of interaction; come up with technologies that prevent data that shouldn’t be combined from being combined
- Discussion:
- How much of this is already in use or is this just setting up the platform at this point?
- With XDOMES we are at an early stage, but in terms of the infrastructure, a lot of it is already productively used
02/02/2016
Attendees:
Bar (Lindsay Barbieri) - Data Analytics Student Fellow
Corinna Gries (North Temperate Lakes LTR site)
Janet Fredericks (Woods Hole, managing coastal observatory and leading XDOMES project)
Vasu Kilaru (EPA Office of Research and Development)
Josh Cole (UMBC Data Manager -- interested in hearing about XDOMES)
Scotty Strachan (Geography Dept at University Nevada Reno, current PhD student)
Pete Murdoch (Science Advisor for NE Region of USGS, also working with DOI)
Ethan McMahon (EPA Office of Environmental Information)
Renee Brown (Univ of New Mexico, Sevilleta LTER and Field Station)
Jane Wyngaard (Post-doc at JPL)
Don Henshaw (Forest Service PNW Research Station)
Notes drafted by Alison Adams (EnviroSensing Student Fellow) according to meeting recording
On the agenda:
Janet on XDOMES
What should EnviroSensing cluster do?
XDOMES (Cross-Domain Observational Metadata Environmental Sensing Network) NSF project funded as part of EarthCube
- cross-domain observational metadata
- focusing on: sensor metadata creation, data quality assessment, sensor interoperability, automated metadata generation, content management
- 4 types of work (funded for two years)
- software development to create Sensor ML Generators and registries for semantic techs and Sensor ML documents
- Trying to engage people like this community, sensor manufacturers to create a network of people interested in promoting adoption of sensor ML techs
- Useability assessment
- Integration of semantic interoperability
- this project is about capturing metadata at the time of CREATION of the data
- GOAL: put out metadata automatically in interoperable ways (community-adopted, standards-based framework)
- use of registered terms to enable interoperability
- can also help manage operations by providing standardized information about how sensors were configured, etc.
- want to develop community within ESIP who will follow and promote this approach
- vet, look at usability assessment, what do you want us to do and is it useful to you?
Discussion
- Vasu says this is related to things EPA has been working on--excited to continue the conversation
- Jane asked about communication with sensor-creators so this can be implemented in new sensors; Janet said that isn’t part of the project at this point
- Potential application for folks with Alliance for Coastal Technologies--they work on sensor validation and I think it’s a slightly different take, but this is definitely something they could benefit from (like if we start describing our sensors in a way that the information can be harvested)
- If people are interested you can see what Janet is putting in the file and think about whether this would be helpful or useful or whether she’s missing something; would you be willing to test this in your own environment?
- Delivering data is kind of beyond the scope of the XDOMES project at this point
- Vision: info is out on the web, and you have an app to go and harvest the information that you need
- Do we want to work this pilot project into public presence in this cluster?
Future WebEx ideas
- Could do a WebEx on how water ML, sensor ML, etc. fit together
- Janet is approaching Krzysztof Janowicz to discuss sensors and semantics with us on the March call
06/22/2015
Today we had Dave Johnson from LiCOR Environmental speak with us about eddy covariance. We had a large group of attendees.
Dave shared with us links to the LiCOR software [3] and a book [4]
- The problem they are solving is that the data coming in with eddy covariance is high resolution and large, more than 300 million records per year.
- They were able to put much of the processing and QC logic into the SMART system, which is on the instruments in the field.
- An eddy covariance system can "see" about 100 x its height above the canopy.
- We were interested in the possibilities for real-time data qc, and also how the information can be transferred between the instruments in the field (i.e. cell modem, line, etc.)
02/24/2015
Today we had William Jeffries from Heat Seek NYC talk to us about his platform for civilian monitoring of home temperatures in apartments using low-cost sensors and wireless mesh networks taking hourly readings.
Attending the call were Fox and Don from Andrews, Josh at UM, Jason in Fairbanks, Becky from Onset, Ryan from UT and Scotty from UNV as well as some listen-in callers.
- Heat seek NYC has nodes containing up to 1000 low-power, low-cost thermometers which are mapped to custom printed circuit boards. The software stack is Ruby on Rails with a Post-Gres backend.
- HSNYC focuses on sensor network's ability to affect and effect policy, and so far has seen that the government views sensor networks as a cool solution to a lot of difficult regulatory challenges. They provide reliable information that officials and citizens can use with printable, court-friendly PDF's.
- Funding is by kickstarter and was started by the NYC Big Apps campaign, a giant Civic Hacking Competition.
- They have been incorporated as a non profit in NY since 2014
- First sensors were Twine Sensors off the shelf temperature sensors; not so good
stale data
- Now they make their own temperature sensors, using a push system instead of a pull system. (if there's a problem with the system just don't get pull out of it.)
- We asked about LCD and the reason for not implementing is that many tenants aren't actually as interested in the data as the response from the policy makers
- Some people will abuse the system so they use tamper evident tape and photos of installation to protect ; landlords have financial incentives and there can be intentional lack of repairs
- Local cache-ing of the sensor occurs; sensors cache at the hub, relay server that sends back
- There are several levels of caching, and also well as flash memory on the XP radios
- A key question was is there a long-term storage that is a local cache? We don't have a solution either
- We had interest in a company called H20degree
- For QC they do indoor and outdoor temperature comparison, comparing the sensors to one another and to store bought sensors before and after installation...
- Frequency for radios/caching is hourly waking up point data
- We are all interested in a low-or-no power source/ transmitter, maybe based on raspberri pi
- need a high peak power envelope-- transmission needs to deal with being fairly far apart... implementation needs to come online and transmit an awesome signal for a short time and go back to sleep.
Note on Webex recording: I am going to check into the recording- my system claims to have been recording but it is nowhere to be found (Fox)
01/27/2015
Rick Susfalk from the Desert Research Institute presented about Acuity Data Portal. Notes from meeting taken by Fox Peterson, please edit as you see fit.
We had in attendance 8 persons.
Acuity Portal System
- Started in 2006
- originally VDV data solution
- improvements to web-interface; sits on top of VDV as Acuity server
- Acuity is a continuous monitoring of key client-driven data
- it includes sensors and data logging deployment and maintenance, telemetry, data storage and analysis, automated airing, web portal for data access
- individualized web presence tailored to client needs
- not a single tool, but instead integrates commercial, open source, and proprietary hardware and tools
- customizable project specific descriptions
- common tools used to provide rapid, cost-effective deployment of individualized portals
- physical infrastructure is shared amongst smaller clients for cost-saving or it can be segregated for larger clients.
- access is controlled down to the variable level-- "we can define who gets to see what"-- for example, public can not see some features
- one view could be "pre-defined graphs" without logging in, but if you want to download the data you must log in at your permissions level
SECURITY
- priority, very important
- DOD certification and accredidation (possible link below)
- http://www.dtic.mil/whs/directives/corres/pdf/832002p.pdf
- have hired a security professional
- https:// everywhere for good protocol
ANALYSIS
- customized thresholding and data-freshness
- trending alerts, for example, know if battery will go bad
- stochastic and numerical modeling
- scoring incoming data for QA/QC processing
QA/QC tools
- web-based GUI
- users and managers can create, edit, and modify alerts online
- groups can be created so that you can schedule management and alerts
- also offers localized redundant alerting
- two-way communication with the campbell data loggers (cr1000)
- more about getting the data to the data managers for more in-depth QA/QC than about providing that part of the tooling
Data graphing features
- Pre-determined graphs for basic users
- Data selector for more advanced users
- "we don't know what the users want to see so we give them the tools to do it" (good idea!)
- anything you can change in Excel you can change in their graphs on the website -
Links
- vista data vision (http://www.vistadatavision.com/)
- another vdv link: (http://www.vistadatavision.com/features/responsive/)
Metadata
- relates your parameters to the network and what other sensors are doing
- current system is getting more flexible
- metadata is still largely user responsibility
Flight plan - safety tool
- field personnel are data
- users put in the travel time for safety
- buddy system, alerted right before you return, then calls boss etc. Many levels hierarchy
Demos
- Portals that are monitoring things
- ability for data refreshment
- colors for indication, ex., data would not be gray if there was lots of new data
- users can change the settings on the data logger
- scrolling, scaling, plotting, etc. via interaction with the user
- can save your own graphs
graphs and alerts
- many parameters
- you can save!
- email, sms, phone
- default settings for users
- lots of personnel management tools in this in general
- cross-station "truly alarm or not" if station1 has a value but station2 has a different one, don't alarm sorts of rules
- lists/user groups appear to be very important with this tool
- sensor and triggers: customize one or more parameters that you are bringing into your database
real-time updates on loggers
- ex. 10 minute data, user comes in and makes a change, the information is saved to the database and then is presented to all other users
- the person will request a change and say what that change is
- when there are different levels of connectivity ie. analog phone modems, before the data has the chance to work its way back into the system there is a lot of validation being done
example
- https://l30l31.dri.edu/
- everglades heat pulse flow meters
- uses google maps
measurements/metadata
- extends beyond the vdv, more than 1 .dat file
- integrate multiple .dat into many tables
- managed by the data managers at DRI
- workflow :
logger net --> vdv --> acuity, ok, let's give access to the DM for all these variables, click on it ok now the manager can see it --> generate an excel file with tables for all this metadata --> enter the data into the excel files--> send back to acuity --> injests, runs queries, back to db-- > metadata in bulk, quickly
- we asked if the system ends before the qa/qc process begins, answer: qa/qc is done at the DRI, near real time QA though
future capability
- direct the managers to the future data problems
- manual decision making
Scotty asked about the duration (long and short term) of projects and how affects funding. most is funded by long-term projects; this is why they do the stats and numerical methods in the future
Amber asked about pricing; pricing is by hour to get up, then a price for maintaining the system for the duration of the project 5-10 .dat files, only 8 hours of person time at DRI to make a portal
11/25/2014
Jordan Read presented the SensorQC R package
Recorded Session Play | Download
10/28/2014
Wade Sheldon presented the GCE Data Toolbox – a short summary follows:
- Community-oriented environmental software package
- Lightweight, portable, file-based data management system implemented in MATLAB
- generalized technical analysis framework, useful for automatic processing, and it's a good compromise using either programmed-in or file-based operations
- Generalized tabular data model
- Metadata, data, robust API, GUI library, support files, MATLAB databases
- Benefits and costs: platform independent, sharing both code and data seamlessly across the systems, version independent as far as MATLAB goes, and is now "free and open source" software. There is a growing community of users in LTER.
Toolbox data model
- Data model is meant to be a self-describing environmental data set-- the metadata is associated with the data, create date and edit date and such are maintained, and its lineage.
- Quality control criteria- can apply custom function or one already in the toolbox
- Data arrays, corresponding arrays of qualifier flags -- similar to a relational database table but with more associated metadata
Toolbox function library
- The software library is referred to as a "toolbox"
- a growing level of analytical functions, transformations, aggregation tools
- GUI functions to simplify the usage
- indexing and search support tools, and data harvest management tools
- Command line API but there is also a large and growing set of graphical form interfaces and you can start the toolbox without even using the command line
Data management framework
- Data management cycle - designed to help an LTER site do all of its data management tasks
- Data and metadata can be imported into the framework and a very mature set of predefined import filters exist: csv, space- and tab-delimited and generic parsers. Also, specialized parsers are available for Sea-Bird CTD, sondes, Campbell, Hobo, Schlumberger, OSIL, etc.
- Live connections i.e. Data Turbine, ClimDB, SQL DB's, access to the MATLAB data toolbox
- Can import data from NWIS, NOAA, NCDC, etc.
- Can set evaluation rules, conditions, evaluations, etc.
- Automated QC on import but can do interactive analysis and revision
- All steps are automatically documented, so you can generate an anomalies report by variable and date range which lets you communicate more to the users of the data
Recorded Session Play | Download
9/23/2014
- Fox Peterson (Andrews LTER) reported on QA/QC methods they are applying to historic climate records (~13 million data points for each of 6 sites).
The challenge was that most automated approaches still produced too many flagged data that needed to be manually checked. Multiple statistical methods were tested based on long-term historical data. The method they selected was to use a moving window of data from the same hour over 30 days and test for 4 standard deviations in that window; E.g., use all data for 1 pm for days 30 - 60 of the year, compute four standard deviations, and set the range for the midpoint day (45) at the 1pm hour to that range.
- Josh Cole reported on his system, which is in development and he will be able to share scripts with the group.
- Brief discussion of displaying results using web tools.
- Great Basin site discussed the variability in their data, which "has no normal"-- how could we perform qa/qc based on statistics and ranges in this case?
- Discussion of bringing Wade Sheldon to call next time / usefulness of the toolbox for data managers
- Discussion of using Pandas package- does anyone have experience, can we get them on?
- Discussion of the trade off between large data stores, computational strength, and power. Good solutions?
- ESIP email had some student opportunities which may be of interest
- Overall, it was considered helpful if people were willing to share scripts. Discussion of a GIT repository for the group, or possibly just use the Wiki.
Recorded Session: Play | Download
8/26/2014
Suggestions for future discussion topics
- Citizen Science contributions to environmental monitoring
- 'open' sensors - non-commercial sensors made in-house, technology, use, best practices
- Latest sensor technologies
- Efficient data processing approaches
- Online data visualizations
- New collaborations to develop new algorithms for better data processing
- Sensor system management tools (communicating field events and associating them with data)