Difference between revisions of "Data Management Course Outline"

From Earth Science Information Partners (ESIP)
Line 7: Line 7:
 
* Agency requirements - Ruth
 
* Agency requirements - Ruth
 
**NSF data management plan - Ruth
 
**NSF data management plan - Ruth
**NASA science data policy  
+
**NASA science data policy - Ron Weaver
 
**NOAA Administrative Order 212-15, Management of Environmental and Geospatial Data and Information
 
**NOAA Administrative Order 212-15, Management of Environmental and Geospatial Data and Information
 
* Return on Investment
 
* Return on Investment
Line 67: Line 67:
 
**Metadata for tracking data processing
 
**Metadata for tracking data processing
 
**Individual agencies, archives and registries may have specific requirements
 
**Individual agencies, archives and registries may have specific requirements
*Working with your archive organization
+
*Working with your archive organization - Ron Weaver/NSIDC
 
**Planning for longer term preservation - Jeff Arnfield/NCDC
 
**Planning for longer term preservation - Jeff Arnfield/NCDC
 
**Work with your archive early and often - Jeff Arnfield/NCDC
 
**Work with your archive early and often - Jeff Arnfield/NCDC
Line 93: Line 93:
 
===Preservation strategies===
 
===Preservation strategies===
  
*Sponsor (e.g., Agency) or institution requirements
+
*Sponsor (e.g., Agency) or institution requirements - Ron Weaver /NSIDC
*Options for archiving your data
+
*Options for archiving your data - Ron Weaver/NSIDC
**What archives are out there?
+
**What archives are out there? - Ron Weaver/NSIDC
 
***Discipline or institutional archives
 
***Discipline or institutional archives
 
***Finding an archive
 
***Finding an archive
**What to do if there is no archive out there
+
**What to do if there is no archive out there - Ron Weaver/NSIDC
*What data goes into a Long-term archive?  
+
*What data goes into a Long-term archive? - Ron Weaver/NSIDC
 
*What do long term archives do with my data?  - Jeff Arnfield/NCDC
 
*What do long term archives do with my data?  - Jeff Arnfield/NCDC
*Data transfer & submission agreements  
+
*Data transfer & submission agreements - Ron Weaver/NSIDC
 
** See "Submission Agreements" section under "For Data Managers"
 
** See "Submission Agreements" section under "For Data Managers"
 
** Agency/archive specific requirements my vary
 
** Agency/archive specific requirements my vary
 
*Intro to the OAIS Reference Model
 
*Intro to the OAIS Reference Model
*Emerging standards for preservation
+
*Emerging standards for preservation - Ron Weaver/NSIDC
 
*Metadata
 
*Metadata
  

Revision as of 09:28, August 18, 2011

NOTE: We agreed that the target audience initially would be scientists

For Scientists

The case for data stewardship

  • Agency requirements - Ruth
    • NSF data management plan - Ruth
    • NASA science data policy - Ron Weaver
    • NOAA Administrative Order 212-15, Management of Environmental and Geospatial Data and Information
  • Return on Investment
    • Return on your investment
    • Expanding the audience for your data
    • Return on public investments
  • Verifiable science
    • Tying your data to standards, metrics, and benchmarks
  • Facilitating science through interoperable discovery and access
  • Enhancing your reputation
  • Preserving the Scientific Record
    • Establishing Relationships with archives
    • Preserving a Record of Environmental Change
    • Other case studies?
  • What Not to do when Archiving Data!

Data Management plans

  • Why do a data management plan?
  • Elements of a plan - Ruth
    • Identify materials to be created - Ruth
    • Identify your audience(s) - Ruth
    • Data organization - Ruth
    • Roles and responsibilities - Ruth
    • Describing and documenting your data, including metadata - Ruth
    • Standards used - Ruth
    • Data access, sharing, and re-use policies - Ruth
    • Backups, archives, and preservation strategy - Ruth
    • ??QUESTION: Should the plan define (an) objective metric(s) to make implementation and compliance measurable?
  • Estimating effort and resources required - Ruth
    • Hardware, software capabilities required - Ruth
    • Personnel resources and skills needed - Ruth
  • Some available resources to help with developing your plan - Ruth

Local Data Management

  • Managing your data - Ruth
    • Data identifiers and locators - Jeff Arnfield/NCDC
    • File naming conventions (Cook)
    • Backing up your data (Cook)
    • Write it down! Maintaining contemporaneous documentation
      • Who, what, when, where, why, how
      • Tracking and describing changes to the data
    • QUESTIONS
      • Should we at least touch on version control, or is that adequately addressed by tracking/describing changes?
  • Data Formats - Ruth
    • Building understandable spreadsheets - Jeff Arnfield/NCDC
    • Using self-describing data formats
    • Choosing and adopting community accepted standards
    • Avoiding proprietary formats
  • Creating documentation and metadata
    • Developing a citation for your data (Cook)
    • Recording provenance and context - Jeff Arnfield/NCDC
    • For your collections as a whole
    • Creating item level metadata
    • Metadata for discovery - Tyler Stevens/GCMD
    • Metadata for access and use - Jeff Arnfield/NCDC
    • Metadata for archiving - Jeff Arnfield/NCDC
    • Metadata for tracking data processing
    • Individual agencies, archives and registries may have specific requirements
  • Working with your archive organization - Ron Weaver/NSIDC
    • Planning for longer term preservation - Jeff Arnfield/NCDC
    • Work with your archive early and often - Jeff Arnfield/NCDC
    • Broadening your user community
  • Advertising your data
    • Agency/institution requirements for publishing metadata
    • Journals and publications
    • Agency/institution web sites
    • Using portals and registries
      • Publishing metadata to a Web Accessible Folder
      • Publishing metadata to GCMD - Tyler Stevens/GCMD
      • Publishing metadata to ECHO
      • Publishing metadata to Data.Gov
      • NOTE: Need to address additional portals and registries beyond GCMD & ECHO. Add other entries as appropriate
    • Datacasting
  • Providing access to your data
    • Evaluating who your audience is
    • Who gets to access your data
      • Agency best practices & policies
    • Access mechanisms
    • Tracking data usage
    • Handling sensitive data
    • Rights

Preservation strategies

  • Sponsor (e.g., Agency) or institution requirements - Ron Weaver /NSIDC
  • Options for archiving your data - Ron Weaver/NSIDC
    • What archives are out there? - Ron Weaver/NSIDC
      • Discipline or institutional archives
      • Finding an archive
    • What to do if there is no archive out there - Ron Weaver/NSIDC
  • What data goes into a Long-term archive? - Ron Weaver/NSIDC
  • What do long term archives do with my data? - Jeff Arnfield/NCDC
  • Data transfer & submission agreements - Ron Weaver/NSIDC
    • See "Submission Agreements" section under "For Data Managers"
    • Agency/archive specific requirements my vary
  • Intro to the OAIS Reference Model
  • Emerging standards for preservation - Ron Weaver/NSIDC
  • Metadata

Responsible Data Use

  • Citation and credit
  • Data restrictions
  • Fair use
  • Feedback and metrics
  • Collaboration
  • Community participation

For Data Managers

  • Data Management plan support
  • Collection or acquisition policies
  • Intro to OAIS reference model
  • Initial Assessment and appraisal
    • Identify information to be preserved
      • main features and properties
      • dependencies on information here or elsewhere
    • Identify objects to be received
    • Establish complementary information needs (e.g., format, data descriptions, provenance, reference information, context, fixity information)
      • What complementary information is needed for data useful for climate studies (USGCRP list)
    • Assessing potential designated communities
    • Assessing probable curation duration
    • Assessing data transfer options
    • Defining access paths
    • Assessing costs and feasibility
    • Metadata, metadata standards, and levels of metadata
  • Submission agreements
    • Data integrity
    • Contacts
    • Schedule
    • Operational Procedures
    • Error reconciliation
    • Constraints
    • other aspects necessary for understanding how to support the data
  • Preparing for ingest
  • Ingesting data
    • Validation checks
    • Identifiers
    • Citations
    • Levels of service
  • Periodic re-assessment
  • Curation activities
    • Media migration
    • Format migration