.. _JSON: https://www.json.org/json-en.html
.. _YAML: https://yaml.org
.. _Dont Repeat Yourself: https://en.wikipedia.org/wiki/Don%27t_repeat_yourself
.. _Yasmine: https://github.com/iris-edu/yasmine-stationxml-editor
.. _ObsPy: https://github.com/obspy/obspy/wiki
.. _JREF: https://datatracker.ietf.org/doc/html/draft-pbryan-zyp-json-ref-03
.. _YAML_anchors_guide: https://medium.com/@kinghuang/docker-compose-anchors-aliases-extensions-a1e4105d70bd
*******************************
Overview
*******************************
#################
Introduction
#################
**obsinfo** is a Pure Python system for creating FDSN-standard metadata files
(StationXML)and standardizing/specifying data processing workflows for ocean
bottom seismometers.
Instrumentation and stations are specified in text-based
information files in `YAML`_ or `JSON`_ format.
The basic philosophy of **obsinfo** is:
1. break down every component of the system into “atomic”, non-repetitive
units (`Dont Repeat Yourself`_)
2. Follow StationXML structure where possible, but:
- Add entities missing from StationXML where necessary
- Specify units for each component
3. Allow full specification of stations using text files, for repeatibility
and provenance
.. _object_model:
#################
Object Model
#################
**obsinfo** uses several layers of information files, which are linked to each
other. Most **obsinfo** users will only use **subnetwork** files
(the purple boxes below), which
provide network and station information.
.. image:: obsinfo_object_model.jpg
#################
Information Files
#################
**obsinfo** inputs **information files** in `JSON`_ or `YAML`_ format,
and outputs StationXML files.
There are 5 main **information file** types:
+--------------------------------------+-----------------------+-----------------+------------------+
| Type | Description | Filled by | When filled |
+======================================+=======================+=================+==================+
| :ref:`subnetwork_template` | Deployed stations, | | after a data |
| | their instruments | OBS facility | collection |
| | and parameters | | campaign |
+--------------------------------------+-----------------------+-----------------+------------------+
| :ref:`instrumentation_base_template` | Instrumentations | OBS facility | new/changed |
| | (OBSs) | | instrumentations |
+--------------------------------------+-----------------------+-----------------+------------------+
| instrument_components | Components of a | OBS facility | |
| :ref:`sensor_base_template` | single channel | -or- | new |
| | | component | components or |
| :ref:`preamplifier_base_template` | | manufacturer | calibrations |
| | | | |
| :ref:`datalogger_base_template` | | | |
+--------------------------------------+-----------------------+-----------------+------------------+
| :ref:`stage_base_template` | One component stage | OBS facility | |
| | (a sensor, an analog | -or- | new |
| | filter/gain, an A/D | component | components or |
| | converter, | manufacturer | calibrations |
| | decimation filters..) | | |
+--------------------------------------+-----------------------+-----------------+------------------+
| :ref:`Filter_templates` | Specific filter | OBS facility | |
| | used in a single | -or- | new |
| | stage | component | components or |
| | single stage | manufacturer | calibrations |
+--------------------------------------+-----------------------+-----------------+------------------+
There are 5 other types that allow you to store commonly used/repeated
information in one place: :ref:`timing_base_template`, :ref:`operator_template`,
:ref:`person_template`, :ref:`location_base_template`, :ref:`network_template`
Only the :ref:`subnetwork_template` files contain deployment-specific information.
For most data collection campaigns, they are all you'll need to fill out.
File Names
========================
Information files should use the following naming convention:
::
{NAME}.{type}.{format}
where
- ``{NAME}`` is chosen by you, and should succintly indentify the object.
- ``{type}`` is one of the **information file** types
- ``{format}`` is one of ``yml``, ``yaml``, ``jsn`` or ``json`` (preferably the 4-character version)
Using these conventions allows **obsinfo** to perform informative error-checking.
Examples:
* ``TI_ADS1281_FIR1.stage_base.yaml`` is a filename for a stage of a
Texas Instruments FIR filter, written in YAML format.
* ``BBOBS.INSU-IPGP.subnetwork.json`` is a filename for a subnetwork of
broad-band stations
deployed by the INSU-IPGP OBS facility, written in JSON format.
Schemas and Templates
========================
You will find annotated schemas for each file in the :ref:`Schemas` document.
You will find templates for each file in the :ref:`templates` document.
You can generate a file containing the complete syntax using the command
``obsinfo template`` (see the tutorial on :ref:`tutorial_subnetwork` for an
example using :ref:`subnetwork_template` files).
File Hierarchy
====================
Each information file can access files in a reference directory (the **DATAPATH**).
We recommend the following structure, for compatibility between implementations:
.. code-block::
├── persons
├── timing_bases
├── networks
├── operators
├── timing_bases
├── location_bases
├── instrumentation_bases
├── datalogger_bases
│ └── stage_bases
│ └── filters
├── preamplifier_bases
│ └── stage_bases
└─── sensor_bases
└── stage_bases
└── filters
.. _json_references:
JSON References
==================
Within each file, you can load information from another file using the
`JREF`_ syntax. For example,
::
author: {$ref: "../persons/Wayne_Crawford.person.yaml#person"}
substitutes the content of the element ``person`` within the file
``../persons/Wayne_Crawford.person.yaml`` into the ``authors`` element
of the current file.
Removing the #{element} suffix
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If the element name is the same as the file ``{type}``, you can omit
the ``#{element}`` anchor:
::
author: {$ref: "../persons/Wayne_Crawford.person.yaml"}
This is not standard `JREF`_ syntax, but it makes files easier to read.
File Metadata
=================
All information files can contain the following top-level elements:
* ``format_version:`` [REQUIRED] - Version of obsinfo for which this file was
written
* ``revision:`` - Who wrote this file and when.
- ``date:`` - date of revision
- ``authors:`` - a list of authors of this document
* ``notes:`` - a list of text strings which provide information
which will **not** be put into the StationXML file.
#####################
Added elements
#####################
The following obsinfo elements are not found in StationXML.
They allow the user to inject important information into StationXML, or to
enter information in simpler format.
+--------------------------------------+------------------------------------+----------------------------------------------+
| Element | Goal | Integration into StationXML |
+======================================+===============================================+===================================+
| ``processing:clock_correction`` | Specify clock drift and/or | Station-level ``Comment``` with |
| | leap seconds | ``subject="Clock correction"`` |
+--------------------------------------+------------------------------------+----------------------------------------------+
| :ref:`digital_filter_template` | Simplify entering a digital gain | :ref:`coefficients_filter_template` with |
| | that has no filtering | empty numerator and denominator |
+--------------------------------------+------------------------------------+----------------------------------------------+
| :ref:`analog_filter_template` | Simplify entering an analog gain | :ref:`poles_zeros_filter_template` with |
| | that has no filtering | empty poles and zeros |
+--------------------------------------+------------------------------------+----------------------------------------------+
| :ref:`ad_conversion_filter_template` | Enter an A/D converter based on | :ref:`coefficients_filter_template` with |
| | its voltage and count ranges | empty numerator and denominator. Gain is |
| | | calculated and compared with the stage gain. |
+--------------------------------------+------------------------------------+----------------------------------------------+
| ``subnetwork:extras`` | Add extra parameters not specified | Station-level ``Comment`` with |
| | in obsinfo | ``subject="obsinfo extras"`` |
+--------------------------------------+------------------------------------+----------------------------------------------+
| ``instrumentation:extras`` | Add extra parameters not specified | Channel-level ``Comment`` with |
| | in obsinfo | ``subject="obsinfo extras"`` |
+--------------------------------------+------------------------------------+----------------------------------------------+
#####################
Resources
#####################
For the YAML specification, see `YAML`_ .
For a tutorial, see `YAML Tutorial `_ .
For linting and validating YAML files, see `YAML Validator `_
For the JSON specification, see `JSON`_ .
For a tutorial, see `JSON Tutorial `_
For linting and validating JSON files, see `JSON Validator `_
For the JREF notation, see `JREF Notation `_
For the StationXML specification, see
`FDSN StationXML Reference `_