Channel Modifications

Channel modifications are at the core of customizing instrumentation in obsinfo. They allow one to completely modify an instrumentation’s components, or specific aspects of those components (advanced/chan_mods). The process is complicated, here is an explanation of the philosophy, the codes involved and the potential bugs.

Files involved

The channel_modifications field is read in station.py and passed down to instrumentation.py then channel.py. Its handling is controlled by the ObsMetaData class defined in obsmetadata.py

in station.schema.json:

The structure of channel_modifications in the input file is specified in station.schema.json, where it is placed within the station object and its format is defined by channel_mods:

"channel_mods": {
    "type": "object",
    "description": "individual changes specified by das channel",
    "patternProperties": {
        "^[N, E, Z, 1, 2, 3, H, \\*]-[0-9, \\*]+$": { "$ref": "#/definitions/channel_modif"}
    }
},
"channel_modif": {
    "type": "object",
    "description": "DAS channel, modifications",
    "properties" : {
        "orientation_code": {"$ref": "instrumentation.schema.json#/definitions/orientation_code"},
        "datalogger" :      {"$ref": "datalogger.schema.json#/definitions/datalogger_wo_required_fields"},
        "preamplifier":     {"$ref": "preamplifier.schema.json#/definitions/preamplifier_wo_required_fields"},
        "sensor" :          {"$ref": "sensor.schema.json#/definitions/sensor_wo_required_fields"},
        "datalogger_configuration" :  {"type": "string"},
        "preamplifier_configuration": { "type": "string"},
        "sensor_configuration" :      {"type": "string"}
    },
    "additionalProperties" : false

The definitions for sensor, datalogger and preamplifier are the same as for their initial definition, except they have less requirements. Change to something called {component}_modifications? And define each to allow a “base” field? Maybe also a “serial_number” and “configuration” field (could remove datalogger_configuration, preamplifier_configuration and sensor_configuration, though retain for now for compatibility)? And add “serial_number” at the channel_modif level, which would allow for a change to the base “equipment” serial_number.

There are several _wo_required_fields parameters, it looks like they could all be renamed to _modifications:

in station.py:

Initializing an “Station`` class runs:

instr_dict = attributes_dict.get('instrumentation', None)
channel_modifs = attributes_dict.get('channel_modifications', {})
if instr_dict:
    self.instrumentation = Instrumentation(
        instr_dict, self.locations, start_date, end_date,
        channel_modifs, self.serial_number)

in instrumentation.py:

Initializing an “Instrumentation`` class calls:

self.channels = [Channel(label, attributes, locations,
                         start_date, end_date,
                         self.equipment.obspy_equipment,
                         channel_default, channel_modifs)
                 for label, attributes in das_channels.items()]

where das_channels comes from instr_dict['channels'] and channel_default comes from das_channels[‘default’]

in channel.py:

Initializing a Channel class calls

selected_channel_modifs = self.get_selected_channel_modifs(
    self.channel_id_code, channel_modifs)
self.instrument = Instrument(self.das_channel, selected_channel_modifs)

and get_selected_channel_modifs() is:

in instrument.py:

Initializing an Instrument class calls

for ic_name in ('datalogger', 'sensor', 'preamplifier'):
    key = ic_name + '_configuration'
    config_selector = attributes.get_configured_element(key, channel_modifs)
    ic_obj = InstrumentComponent.dynamic_class_constructor(
        component, attributes_dict, channel_modifs, config_selector)
    setattr(self, ic_name, ic_obj)  # equivalent to self.ic_name = ic_obj

where channel_modifs is the selected_channel_modifs in Channel.__init()__

Obsmetadata.get_configured_element(key, channel_modifs={}, selected_config={}, default=None) returns the value corresponding to channel_modifs[key], else selected_config[key], else self[key], else default. Weird thing is that it won’t return selected_config[key] if it’s a dict or ObsMetadata, though this doesn’t matter here.

Here it just returns the value in channel_modifs[‘{inst_component}_configuration’] or None, as far as I can tell

I changed the code to:

for ic_type in ('datalogger', 'sensor', 'preamplifier'):
    ic_config_key = ic_type + '_configuration'
    if ic_type in channel_modifs:
        ic_modifs = channel_modifs[ic_type]
        # Pop out keywords
        config = ic_modifs.pop('configuration', None)
        sn = ic_modifs.pop('serial_number', None)
        base = ic_modifs.pop('base', None)
        # replace ic by channel_modifs[ic_type][]'base'] if it exists
        if base is not None:
            logger.info('Replacing {ic_type}')
            attributes[ic_type] = base
        if sn is not None:
            if 'equipment' in ic_modifs:
                if 'serial_number' in ic_modifs['equipment']:
                    logger.warning('equipment:serial_number and serial_number specified, equipment:serial_number overrides')
                else:
                    ic_modifs['equipment']['serial_number'] = sn
            else:
                ic_modifs['equipment'] = {'serial_number': sn}
        if config is not None:
            # For now, just replace v0.110 "*_configuration" keyword
            if ic_config_key in attributes:
                msg = 'attributes[{}]={} replaced by {{"{}": {{"configuration": {}}}}}'.format(
                    ic_config_key, attributes[ic_config_key], ic_type, config)
                warnings.warn(msg)
                logger.warning(msg)
            attributes[ic_config_key] = config
    config_selector = attributes.get_configured_element(ic_config_key,
                                                        channel_modifs)
    ic_obj = InstrumentComponent.dynamic_class_constructor(
        ic_type, attributes, channel_modifs, config_selector)
    setattr(self, ic_type, ic_obj)  # equivalent to self.ic_type = ic_obj

Seems to work

config_selector won’t need ic_name + 'configuration' in v0.111, as the congfiguration will use the same keyword (configuration) for each InstrumentComponent field.

Finally, each component class has a dynamic_class_constructor() that modfies the ObsMetadata as requested and returns the result:

in instrument_component.py:

InstrumentComponent

This is the function called at first, it passes on to the specific dynamic_class_constructor class

@staticmethod
def dynamic_class_constructor(component_type, attributes_dict,
                              channel_modif={}, config_selector=''):
    """
    Creates an appropriate Instrument_component subclass (Sensor,
    Preamplifier, Datalogger) from an attributes_dict

    Args:
        component_type (str): type of component. Used for selection of
            adecuate class.
        attributes_dict (dict or :class:`ObsMetadata`): component
            attributes
        channel_modif (dict or :class:`ObsMetadata`): channel modifications
            inherited from station
        delay_correction (float or None): delay correction in seconds:
            if a float: set last component stage to this, others to 0.
            if None: all component stage corrections are set = delay
        config_selector (str): selector of configuration coming from
            Instrument

    Returns:
        object of the adequate subclass
    """
    if not attributes_dict.get(component_type, None):
        if component_type == 'preamplifier':  # Only preamps are optional
            return None
        else:
            msg = f'No {component_type}'
            warnings.warn(msg)
            logger.error(msg)
            raise TypeError(msg)

    selected_config = InstrumentComponent.retrieve_configuration(
        component_type, attributes_dict[component_type], config_selector)

    if component_type == 'datalogger':
        theclass = Datalogger
    elif component_type == 'sensor':
        theclass = Sensor
    elif component_type == 'preamplifier':
        theclass = Preamplifier
    else:
        msg = f'Unknown InstrumentComponent "{component_type}"'
        warnings.warn(msg)
        logger.error(msg)
        raise TypeError(msg)
    obj = theclass.dynamic_class_constructor(
        ObsMetadata(attributes_dict[component_type]),
        channel_modif.get(component_type, {}),
        selected_config)
    return obj

Sensor

def dynamic_class_constructor(cls, attributes_dict, channel_modif={},
                              selected_config={}):
    """
    Create Sensor instance from an attributes_dict

    Args:
        attributes_dict (dict or :class:`ObsMetadata`): the base sensor
        channel_modif (dict or :class:`ObsMetadata`): channel modifications
            inherited from station
        selected_config (dict or :class:`ObsMetadata`): the configuration
            description that will override or complement default values
    Returns:
        (:class:`Sensor`)
    """

    if not attributes_dict:
        return None
    if not selected_config:
        # Avoids a syntax error in the yaml file: two consecutive labels
        # with no response stages
        selected_config = {}

    seed_dict = ObsMetadata(attributes_dict).get_configured_element(
        'seed_codes', channel_modif, selected_config, {})

    # The next line of code will totally override response states in
    # attribute_dict IF there is a selected_config with response_stages
    response_stages_list = attributes_dict.get_configured_element(
        'response_stages', {}, selected_config, None)

    response_stages = ResponseStages(
        response_stages_list,
        channel_modif.get('response_modifications', {}),
        selected_config.get('response_modifications', {}),
        None)

    obj = cls(Equipment(ObsMetadata(attributes_dict.get('equipment',
                                                        None)),
                        channel_modif.get('equipment', {}),
                        selected_config.get('equipment', {})),
              ObsMetadata(seed_dict).get_configured_element(
                'band_base', channel_modif, selected_config, None),
              ObsMetadata(seed_dict).get_configured_element(
                'instrument', channel_modif, selected_config, None),
              response_stages,
              attributes_dict.get_configured_element(
                'configuration_description', channel_modif,
                selected_config, ''))
    return obj

Preamplifier

@classmethod
def dynamic_class_constructor(cls, attributes_dict, channel_modif={},
                              selected_config={}):
    """
    Create Preamplifier instance from an attributes_dict

    Args:
        attributes_dict (dict or :class:`ObsMetadata`): attributes of
            component
        channel_modif (dict or :class:`ObsMetadata`): channel modifications
            inherited from station
        selected_config (dict or :class:`ObsMetadata`): the configuration
            description that will override or complement default values
    Returns:
        (:class:`Preamplifier`)
    """

    if not attributes_dict:
        return None
    if not selected_config:
        # Avoids a syntax error in the yaml file: two consecutive labels
        # with no response stages
        selected_config = {}

    # The next line of code will totally override response states in
    # attribute_dict IF there is a selected_config with response_stages
    response_stages_list = attributes_dict.get_configured_element(
        'response_stages', {}, selected_config, None)
    config_description = attributes_dict.get_configured_element(
        'configuration_description', channel_modif, selected_config, '')

    response_stages = ResponseStages(
        response_stages_list,
        channel_modif.get('response_modifications', {}),
        selected_config.get('response_modifications', {}),
        None)

    obj = cls(Equipment(ObsMetadata(attributes_dict.get('equipment',
                                                        None)),
                        channel_modif.get('equipment', {}),
                        selected_config.get('equipment', {})),
              response_stages,
              config_description)

    return obj

Datalogger

def dynamic_class_constructor(cls, attributes_dict, channel_modif={},
                              selected_config={}):
    """
    Create Datalogger instance from an attributes_dict

    Args:
        attributes_dict (dict or :class:`ObsMetadata`): component
            attributes
        channel_modif (dict or :class:`ObsMetadata`): channel modifications
            inherited from station
        selected_config (dict or :class:`ObsMetadata`): the configuration
            description that will override or complement default values
    Returns:
        (:class:`Datalogger`)
    """
    if not attributes_dict:
        return None
    if not selected_config:
        # Avoids a syntax error in the yaml file: two consecutive labels
        # with no response stages
        selected_config = {}

    sample_rate = attributes_dict.get_configured_element(
        'sample_rate', channel_modif, selected_config, None)
    delay_correction = attributes_dict.get_configured_element(
        'delay_correction', channel_modif, selected_config, None)
    config_description = attributes_dict.get_configured_element(
        'configuration_description', channel_modif, selected_config, '')

    # The next line of code will totally override response states in
    # attribute_dict IF there is a selected_config with response_stages
    response_stages_list = attributes_dict.get_configured_element(
        'response_stages', {}, selected_config, None)

    response_stages = ResponseStages(
                        response_stages_list,
                        channel_modif.get('response_modifications', {}),
                        selected_config.get('response_modifications', {}),
                        delay_correction)

    obj = cls(Equipment(ObsMetadata(attributes_dict.get('equipment',
                                                        None)),
                        channel_modif.get('equipment', {}),
                        selected_config.get('equipment', {})),
              sample_rate,
              delay_correction,
              response_stages,
              config_description)

    return obj

This seems like some code duplication, but that’s not my problem right now and this part is too far downstream for me to worry about: the “magic” part (that currently screws up for importing a new sensor, for example) is at the Instrument level when ObsMetadata.get_configured_element() is called.

My first guess it that get_configured element() is substituting all the new information into the existing sensor, but since the new information has no configuration_default the old value for this (and other elements) is left.

The simplest (and best?) solution would be to first substitute the entire InstrumentCompoent attributes_dict when “sensor:base”, “datalogger:base” or “preamplifier:base” is specified. This implies that I “change” the input file specification to use “base”, but since it doesn’t function right now anyway, this should not break anything.