Eva Interactive Intro#
Jupyter Notebooks#
Jupyter notebook is a web browser-based tool for interacting with python code in a modular fashion. The start-stop functionality of the notebook cells offers a way to work with code that is more explainable, which can be useful in areas such as scientific communication, visualizations, and software development. The web page you are currently viewing has been built using Jupyter notebooks. Below is an example of an of executed code block and it’s output:
x = 10
print('x + 5 =', x+5)
x + 5 = 15
EvaInteractive#
Objective#
The objective of eva interactive is to provide a way to dynamically interact with data collections and produce plots on-the-fly for focused data analysis. This is done by creating a Jupyter notebook-friendly interface for eva.
To load in eva interactive, run the following:
from eva.eva_interactive import EvaInteractive
eva = EvaInteractive()
Data Collection#
Eva Interactive saves information loaded from files into eva’s underlying data collection structure. A data collection can be loaded and viewed within Jupyter Notebook using one command. Simply run load_collection(collection_name, file, data_space). The collection name is a name created by the user to label which data collection is loaded in. Multiple data collections can be loaded in during one eva interactive session. The data_space argument tells eva which data ingest function to use under the hood.
file = '../../../data/ioda_obs_space.aircraft.hofx.2020-12-14T210000Z.nc4'
collection_name = 'data'
data_space = 'IodaObsSpace'
dc = eva.load_collection(collection_name, file, data_space)
INFO EvaInteractive: Initializing eva IodaObsSpace object
INFO DataCollections: --------------------------------------------------------------------------------
INFO DataCollections: Collections available:
INFO DataCollections:
INFO DataCollections: Collection name: data
INFO DataCollections:
INFO DataCollections: Dimensions:
INFO DataCollections: Location: 99
INFO DataCollections:
INFO DataCollections: Coordinates:
INFO DataCollections: Location
INFO DataCollections:
INFO DataCollections: Data (group::variable):
INFO DataCollections: EffectiveError::airTemperature (float32) | Min=+1.0000e+00, Max=+1.4706e+00, RMS=+1.0771e+00
INFO DataCollections: EffectiveError::windEastward (float32) | Min=+2.5000e+00, Max=+2.5000e+00, RMS=+2.5000e+00
INFO DataCollections: EffectiveError::windNorthward (float32) | Min=+2.5000e+00, Max=+2.5000e+00, RMS=+2.5000e+00
INFO DataCollections: EffectiveQC::airTemperature (int32 ) | Min= +0, Max= +0
INFO DataCollections: EffectiveQC::windEastward (int32 ) | Min= +0, Max= +0
INFO DataCollections: EffectiveQC::windNorthward (int32 ) | Min= +0, Max= +0
INFO DataCollections: GsiAdjustObsError::airTemperature (float32) | Min=+1.0000e+00, Max=+2.7137e+00, RMS=+1.2265e+00
INFO DataCollections: GsiAdjustObsError::windEastward (float32) | Min=+2.5000e+00, Max=+5.1534e+00, RMS=+2.7953e+00
INFO DataCollections: GsiAdjustObsError::windNorthward (float32) | Min=+2.5000e+00, Max=+5.1534e+00, RMS=+2.7953e+00
INFO DataCollections: GsiEffectiveQC::airTemperature (int32 ) | Min= +0, Max= +1
INFO DataCollections: GsiEffectiveQC::windEastward (int32 ) | Min= +0, Max= +1
INFO DataCollections: GsiEffectiveQC::windNorthward (int32 ) | Min= +0, Max= +1
INFO DataCollections: GsiFinalObsError::airTemperature (float32) | Min=+1.0000e+00, Max=+1.4548e+01, RMS=+1.9612e+00
INFO DataCollections: GsiFinalObsError::windEastward (float32) | Min=+2.5000e+00, Max=+9.0386e+00, RMS=+2.9692e+00
INFO DataCollections: GsiFinalObsError::windNorthward (float32) | Min=+2.5000e+00, Max=+9.0386e+00, RMS=+2.9692e+00
INFO DataCollections: GsiHofX::airTemperature (float32) | Min=+2.1099e+02, Max=+2.9181e+02, RMS=+2.5772e+02
INFO DataCollections: GsiHofX::windEastward (float32) | Min=-3.9112e+00, Max=+5.3673e+01, RMS=+2.3508e+01
INFO DataCollections: GsiHofX::windNorthward (float32) | Min=-4.5300e+01, Max=+2.8187e+01, RMS=+1.3531e+01
INFO DataCollections: GsiHofXBc::airTemperature (float32) | Min=+2.1134e+02, Max=+2.9142e+02, RMS=+2.5761e+02
INFO DataCollections: GsiHofXBc::windEastward (float32) | Min=-3.9112e+00, Max=+5.3673e+01, RMS=+2.3508e+01
INFO DataCollections: GsiHofXBc::windNorthward (float32) | Min=-4.5300e+01, Max=+2.8187e+01, RMS=+1.3531e+01
INFO DataCollections: GsiInputObsError::windEastward (float32) | Min=+2.5000e+00, Max=+2.5000e+00, RMS=+2.5000e+00
INFO DataCollections: GsiInputObsError::windNorthward (float32) | Min=+2.5000e+00, Max=+2.5000e+00, RMS=+2.5000e+00
INFO DataCollections: GsiQCWeight::airTemperature (float32) | Min=+4.0000e+00, Max=+4.0000e+00, RMS=+4.0000e+00
INFO DataCollections: GsiQCWeight::windEastward (float32) | Min=+4.0000e+00, Max=+4.0000e+00, RMS=+4.0000e+00
INFO DataCollections: GsiQCWeight::windNorthward (float32) | Min=+4.0000e+00, Max=+4.0000e+00, RMS=+4.0000e+00
INFO DataCollections: GsiUseFlag::airTemperature (int32 ) | Min= -1, Max= +1
INFO DataCollections: GsiUseFlag::windEastward (int32 ) | Min= -1, Max= +1
INFO DataCollections: GsiUseFlag::windNorthward (int32 ) | Min= -1, Max= +1
INFO DataCollections: MetaData::dateTime (int64 ) | Min=+1607979779, Max=+1608000419
INFO DataCollections: MetaData::latitude (float32) | Min=+1.9650e+01, Max=+4.4647e+01, RMS=+3.5280e+01
INFO DataCollections: MetaData::longitude (float32) | Min=+2.3754e+02, Max=+2.8530e+02, RMS=+2.6559e+02
INFO DataCollections: MetaData::pressure (float32) | Min=+1.7870e+04, Max=+1.0173e+05, RMS=+6.4129e+04
INFO DataCollections: MetaData::stationElevation (float32) | Min=-7.8000e+01, Max=+1.2497e+04, RMS=+7.1935e+03
INFO DataCollections: MetaData::height (float32) | Min=-3.4000e+01, Max=+1.2497e+04, RMS=+6.5524e+03
INFO DataCollections: MetaData::stationIdentification (object )
INFO DataCollections: ObsBias::airTemperature (float32) | Min=+0.0000e+00, Max=+0.0000e+00, RMS=+0.0000e+00
INFO DataCollections: ObsBias::windEastward (float32) | Min=+0.0000e+00, Max=+0.0000e+00, RMS=+0.0000e+00
INFO DataCollections: ObsBias::windNorthward (float32) | Min=+0.0000e+00, Max=+0.0000e+00, RMS=+0.0000e+00
INFO DataCollections: ObsError::airTemperature (float32) | Min=+1.0000e+00, Max=+1.4706e+00, RMS=+1.0771e+00
INFO DataCollections: ObsError::windEastward (float32) | Min=+2.5000e+00, Max=+2.5000e+00, RMS=+2.5000e+00
INFO DataCollections: ObsError::windNorthward (float32) | Min=+2.5000e+00, Max=+2.5000e+00, RMS=+2.5000e+00
INFO DataCollections: ObsType::airTemperature (int32 ) | Min= +133, Max= +133
INFO DataCollections: ObsType::windEastward (int32 ) | Min= +233, Max= +233
INFO DataCollections: ObsType::windNorthward (int32 ) | Min= +233, Max= +233
INFO DataCollections: ObsValue::airTemperature (float32) | Min=+2.0815e+02, Max=+2.9125e+02, RMS=+2.5780e+02
INFO DataCollections: ObsValue::windEastward (float32) | Min=-6.1000e+00, Max=+5.5300e+01, RMS=+2.3130e+01
INFO DataCollections: ObsValue::windNorthward (float32) | Min=-4.7000e+01, Max=+3.0400e+01, RMS=+1.4299e+01
INFO DataCollections: PreQC::airTemperature (int32 ) | Min= +1, Max= +13
INFO DataCollections: PreQC::windEastward (int32 ) | Min= +1, Max= +13
INFO DataCollections: PreQC::windNorthward (int32 ) | Min= +1, Max= +13
INFO DataCollections: PreUseFlag::airTemperature (int32 ) | Min= +0, Max= +101
INFO DataCollections: PreUseFlag::windEastward (int32 ) | Min= +0, Max= +101
INFO DataCollections: PreUseFlag::windNorthward (int32 ) | Min= +0, Max= +101
INFO DataCollections: hofx::airTemperature (float32) | Min=+2.1094e+02, Max=+2.9195e+02, RMS=+2.5773e+02
INFO DataCollections: hofx::windEastward (float32) | Min=-3.8279e+00, Max=+5.3992e+01, RMS=+2.3526e+01
INFO DataCollections: hofx::windNorthward (float32) | Min=-4.6586e+01, Max=+2.8748e+01, RMS=+1.3622e+01
INFO DataCollections: --------------------------------------------------------------------------------
The output from the executed code block above details the fields available in the data collection. These fields can be used for running transforms and plotting.
Transforms#
Eva has two transforms available: arithmetic and accept_where. These transforms are available as their own functions, and the arguments of these functions closely mirror what is expected in eva YAML config files. Users don’t need to specify which field variables to apply the transform to. If a variable list isn’t supplied, the function will apply the transform to all variables in a given field.
eva.arithmetic('ObsValueMinusHofX', 'ObsValue-hofx', collection_name)
eva.arithmetic('ObsValueMinusGsiHofXBc', 'ObsValue-GsiHofXBc', collection_name)
INFO EvaInteractive: Added 'ObsValueMinusHofX' to data collection 'data'.
INFO EvaInteractive: Added 'ObsValueMinusGsiHofXBc' to data collection 'data'.
eva.accept_where('ObsValueMinusHofXPassedQc', 'ObsValueMinusHofX', ['EffectiveQC == 0'], collection_name)
INFO EvaInteractive: Added 'ObsValueMinusHofXPassedQc' to data collection 'data'.
New fields created by eva interactive’s transforms are added to the data collection.
Plotting#
Fields in the data collection can be used within eva interactive’s plotting functions. The next section shows an overview of available eva interactive plots. For a scatter plot, run the following:
variable = 'windEastward'
eva.scatter(collection_name+'::ObsValueMinusHofX::'+variable, collection_name+'::ObsValueMinusGsiHofXBc::'+variable)
The eva interactive plotting functions require the collection name and variable of the fields to be plotted. Eva interactive’s plotting functions rely on hvplot, which can be set to different backends such as bokeh (eva interactive’s default hvplot backend) and matplotlib. Bokeh plots can be manipulated using the default sidebar. Users can zoom in, adjust the plot view, and save the file to desktop.