trepr.analysis module

Data analysis functionality.

Processing vs. analysis steps

The key difference between processing and analysis steps: While a processing step modifies the data of the dataset it operates on, an analysis step returns a result based on data of a dataset, but leaves the original dataset unchanged.

Key to reproducible science is automatic documentation of each analysis step applied to the data of a dataset. Each analysis step is self-contained, meaning it contains every necessary information to perform the analysis task on a given dataset.

Analysis steps, in contrast to processing steps (see trepr.processing for details), operate on data of a trepr.dataset.Dataset, but don’t change its data. Rather, some result is obtained that can be everything from a scalar to a full (calculated) dataset, depending on the actual analysis task at hand.

In order to quantify the quality of a measured spectrum or to interpret it, it is often necessary to perform some analysis steps.

Due to inheritance from the aspecd.analysis module all analysis steps provided are fully self-documenting, i.e. they add all necessary information to reproduce each analysis step to the aspecd.dataset.Dataset.history attribute of the dataset.

Concrete analysis steps

Due to inheritance from the aspecd.analysis module all analysis steps defined there are available from within the trepr package. Furthermore, a number of analysis steps specific for tr-EPR spectroscopy have been implemented here.

The first block of analysis steps is concerned with evaluating the quality of the recorded data and revealing some possible problems during data acquisition:

• MWFrequencyDrift

  Calculate the frequency drift and compare it with the step size.

• MWFrequencyValues

  Extract microwave frequency values recorded for each time trace.

• TimeStampAnalysis

  Calculate the time spent for recording each time trace.

Other analysis steps are “true” analysis steps in terms of obtaining information from the data:

• TransientNutationFFT

  Perform FFT to extract transient nutation frequencies.

And the following classes have been extended with respect to the functionality available from the ASpecD classes:

• BasicCharacteristics

  Extract basic characteristics of a dataset.

Note to developers

Processing steps can be based on analysis steps, but not vice versa. Otherwise, we get cyclic dependencies what should obviously be avoided in order to keep code working.

Furthermore, if an analysis step returns a (calculated) dataset, use the aspecd.analysis.AnalysisStep.create_dataset() method to obtain an empty instance of this dataset. This will automatically set some parameters of the created dataset related to the analysis step.

Module documentation

class trepr.analysis.MWFrequencyDrift

Bases: aspecd.analysis.SingleAnalysisStep

Calculate the frequency drift and compare it with the step size.

In order to estimate the quality of a spectrum, it can be helpful to know the extent the frequency drifted during the measurement. Therefore, the frequency drift is converted into magnetic field units and can thus be compared to the step width of the magnetic field axis.

This requires the software used to record the tr-EPR data to save the microwave frequency value for each individual time trace. As often, tr-EPR spectra are recorded using individual software, this is merely a design question of the program.

Different outputs, e.g. a scalar value or a calculated dataset, are available, for details see below.

parameters

All parameters necessary for this step.

kindstr

Kind of characteristic to extract from the data

Valid values are “ratio” and “drift”.

In case of “ratio”, the ratio between drift and magnetic field step will be returned, in case of “drift”, the amplitude of the microwave frequency values converted into magnetic field units (mT).

Only relevant in case of “output” set to “value” or “dataset”.

Default: “ratio”

outputstr

Kind of output: value, dict. or dataset

Valid values are “value” (default), “dataset”, and “dict”. Usually, only values and datasets can be easily used within a recipe.

Default: “value”

Type

dict

result

Results of the microwave frequency drift analysis.

Depending on the output option set via the “output” parameter, either a scalar value (ratio or drift), , a dict containing both values, or a dataset containing either the ratios or the drifts as function of the magnetic field.

In case of datasets, note that due to calculating a difference between microwave frequency points, the size of the data is -1 compared to the size of the original microwave frequency values.

Type

float, dict, or aspecd.dataset.CalculatedDataset

Note

If you set the output to “dataset”, you will get a field axis with values centred between the original field values, as the microwave frequency drift analysis requires to calculate differences between points, hence returning vectors with one element less than the original vector.

Examples

For convenience, a series of examples in recipe style (for details of the recipe-driven data analysis, see aspecd.tasks) is given below for how to make use of this class. The examples focus each on a single aspect.

If you have recorded a dataset containing microwave frequency values for each individual time trace and are interested in the drift of the microwave frequency converted into magnetic field values, you can obtain a calculated dataset with the values and plot the results:

- kind: singleanalysis
  type: MWFrequencyDrift
  properties:
    parameters:
      kind: drift
      output: dataset
  result: mwfreq-drift

- kind: singleplot
  type: SinglePlotter1D
  properties:
    parameters:
      tight_layout: true
    filename: mwfreq-drift.pdf
  apply_to: mwfreq-drift

While getting the absolute values is often quite helpful, dividing the values by the magnetic field step width (assuming an equidistant axis) allows to directly assess whether the drift is relevant for the particular dataset. Which ratio is tolerable depends on the kind of spectra, but it should be <1 (better <0.5). Obtaining the ratio of frequency drift (in magnetic field units) and the step width is similar to the example above, and again, plotting the results is quite helpful:

- kind: singleanalysis
  type: MWFrequencyDrift
  properties:
    parameters:
      kind: ratio
      output: dataset
  result: mwfreq-ratio

- kind: singleplot
  type: SinglePlotter1D
  properties:
    parameters:
      tight_layout: true
    filename: mwfreq-ratio.pdf
  apply_to: mwfreq-ratio

If you are only interested in the values of the maximum drift amplitude and maximum ratio, change the parameter output from “dataset” to “value”.

See also

trepr.analysis.MWFrequencyValues

Extract microwave frequency values recorded for each time trace.

Changed in version 0.2: New parameter output controlling output format and kind controlling the kind of result. Fix with calculating the ratio. Renamed class from MwFreqAnalysis.

static applicable(dataset)

Check whether the processing step is applicable to the given dataset.

To be able to perform a microwave frequency drift analysis, the values for the microwave frequency for each individual time trace need to be recorded and available from the dataset.

class trepr.analysis.TimeStampAnalysis

Bases: aspecd.analysis.SingleAnalysisStep

Calculate the time spent for recording each time trace.

Can be helpful for debugging the spectrometer and for assessing whether delays during data acquisition are responsible for artifacts in a recorded dataset.

This requires the software used to record the tr-EPR data to save the time stamp for each individual time trace. As often, tr-EPR spectra are recorded using individual software, this is merely a design question of the program.

parameters

All parameters necessary for this step.

kindstr

Kind of characteristic to extract from the data

Valid values are “delta” and “time”.

In case of “delta”, the time difference (delta) in seconds between adjacent magnetic field points will be returned, in case of “time”, the time (in seconds) for each time trace since start of the measurement will be returned.

Default: “delta”

outputstr

Kind of output: value, dict. or dataset

Valid values are “value” (default), “dataset”, and “dict”. Usually, only values and datasets can be easily used within a recipe.

Default: “value”

Type

dict

result

Results of the microwave frequency drift analysis.

Depending on the output option set via the “output” parameter, either a scalar value (ratio or drift), , a dict containing both values, or a dataset containing either the ratios or the drifts as function of the magnetic field.

In case of datasets, note that due to calculating a difference between microwave frequency points, the size of the data is -1 compared to the size of the original microwave frequency values.

Type

float or aspecd.dataset.CalculatedDataset

Note

If you set the output to “dataset” and the kind to “delta”, you will get a field axis with one value less than the original field axis, as the analysis requires to calculate differences between points, hence returning vectors with one element less than the original vector.

Examples

For convenience, a series of examples in recipe style (for details of the recipe-driven data analysis, see aspecd.tasks) is given below for how to make use of this class. The examples focus each on a single aspect.

If you have recorded a dataset containing time stamps for each individual time trace and are interested in the time spent between adjacent magnetic field points, you can obtain a calculated dataset with the values and plot the results:

- kind: singleanalysis
  type: TimeStampAnalysis
  properties:
    parameters:
      kind: delta
      output: dataset
  result: time-delta

- kind: singleplot
  type: SinglePlotter1D
  properties:
    parameters:
      tight_layout: true
    filename: time-delta.pdf
  apply_to: time-delta

Please note that depending on the mode of measurement you will get different times between adjacent field points. In case of a background signal regularly recorded every n-th trace during measurement, you will regularly see about twice the time spent, and if you do not record linearly (up or down), but inward or outward with respect to the magnetic field axis, this will be reflected in the time deltas as well.

Sometimes you may be interested in the relative times rather than the time deltas. This can be achieved similarly to the example above. Simply change “kind” from “delta” to “time”:

- kind: singleanalysis
  type: TimeStampAnalysis
  properties:
    parameters:
      kind: time
      output: dataset
  result: time-values

- kind: singleplot
  type: SinglePlotter1D
  properties:
    parameters:
      tight_layout: true
    properties:
      drawing:
        marker: '*'
    filename: time-values.pdf
  apply_to: time-values

To better see the individual time points, here, additional markers are added to the plot. In case of outward measurement scheme, you will see a v-shaped result.

Changed in version 0.2: New parameter output controlling output format. Returns time deltas in seconds.

static applicable(dataset)

Check whether the processing step is applicable to the given dataset.

To be able to perform a time stamp analysis, time stamps need to be available for each individual time trace of the dataset.

class trepr.analysis.BasicCharacteristics

Bases: aspecd.analysis.BasicCharacteristics

Extract basic characteristics of a dataset.

This class extends the ASpecD class by the possibility to return axes values and indices for only one axis in case of ND datasets with N>1.

Currently a workaround, the functionality may move upwards in the ASpecD framework eventually.

parameters

All parameters necessary for this step.

Additionally to those from aspecd.analysis.BasicCharacteristics, the following parameters are allowed:

axisint

Number of the axis to return the axes values or indices for.

Type

dict

New in version 0.2.

class trepr.analysis.MWFrequencyValues

Bases: aspecd.analysis.SingleAnalysisStep

Extract microwave frequency values recorded for each time trace.

tr-EPR measurements can take a rather long time, up to 12-24 hours, and the stability of the microwave frequency is not always guaranteed. Crucial here is that the frequency drift is small compared to the magnetic field step width.

What does that mean practically? Using the resonance condition of magnetic resonance, one can convert magnetic field values in frequencies, and as a rule of thumb, 1 mT = 28 MHz. Therefore, if the microwave frequency drifted by < 1 MHz during the experiment, in most cases (except of experiments with very high field resolution) you should be safe and there is no reason to expect distortions of the spectra shape due to a drift in microwave frequency.

Not all software used to record tr-EPR data does record the MW frequency for each individual time trace, though. Therefore, only those datasets containing the microwave frequency values can be analysed.

The analysis step returns a calculated dataset with the magnetic field axis as the first axis and the microwave frequency values as data.

Examples

For convenience, a series of examples in recipe style (for details of the recipe-driven data analysis, see aspecd.tasks) is given below for how to make use of this class. The examples focus each on a single aspect.

If you have recorded a dataset containing microwave frequency values for each individual time trace and are interested in the drift of the microwave frequency, you can extract the microwave frequency values as function of the magnetic field as a calculated dataset:

- kind: singleanalysis
  type: MWFrequencyValues
  result: mwfreq

To plot the data contained in the newly obtained calculated dataset, simply proceed with a plotter of your choice:

- kind: singleplot
  type: SinglePlotter1D
  properties:
    filename: mwfreq.pdf
  apply_to: mwfreq

Don’t be surprised if the drift seems not to be linear or even appears to be discontinuous. Depending on how you have recorded your data, i.e. what scheme of sampling the magnetic field axes you have used, this is perfectly normal.

See also

trepr.analysis.MWFrequencyDrift

Calculate the frequency drift and compare it with the step size.

New in version 0.2.

static applicable(dataset)

Check whether the processing step is applicable to the given dataset.

To be able to extract the microwave frequency values for each individual time trace, these values need to be available from the dataset.

class trepr.analysis.TransientNutationFFT

Bases: aspecd.analysis.SingleAnalysisStep

Perform FFT to extract transient nutation frequencies.

A tr-EPR time trace showing transient nutations can be described by a zero-order Bessel function of first kind. Due to relaxation, the Bessel function is damped with an exponential.

One way to extract the transient nutation frequencies is to perform a 1D discrete Fourier transform (using the FFT algorithm) along the time axis. In case of 2D datasets, this means to perform the FFT for each time trace individually.

In any case, the analysis step will return a calculated dataset with the data representing the FFT for all time traces. Here, the scipy function scipy.fft.rfft() returning real-valued results gets used.

In case the time trace has been recorded for negative times (pre-trigger), the FFT will only be performed starting at t = 0.

Generally, if full time traces including the raising flank are analysed, it is best to cut the time traces from the left up to the extremum (maximum of the absolute value) to suppress low-frequency background of the resulting FT. This is the default behaviour and can be controlled using the start_in_extremum parameter (see below). In case of 2D datasets, the global extremum will be used for all time traces.

As long as the oscillation clearly dominates the time trace(s), i.e. the signal intensity crossing the zero line multiple times, there should be no need for further preprocessing. Things are different, however, if the time trace is dominated by an exponential decay and the oscillation merely modulates this decay. In this case, you will usually need to subtract the exponential decay from the data to obtain meaningful results from the FFT. This can be done by setting the parameter subtract_decay (see below).

Additionally, sometimes the frequency domain signal after the Fourier transform exhibits prominent side lobes. This can be suppressed by apodising the data before Fourier transform using a window function. Windows can be set using the window parameter, and if a window requires additional parameters, use the window_parameters parameter to supply them.

Choosing the optimal window and taking informed decisions requires a lot of background knowledge. Just bear in mind that different windows can have different effects on the resulting signal in the frequency domain, including shifting the maxima. Therefore, as long as you are only interested in clearly distinct frequencies rather than arbitrarily exact absolute frequencies, apodisation is perfectly fine. Distinguishing between paramagnetic species with different spin multiplicity should not be affected by whatever window you apply.

As a side note: An alternative way to analyse transient nutations would be to fit a damped Bessel function of first kind to the data. Thus one can extract both, frequency and relaxation rate directly from the data. Note, however, that this will only be possible if the oscillation clearly dominates the time trace. In cases where the oscillation merely resides on top of a dominating exponential decay, this will typically not be possible.

parameters

All parameters necessary for this step.

start_in_extremumbool

Whether to cut the time trace(s) from the left at the extremum.

Extremum here means the maximum of the absolute values of the data of the dataset.

Default: True

paddingint

Factor the original length of the time trace used for FFT should be elongated and padded with zeros (“zero filling”).

Usually, to reasonably resolve lower frequencies, a factor of 3–5 is useful.

Default: 1

subtract_decaybool

Whether to subtract a fitted exponential decay from each time trace.

Sometimes the time trace is dominated by an exponential decay rather than the transient nutation, i.e. the nutation appears as modulation on top of an exponential decay. In this case, the FFT will usually not allow for reliably extracting the nutation frequencies, due to a huge background at the low-frequency end.

In this case, fitting and afterwards subtracting an exponential decay from each time trace helps a lot. As usually the exact form and parameters of the exponential decay are not known, here, an exponential decay with two parameters in the form f(x) = a * exp(-b * t) gets used. While leading to quite reliable results in the FFT, the fitted parameters can usually not be interpreted.

Default: False

windowstr

Name of the window for apodisation of the data.

Windows are often applied to the signal before Fourier transform to suppress artifacts (side lobes). Note, however, that applying windows might shift your frequencies, particularly in the low-frequency range, towards higher frequency. Therefore, it is always good to compare non-windowed and windowed signals if you need to extract reliable nutation frequencies from your data.

All types of windows supported by scipy.signal.windows can be used. See there for further details. If a window requires additional parameters, use the parameter window_parameters. For details, see below.

Internally, the function scipy.signal.windows.get_window() is used to obtain the respective window. Note that only the right half of the (symmetric) window is applied to the data.

Default: None

window_parametersfloat or list

Parameters used for the window for apodising the data.

For details of data apodisation, see above.

Default: None

Type

dict

Examples

For convenience, a series of examples in recipe style (for details of the recipe-driven data analysis, see aspecd.tasks) is given below for how to make use of this class. The examples focus each on a single aspect.

In its simplest form, you just apply the analysis step to a dataset and assign it to a result:

- kind: singleanalysis
  type: TransientNutationFFT
  result: fft

To plot the data contained in the newly obtained calculated dataset, simply proceed with a plotter of your choice (in this case for a 1D dataset):

- kind: singleplot
  type: SinglePlotter1D
  properties:
    filename: fft.pdf
  apply_to: fft

Often the resolution in the frequency domain is insufficient if only the original signal is used for Fourier transform. One way to overcome this problem is to apply padding (zero-filling) to the data. From experience, a factor of 3-5 compared to the original signal length is a good idea:

- kind: singleanalysis
  type: TransientNutationFFT
  properties:
    parameters:
      padding: 5
  result: fft

In this case, the signal will be padded with zeros and be five times as long as the original signal.

Sometimes, time traces are dominated by an exponential decay rather than the oscillation. Fourier transform of such signals results in a large background particularly in the low-frequency range, obfuscating the unequivocal extraction of the frequencies of the transient nutations. One way to deal with this problem is to automatically fit and subtract an exponential to each time trace prior to Fourier transform.

- kind: singleanalysis
  type: TransientNutationFFT
  properties:
    parameters:
      subtract_decay: true
  result: fft

Another aspect often occurring during Fourier transform is apodisation to suppress artifacts (side lobes) in the frequency domain. Here, a window function is multiplied to the data (before padding). You can choose from a large range of window functions, actually all window functions that are supported by scipy.signal.windows. Typical windows are Hann and Hamming, but it seems that a cosine window has less effects on the positions of the maxima in the frequency domain.

- kind: singleanalysis
  type: TransientNutationFFT
  properties:
    parameters:
      window: cosine
  result: fft

In any case, take care of the effects windows and apodisation may have to the signals in the frequency domain beyond suppressing artifacts. If you want to use a window requiring extra parameters, provide these parameters as scalar or list:

- kind: singleanalysis
  type: TransientNutationFFT
  properties:
    parameters:
      window: kaiser
      window_parameters: 3
  result: fft

To compare the effect different windows have on the frequency spectrum, you can perform a series of transforms with different windows and compare the results graphically:

- kind: singleanalysis
  type: TransientNutationFFT
  properties:
    parameters:
      padding: 5
  result: fft
- kind: singleanalysis
  type: TransientNutationFFT
  properties:
    parameters:
      padding: 5
      window: hann
  result: fft-hann
- kind: singleanalysis
  type: TransientNutationFFT
  properties:
    parameters:
      padding: 5
      window: cosine
  result: fft-cosine
- kind: multiplot
  type: MultiPlotter1D
  properties:
    parameters:
      show_legend: true
    properties:
      drawings:
        - label: no window
        - label: Hann window
        - label: cosine window
    filename: fft-compare-windows.pdf
  apply_to:
    - fft
    - fft-hann
    - fft-cosine

Of course, you could compare other parameter settings as well in a similar manner.

New in version 0.2.

static applicable(dataset)

Check whether the processing step is applicable to the given dataset.

To be able to analyse transient nutations, a time axis needs to be present.