TMPL Advanced usage¶
This page describes some of the more advanced features of TMPL.
Data: Where data can be stored in TMPL objects
Resources : Adding extra resources
Processing : How to add processing to a Measurement class using the process() method
Customisation: Using custom configuration for Measurements
Services : Make common functions available to all objects in a test sequence
Sequencing : How to make Measurements execute only under certain conditions or stages of the sequence
Data¶
TMPL objects have several internal data structures that can be used for static and temporary storage. These are the following:
config: static storage, usually defined in the object’s initialise method. config is intended for configuration settings that are set once per sequence run.
local_data: storage structure local to measurements and conditions. This can be used to store temporary data that is local to a Measurement or SetupCondition.
global_data: temporary storage available to all measurements and conditions. global_data provides a way for different measurements and conditions in a sequence to pass data between each other. It should be used with care because relying on it will make objects less modular.
All of these data structures are objects of the class ObjDict. This means they are basically dictionaries. They have all the same methods as a dictionary. The main difference is that the keys can be specified using the dot,’.’, notation. Some examples are shown below:
# Set/get a flag
test_seq.config.flag = True
flag = test_seq.config.flag
# Change number of averages in a measurement
test_seq.meas.Current.config.ammeter_averages = 16
# Set global data in test sequence
test_seq.global_data.my_name = 'fred'
# Read back global data in measurement
name = test_seq.meas.Current.global_data.my_name
# Set local data in measurement
test_seq.meas.Current.local_data.timer_count = 1
Adding extra resources¶
resources are usually added as a dictionary when a TMPL class is instantiated. Each item in the dictionary is added as a property to all TMPL classes under the main TestManager object.
There may be occasions where a resource is generated inside a TMPL class, e.g. a Measurement class, and is required to be available to other classes. An example might be an object that is created at the start of a test sequence, like creating a test instrument object.
In this case, all TMPL classes have a method for adding resources dynamically, add_resources. Passing a dictionary to this method will automatically make new properties in all TMPL class objects in the test sequence. The properties will take the names of the dictionary keys, so these must be in a suitable form, i.e. no spaces.
This code shows an example of using add_resources to make a server object at the start of a sequence:
class ConnectToServer(tmpl.AbstractMeasurement):
"""
Connect to server at start of test sequence
"""
def initialise(self):
self.run_on_startup(True)
# Set up configuration values
self.config.ip_address = '10.192.145.10'
def meas_sequence(self):
"""
Create server object and make it a resource for all
other classes
"""
# Make server
server_obj = make_server(self.config.ip_address)
# Add server to resources of all TMPL objects in sequence
self.add_resources({'server':server_obj})
Processing¶
Measurement objects can have an optional method, process(), which if defined in the class will be called automatically after meas_sequence(). This method is intended for processing data that has been measured in meas_sequence(). Like meas_sequence() there are no restrictions on what can be done in process() its name is just a guideline.
Here is an example of a Measurement class that includes a process() method:
class VoltageSweeper(tmpl.AbstractMeasurement):
"""
Example of a Measurement that adds its own coordinates and has
a process method
Measurement method:
* Sweep voltage
* Measure current at each voltage step
* Process voltage and current to calculate resistances
"""
name = 'VoltageSweep'
def initialise(self):
# Set up the voltage values to sweep over
self.config.voltage_sweep = np.linspace(0,1,10)
def meas_sequence(self):
# Do the measurement
current = np.zeros(self.config.voltage_sweep.shape)
for index,V in enumerate(self.config.voltage_sweep):
# Set voltage
self.voltage_supply.set_voltage(V,self.resistor)
# Measure current
current[index] = self.resistor.current_A
# Store the data
self.store_coords('swp_voltage',self.config.voltage_sweep)
self.store_data_var('current_A',current,coords=['swp_voltage'])
# Debug point
self.log('finished sweep')
@tmpl.with_results(data_vars=['current_A'])
def process(self):
# Get measurement data for current set of conditions
ds = self.current_results
# Fit a line to current vs voltage
p = ds.current_A.polyfit('swp_voltage',1)
# Get resistance from slope of line
resistance_ohms = p.polyfit_coefficients.sel(degree=1).values
# Store data into self.ds_results
self.store_data_var('resistance_ohms',[resistance_ohms])
The process() method in the example makes use of several TMPL features:
The @tmpl.with_results decorator is used to ensure that the data required for processing is present in the self.ds_results Dataset.
The current_results property is used to pull data from the last run of meas_sequence() into a local xarray Dataset that only contains data for the current SetupConditions.
The actual calculation makes use of xarray polynomial fitting to fit a line and get its slope.
Finally the result is store into self.ds_results using the store_data_var() method.
Like any other method in the class process() can also access data in the other data storage properties: config, local_data and global_data. These can all be used for passing data between methods.
The process() method can also be used as a top level method that calls other processing methods or functions.
def process(self):
"""
Top level process method, call other method to do
actual processing
"""
self.process_convert_degC_to_degK()
self.process_smooth_data()
self.process_curve_fit()
def process_convert_degC_to_degK(self):
# :
def process_smooth_data(self):
# :
def process_curve_fit(self):
# :
Processing only classes¶
There is no specific class for purely processing data. This is because it would just be the same as a Measurement class. However if you want to make a processing only class then it can either be done the same way as any other Measurement class, using the meas_sequence() method as the top level code or by creating an empty meas_sequence() and putting the main code in the process() method as in this example:
class ProcessOnly(tmp.AbstractMeasurement):
def meas_sequence(self):
# Empty method
pass
def process(self):
# Main code goes here
# :
Post processing¶
If post-processing is required after all measurements have been executed over all conditions then the class can be tagged to run only at the end in the teardown stage.
class PostProcess(tmp.AbstractMeasurement):
def initialise(self):
# Tag this class to run only at the end
self.run_on_teardown(True)
def meas_sequence(self):
# Empty method
pass
def process(self):
# Main code goes here
# :
Customisation¶
All the TMPL objects have a .config property. From its name it is intended to hold configuration data. This is usually static values that are required for performing measurements or processing.
class CustomisableMeasurement(tmpl.Measurement):
def initialise(self):
# Add customisable parameters to self.config
self.config.number_averages = 16
self.config.ammeter_range_A = 0.01
def meas_sequence(self):
# Use config setting in measurement
# Instrument setting
self.ammeter.range_A = self.config.ammeter_range_A
# Measurement setting
readings = []
for n in self.config.number_averages:
readings.append(self.ammeter.read_current())
ave_current = np.mean(readings)
The values defined in .config should be regarded as defaults. When running the Measurement from a TestManager sequence the values in .config can be changed for experimentation. For example if the CustomisableMeasurement class above were to be run from a TestManager sequence called seq then the .config settings can be changed by accessing them through the TestManagers meas property.
# Change a config value from TestManager object
seq.meas.CustomisableMeasurement.config.number_averages = 4
Global configuration¶
Setting configurations through each Measurement object may not always be desired, especially if Measurement objects share a common configuration value. For this reason TestManager objects have .config properties that will be copied to all Measurement and SetupCondition objects when the TestManager object is created.
class SeqWithGlobalConfig(tmp.AbstractTestManager):
"""
Test sequence that defines global config settings
"""
def initialise(self):
# Define global config settings here
self.config.length_units = 'cm'
self.config.storage_path = '/home/experimental_data'
def define_setup_conditions(self):
# :
def define_measurements(self):
# :
Another way to define global configuration settings is to pass a dictionary into the TestManager object when creating it. The contents of the dictionary will be copied into the .config property of the TestManager and all Measurement and SetupCondition objects contained inside it.
# Config values defined in external dict
my_config = {'serial_number':'B345',
'lab_name':'Maxwell_House'}
# resources
my_res = {'voltmeter':voltmeter_object}
# Pass config values as optional input argument
test_seq = SeqWithGlobalConfig(my_res,config=my_config)
# Access config values from TestManager object
test_seq.config.lab_name
# or measurement objects
test_seq.meas.VoltageSweep.config.serial_number
# or SetupCondition objects
test_seq.conditions.temperature.config.lab_name
The global config settings will also be available at the individual class level from self.config, for example:
self.config.lab_name
self.config.serial_number
Values passed in through a dictionary as shown above will overwrite config values with the same name defined locally in the classes. This is useful if several Measurement or SetupCondition classes rely on the same config setting. The default values can be defined locally in the class and overwritten by passing in a dictionary with same name as the local classes. The serial_number property in the code above is a good example of this. Many classes may want to know this value. The code below shows how this might work:
# Measurement classes
class Meas1(tmpl.AbstractMeasurement):
def initialise(self):
self.config.serial_number = 'default_ser_num'
# :
class Meas2(tmpl.AbstractMeasurement):
def initialise(self):
self.config.serial_number = 'default_ser_num'
# :
class TestSequence(tmpl.AbstractTestManager):
def initialise(self):
self.config.serial_number = 'default_ser_num'
# :
# Set serial number for all objects in test sequence
my_config = {'serial_number':'AG678'}
seq = TestSequence(resources,config=my_config)
Services¶
Services are functions that can be accessed by any of the TMPL objects. For example a service might be a function to convert metres into centimetres called m_to_cm. This would be called from inside a TMPL object like this:
length_cm = self.services.m_to_cm(length_m)
Services can be added in the following ways:
Adding in the define_services() method of the TestManager class
Tagging methods in Measurement or SetupCondition classes using the decorator @tmpl.service
These are described in more detail in the following sections.
Adding Services to TestManager class¶
Services can be added globally to the TestManager class using the optional method define_services(). Services are basically functions and can be added directly to the .services property of the TestManager as shown in this example
# Define a function to use as a service
def percent(fraction):
return 100*fraction
class ExampleTestSequence(tmpl.AbstractTestManager):
def define_setup_condition(self):
# ...
def define_measurements(self):
# ...
def define_services(self):
"""
Define service functions
"""
# Add service as function reference
self.services.percent = percent
# Or lambda function
self.services.meters_to_cm = lambda m: m*100
# dict style
self.services['kg_to_g'] = lambda kg: kg*1000
Services defined in define_services should be stand alone functions. However services can also be derived from Measurement or SetupCondition classes as described next.
Services from Measurement and SetupCondition classes¶
Sometimes data generated or measured in one Measurement or SetupCondition class is useful in another class. For example a Measurement class may take a series of readings that act as a look up table for other Measurement classes. This data could be pushed into the ds_results Dataset or global_data or local_data so other classes could access it. In the case of a look up table each class that wanted to use it would have to implement its own code for cross referencing through the table. It would be more convenient if the class that generated the look up table could provide a “service” that implements the cross referencing code. Then other classes can just call the service function. This can be done by tagging class methods with the @tmpl.service decorator.
Any class method can be tagged using the @tmpl.service decorator to turn it into a service. The following example shows one class creating a service and another class using it.
class MeasurementWithService(tmpl.AbstractMeasurement):
"""
This measurement class takes some readings and makes the results
available for others via a service called "lut_lookup"
"""
def meas_sequence(self):
# Top level measurement sequence
# Takes readings that are logged into a table in self.local_data
# Measurement code
# ...
# Store data locally
self.local_data.lut_dict = lut_measured
@tmpl.service
def lut_lookup(self,key_value):
"""
Provide a lookup service to other classes
Parameters
-----------
key_value: str
key to cross reference in lookup table
"""
return self.local_data.lut_dict[key_value]
class MeasurementUsingService(tmpl.AbstractMeasurement):
"""
This measurement uses the "lut_lookup" service
"""
def initialise(self):
self.config.lut_key = 'chamber_id'
@tmp.with_service(['lut_lookup'])
def meas_sequence(self):
# Get value from lookup table
self.services.lut_lookup(self.config.lut_key)
# Measurement code
# ...
In the example the second Measurement class that uses the service has a decorator tmpl.with_service. This accepts a list of service names. The decorator just checks if the services listed are available and if not will throw an error. The with_service decorator is entirely optional, it just adds a layer of robustness.
Alternatively the services available can be checked at any time by checking the property services_available which is a list of service names. For the lookup table example this might be something like:
if 'lut_lookup' not in self.services_available:
raise RuntimeError('No service')
Although this section has presented all the examples using Measurement classes, the same can be used from within SetupCondition classes as well.
Sequence states¶
When a TestManager sequence is executed using the run() method it creates a loop. Before starting the loop a table of setup conditions is created. Each row of the table represents a combination of setup conditions. Each iteration of the loop is one row of the table. Within this loop there are particular states where Measurement classes can be executed. These states are Startup, Setup, Main, After, Teardown and Error. The sequence loop runs in the order below, with the states shown in bold.
Build setup conditions table
Startup : Run anything that needs setting up
- Load first set of conditions from table
Setup stage
- For each condition in the set
Set condition to current value
Run any measurements for this condition
Main : Run main measurements
After : Run teardown for current set of conditions
Teardown: Run global teardown measurements, e.g. shutdown, store results
Error : Run when error occurs.
Measurement classes can be set to execute in one or more of these states using methods with the prefix run_.
run_on_startup : Run before any conditions have been set at the Startup state
run_on_setup : Run after a specific condition has been set in the Setup state
run_on_main : Run in the main loop. This is the default state and does not need to be explicitly set.
run_after: Run after all main measurements have been executed for one set of conditions.
run_on_teardown: Run at the end of the sequence. This is useful for shutting down.
run_on_error: Run when an error occurs
Sequencing example¶
This section will go through an example of sequencing measurements. We will start with the TestManager definition for the simple resistor measurement.
class ExampleTestSequence(tmpl.AbstractTestManager):
"""
Example test sequence
Runs a dummy measurement sequence over temperature and humidity conditions.
Measurement sequence is:
* Turn on equipment
* Wait for stabilisation
* Run a voltage sweep
* Turn off equipment
"""
name = 'ExampleResistorTest'
def define_setup_conditions(self):
"""
Add the setup conditions here in the order that they should be set
"""
self.add_setup_condition(TemperatureConditions)
self.add_setup_condition(HumidityConditions)
def define_measurements(self):
"""
Add measurements here in the order of execution
"""
# Setup links to all the measurements
self.add_measurement(TurnOn) # Run on startup
self.add_measurement(Stabilise) # Run after temperature is set
self.add_measurement(VoltageSweeper) # Run in main loop
self.add_measurement(TurnOff) # Run on teardown
# Go here if errors occur
self.add_measurement(HandleError) # Run on error
Looking at the define_measurements() method at the end of the class definition, the comments indicate where we want each Measurement class to run. There are two ways to set when Measurement classes run:
Set run state in class definition¶
The first method is to set the run state from within the Measurement class definition itself. The code below shows the class definitions for TurnOn, TurnOff, Stabilise and HandleError. For each class the run state is set in the initialise() method.
class TurnOn(tmpl.AbstractMeasurement):
"""
Turn on all equipment
"""
def initialise(self):
self.run_on_startup(True) # <- Set run state
def meas_sequence(self):
self.log('TurnOn measurement')
class TurnOff(tmpl.AbstractMeasurement):
"""
Turn off all equipment
"""
def initialise(self):
self.run_on_teardown(True) # <- Set run state
def meas_sequence(self):
self.log('TurnOff measurement')
class HandleError(tmpl.AbstractMeasurement):
"""
Handle any errors generated
e.g. shut off critical equipment, store data etc
"""
def initialise(self):
self.run_on_error(True) # <- Set run state
def meas_sequence(self):
self.log('Error handler')
class Stabilise(tmpl.AbstractMeasurement):
"""
Wait for stabilisation
- only run this after temperature has be set
"""
def initialise(self):
# Run in the "Setup" stage only after
# temperature has been set
self.run_on_setup('temperature_degC')
def meas_sequence(self):
self.log('Stabilising')
Setting the run state is mostly a boolean operation. However when using run_on_setup the name of the condition is specified. This is the value of the name property of the SetupCondition class. If name was not specified in the class definition then it will default to the class name. In the example above it is assumed that name in the TemperatureConditions class is “temperature_degC”.
Set run state from define_measurements¶
An alterative way to set the run state is from inside the TestManager class definition. This is better if the same Measurement class is reused in different sequences where it might be required to run in different states.
Here is a revised version of the define_measurements method with the run state settings added:
def define_measurements(self):
"""
Add measurements here in the order of execution
- Also add the run state of each measurement
"""
# Setup links to all the measurements
self.add_measurement(TurnOn,run_state=self.RUN_STAGE_STARTUP)
self.add_measurement(Stabilise,
run_state={self.RUN_STAGE_SETUP:'temperature_degC'})
# Runs in main loop no need to set state
self.add_measurement(VoltageSweeper)
self.add_measurement(TurnOff,run_state=self.RUN_STAGE_TEARDOWN)
# Go here if errors occur
self.add_measurement(HandleError,run_state=self.RUN_STAGE_ERROR)
The define_measurements method has an optional keyword argument run_state which takes a string, list or dict as the input. For convenience the string values are all properties of the TestManager class with the prefix RUN_STAGE_. They are listed below:
# Run stages
# These labels are used to reference all the stages where a measurement
# can be run
RUN_STAGE_STARTUP = 'STARTUP'
RUN_STAGE_TEARDOWN = 'TEARDOWN'
RUN_STAGE_SETUP = 'SETUP'
RUN_STAGE_MAIN = 'MAIN'
RUN_STAGE_AFTER = 'AFTER'
RUN_STAGE_ERROR = 'ERROR'
