Unit Testing VNA Calibration Code

The Problem

There are an alphabet soup of VNA calibration types (SOLT, TRL, LRL, TRM, SOLR, QSOLT, many more), some of which will be supported by the Harmon Instruments VNA and run by the end user. Testing all of the supported calibration types through the normal user interface and measuring verification components for each could easily take an entire day. Using physical calibration standards, uncertainties would be limited to those of the standards. An additional concern with testing on hardware is the wear and tear on the fragile and expensive calibration standards.

Photo of VNA Calibration Standards

Breaking it Down

The data path from ADC values to corrected S-parameters for display is split into layers with independent test coverage. The lowest level, implemented in an FPGA, taking signals from the ADCs has test coverage using the excellent CocoTB package. The next layer, written in C++, takes integer vector voltage data from the FPGA and produces raw S-parameters. These first two layers are relatively simple mathematical operations. The final layer, correcting the raw S-parameters, is where the complexity lies and is what will be discussed here.

Modeling the VNA and Calibration Standards

For simplicity, we will focus on a 2 port VNA calibrations using an 8 term error model (TRL, SOLR, similar). This model contains a fictitious 2 port "error adapter" to either side of the device under test (DUT). These represent internal leakages, losses, gain imbalances, etc as well as the test port cables. The raw S-parameters are simply those of the DUT in cascade with the two error adapters. In a calibration, sufficient standards are measured to solve for the error adapters.

Knowing the contents of the error adaptors, it is possible to run the calibration backwards and calculate the raw S-parameters from the true S-parameters of the devices being measured. Realistic models of the calibration standards are used with airlines, opens and shorts having loss. Simulated raw data is genereated for the required set of standards as well as a few challenging DUTs and the data is sent throught the calibration acquisition and correction software. The corrected data is compared with the originals and verified to be within reasonable rounding error. The tests on the calibration algorithms complete in less than 1 second and run every time the software is built causing the build to fail if any test does.

For the purposes of testing the calibration software, the VNA is presumed to be linear and time invariant. In the future, for uncertainty analysis, it may be desirable to add drift and nonlinearity based on characterization of real hardware.

Offline Processing of Acquired Data

By saving raw data for many types of calibration and verification standards, it is possible to generate corrected S-parameters using any calibration type using those standards. QSOLT had not yet been coded when this data set was aquired, but it is possible to generate QSOLT corrected data as all of the required standards are present in the data dumps. This is mostly useful to see how the algorithms compare with imperfect standards.

Plot of
  Maury Microwave 60 mm airline S22 with QSOLT, TRL, TRM and data
  based QSOLT calibrataions

The above plot shows significant differences between calibrations assuming the load standards are ideal (QSOLT, TRM), TRL which uses airlines rather than loads and a data based QSOLT calibration which uses more accurate models of the short, open and load. These differences are not due to software, but the physical standards and their definitions.

Implementation

The S-parameter handling code (C++) on the Harmon Instruments VNA has a SWIG generated Python wrapper and the Python unit testing framework is used to run the tests. Google Test would have been another option. Python works well here due to libraries like NumPy, SciPy and Matplotlib providing mathematical functions and plotting. For instance, to test a C++ implementation of the Kaiser window, I added a test to compare it with one generated by SciPy.