Skip to content

calibrate SSP parameters against reference data - #1675

Merged
adrpo merged 3 commits into
OpenModelica:masterfrom
adrpo:ssp-calibration
Oct 20, 2026
Merged

adrpo merged 3 commits into
OpenModelica:masterfrom
adrpo:ssp-calibration

Conversation

@adrpo

@adrpo adrpo commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Fixes #1674.

Adds OMSimulator.calibration. It fits the parameters of a Calibration activity of the OpenSCALING UQ layered standard to the reference data in a CSV file, on an SSP. The fit is bounded nonlinear least squares (scipy.optimize.least_squares). The UQ file is the one the SSD references by ssc:MetaData of type application/x-uncertainty-quantification.

OMSimulator --calibrate cal_01 --calibrationOutput calibrated.ssp model.ssp
from OMSimulator.calibration import calibrateSSP
result = calibrateSSP("model.ssp", "cal_01", output="calibrated.ssp")

How it works

  • The SSP is instantiated once. Each evaluation runs reset, sets the fixed ParameterSet values and the targets, runs initialize, then stepUntil each reference time point and getValue on the observed variables. No result file is written.
  • The model starts at its own start time. A calibration window only selects which samples are compared.
  • Results go back into the UQ file. The ResultSet gets an ErrorMetric type="RMSE" per observed variable and refers to <activity>_calibrated.ssv, an SSP parameter set with the calibrated values written next to the UQ file. Both validate against their schemas (UQ specification-v1, SSP 2.0).
  • The standard does not yet define the column mapping, weights, time window or optimizer settings. They are read from <uq:KeyValues type="org.openmodelica.omsimulator.calibration"> in the SimulationSetting, which the schema allows: dataName:<var>, weight:<var>, startTime, stopTime, scaling, relativeTolerance, maxEvaluations, diffStep. By default an observed variable maps to the CSV column with its name, optionally followed by |unit.
  • numpy and scipy are imported on first use: pip install OMSimulator[calibration]. The omsimulator build-deps images already have scipy.

Tests

calibration/HelloWorldCalibration.py fits a and x_start of HelloWorld (der(x) = a*x) to exp(-t). cal_01 uses all samples; cal_02 uses only t ≥ 0.5 while the model still starts at 0. Both recover a = -1, x_start = 1. Checked that cal_02 fails (x_start = 0.6065) if the simulation starts at the first compared sample. Run with runtest.py against an installed OMSimulator (pin ffc28b1) plus this branch's Python sources.

Validation on the OpenSCALING DC-motor use case

This is the OpenSCALING DC-motor calibration (d2.1-uq-layered-standard, use_cases/dc_motor, activity cal_01): 6 parameters, 502 noisy samples of speed and current at 13 V / 0.25 N·m. The SSP was rebuilt from three FMUs exported by omc 1.28 as model exchange (one CVODE integrates the whole system), and it matches the omc simulation of DCMotor_L3 to six digits. The run takes 485 evaluations and 15 s:

parameter nominal this PR reference notebook
R 0.1 0.099968 0.099973
L 1e-4 1.0246e-4 1.0348e-4
c_m 0.01 0.010013 0.0100116
c_g 0.01 0.010215 0.010696
J 5e-5 5.132e-5 5.426e-5
d 1e-4 1.067e-4 1.24e-4
cost 104360 104041 104740

Using the use case needed three changes, which were made in the test SSP, not in this PR:

  • ParameterSet names constantVoltage.V / constantTorque.tau_constant are calculated parameters in an FMU; Vbattideal / Tmechload were used instead.
  • The observed level_3_dc_motor.I is the negative of the CSV column dc_motor_electric1.I, so dc_motor.dc_motor_electric1.I is observed.
  • The uq:Settings element is no longer in the schema and was dropped.

With co-simulation (the default for me_cs FMUs) the fit is off (c_g at its upper bound), because the electrical time constant of 1 ms is no larger than the communication step.

Not in this PR

  • Every evaluation prints the native info: lines (step size, final statistics), since info messages cannot be silenced from Python.
  • No unit conversion between observed variables and CSV columns.
  • HTML report.

Error reporting

An invalid calibration problem (unknown activity id, no UQ file, unknown observed variable, ...) is reported as a single error: line on stderr with exit status 1, not a Python traceback.

Coverage workflow

The calibration test needs numpy and scipy, which the bare GitHub runner of the coverage workflow did not have. The job now runs in docker.openmodelica.org/build-deps:ubuntu-24.04-omsimulator, the image the linux64-noble Jenkins stage uses, so coverage and Jenkins test in the same environment. OpenModelica/build-deps#101 also adds numpy and scipy to every apt image, not only the omsimulator add-on.

OMEdit

The OMEdit front end is OpenModelica/OpenModelica#17360 (issue OpenModelica/OpenModelica#17359): a Calibrate dialog for SSP models that runs OMSimulator --calibrate. It needs an OMSimulator submodule bump after this PR is merged.


Generated by Claude Code

🤖 Generated with Claude Code

Add OMSimulator.calibration: fits the parameters of a Calibration activity
of the OpenSCALING UQ layered standard to the reference data of a CSV file
by bounded nonlinear least squares (scipy.optimize.least_squares). The UQ
file is the one the SSD references by ssc:MetaData of type
application/x-uncertainty-quantification.

* each evaluation resets the instantiated model, sets the parameters,
  initializes and steps through the reference time points, no result file
* the model starts at its own start time; a calibration window only
  selects the compared samples
* RMSE per observed variable as ErrorMetric in the ResultSet, calibrated
  values as <activity>_calibrated.ssv next to the UQ file
* column mapping, weights, time window, scaling and optimizer settings,
  not yet in the standard, from a KeyValues element of type
  org.openmodelica.omsimulator.calibration in the SimulationSetting
* command line: OMSimulator --calibrate [ACTIVITY] [--calibrationOutput]
  [--uqFile] model.ssp
* pip extra OMSimulator[calibration] for numpy and scipy
* test calibration/HelloWorldCalibration.py fits a and x_start of
  HelloWorld to exp(-t), over all samples and over t >= 0.5

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@adrpo adrpo self-assigned this Oct 20, 2026
A wrong activity id, a missing UQ file or an unknown observed variable
made `OMSimulator --calibrate` print a Python traceback. They are user
errors, so print them as a single "error:" line on stderr and exit
with status 1. OMEdit shows that line in its calibration dialog.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@arun3688 arun3688 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good

calibration.HelloWorldCalibration failed in the coverage workflow with
"No module named 'numpy'": the bare GitHub runner only had what the
workflow installed with apt, and the calibration needs the optional
OMSimulator[calibration] dependencies.

Run the job in docker.openmodelica.org/build-deps:ubuntu-24.04-omsimulator
instead, the image the linux64-noble Jenkins stage already uses. It has
the compilers, flex, ccache, gcovr and the testsuite's Python packages
(numpy and scipy included), so the apt step goes away and the coverage
runs see the same environment as Jenkins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 28.35%. Comparing base (8a9d93d) to head (cba3cee).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1675      +/-   ##
==========================================
+ Coverage   27.69%   28.35%   +0.65%     
==========================================
  Files          72       72              
  Lines       13976    13976              
  Branches     9020     9020              
==========================================
+ Hits         3871     3963      +92     
+ Misses       8876     8775     -101     
- Partials     1229     1238       +9     
Flag Coverage Δ
linux 28.35% <ø> (+0.65%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI/disable macOS-arm64 project/openscaling OpenSCALING related issues and pull requests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Calibrate SSP parameters against reference data (UQ layered standard Calibration activity)

2 participants