Articles › Model Based System Engineering (MBSE)

Shift-Left Verification with Executable Models & Digital Twins

By Mike WaynePublished

On 4 June 1996, Ariane 5 flew for 37 seconds. Then both of its inertial reference units shut down, the rocket swung off course and broke up, taking four Cluster science satellites with it.

The software that failed had been reused from Ariane 4, where it had worked for years. The inquiry board found that Ariane 5's own trajectory data had never been part of that unit's specification, and that the unit was never run against a realistic Ariane 5 flight profile before launch. A horizontal velocity value larger than anything Ariane 4 produced overflowed a conversion, and the unit stopped.

The fault was sitting in the design for years. Nobody ran the design against the mission it was actually going to fly. Shift-left verification is the practice of doing exactly that, early, against models, while a fix is still a few lines of change rather than a lost vehicle.

What shift-left means

The term comes from software. Larry Smith coined it in a 2001 Dr. Dobb's Journal article, "Shift-Left Testing", arguing that testers belonged at the start of a project, not queued up at the end of it. Draw the project as a timeline running left to right and the testing moves left.

In systems engineering the timeline is usually drawn as the V-model. The left arm goes down through requirements, architecture and detailed design. The right arm climbs back up through unit test, integration and system test, and each level on the right verifies the matching level on the left.

The traditional V does all its verification on the right arm, after something has been built. Shift-left keeps every one of those verification links and runs as many of them as it can on the way down, against an executable model of the thing rather than the thing itself.

The cost curve behind it

The case for moving left is a cost curve, and the most cited version of it comes from NASA. In 2004 a team at Johnson Space Center published "Error Cost Escalation Through the Project Life Cycle", which pulled together earlier studies and NASA's own project data to estimate what it costs to fix a requirements error depending on when it is found.

Error found duringRelative cost to fix
Requirements1x
Design3x to 8x
Build7x to 16x
Integration and test21x to 78x
Operations29x to more than 1,500x
Cost to fix a requirements error, relative to fixing it in the requirements phase. Source: Stecklein et al., NASA Johnson Space Center, 2004.

The ranges are wide because projects differ, but the shape never changes. The same mistake gets an order of magnitude more expensive somewhere between design and test, because by then it has hardware, drawings, code, suppliers and other decisions built on top of it. Every one of those has to be unpicked.

Shift-left is an attempt to find more errors in the first two rows.

What you can check on the left arm of the V

A drawing cannot be tested. An executable model can. That is the whole difference: a model with precise behaviour can be given inputs and run, and its outputs can be compared with what the requirements say should happen. Semantics precise enough to run are a large part of why SysML v2 was rebuilt from the ground up rather than patched.

What that buys you depends on what you model:

What you checkAgainst whatThe kind of fault it finds
Requirement consistencyThe requirement set itselfTwo requirements that cannot both be met, or a value that is never defined.
Behavioural logicA state machine or activity modelA mode the system can enter and never leave, or a command issued in the wrong state.
InterfacesThe interface definitions, with unitsOne side sending pound-force seconds while the other expects newton-seconds.
Performance and timingA physics or timing simulationA response time that is met at room temperature and missed in the cold.
Operating envelopeThe real mission or duty cycleReused equipment meeting inputs it was never designed for, as on Ariane 5.

The interfaces row is not hypothetical either. NASA lost the Mars Climate Orbiter in September 1999 because ground software supplied thruster impulse data in pound-force seconds when the interface specification called for newton-seconds. The navigation error built up over the nine-month cruise and the spacecraft arrived at Mars far too low. An interface model that carried units would have refused to connect the two sides.

Worked example: a door latch at minus 30

Here is shift-left on a smaller scale than a launch vehicle. The programme is illustrative, but the steps are the ones a real team would go through.

An electric vehicle has flush door handles and an electrically released latch. The requirement reads:

REQ-DR-041: The door latch shall release within 200 ms of handle actuation across the full operating temperature range of minus 30 °C to plus 85 °C.

Followed both ways:

StepVerified on the right arm onlyShifted left
DesignLatch motor chosen from the supplier datasheet at 20 °C. Timing looks comfortable.Latch modelled as motor, gearing and spring, with friction and supply voltage as inputs. Simulated across the temperature range in the first week.
FindingNone yet.At minus 30 °C, stiffer grease and the low-voltage supply at a cold start push release time to about 260 ms. REQ-DR-041 fails in the model.
FixNone yet.A higher-torque motor variant is selected. The model is rerun and passes at 170 ms. The change costs an afternoon and a revised part number.
Prototype buildDoors built and fitted with the original motor.Doors built with the motor that already passed in simulation.
Cold-chamber testRelease time fails at minus 30 °C. Motor change, new bracket, new wiring harness, retest, and a slipped validation milestone.Passes. The test confirms the model rather than discovering the problem.

The physical cold-chamber test happens in both columns. Shift-left did not remove it. It changed what the test was for: in the right-hand column the test confirms an answer the team already had, instead of being the first place anyone asked the question.

Where digital twins fit, and where they don't

Digital twins get mentioned in almost every article on shift-left, and the claim needs care. A digital twin in the strict sense is a model kept in step with a specific physical system using data from that system. Before the hardware exists there is nothing for it to twin. What you have at that point is a simulation or a virtual prototype, which is exactly the right tool for the left arm.

Twins earn their place in shift-left one step later. A twin of a fleet already in service gives you measured operating data: real temperatures, real duty cycles, real wear. Feed that into the models for the next variant or the next upgrade and the left-arm simulations run against conditions the system has actually seen, not assumed ones. On Ariane 5, the trajectory that broke the reused software was known before launch. The problem was that nobody ran it.

What shift-left does not replace

A model only shows you what you put in it. If the latch model leaves out the low-voltage condition, it will pass happily at 170 ms and the cold chamber will still find the fault. That is why models need validating against test data too, starting with the simplest component tests, so the team knows how far each model can be trusted.

For the same reason, shift-left does not remove qualification testing, and regulators in safety-critical industries will still expect evidence from the real system. What changes is the likely result. The NASA figures above are about when an error is found, not whether the right arm exists. Find most errors in the first two rows and the right arm stops being where the bad news arrives.

What it takes to make it work

  1. Requirements that can be checked. A model cannot test "the latch shall release quickly". It can test "within 200 ms across minus 30 °C to plus 85 °C". Writing requirements to the standard in the INCOSE Guide to Writing Requirements is the first step of shift-left, not a separate activity.
  2. A verification method on every requirement from day one. Decide at the moment of writing whether each requirement will be verified by analysis, simulation, inspection, demonstration or test, and which model or test carries it. A requirement with no planned method is one nobody will check early.
  3. Executable models for the risky parts. You do not need to simulate the whole system. Start where a late failure would hurt most: interfaces, timing, safety functions and anything reused from a previous programme.
  4. Traceability from requirement to result. Each simulation run is evidence. It has to be linked to the requirement it verifies, with the model version and the result, or the early verification is lost the moment the engineer who ran it moves on.

The last point is where most teams struggle, because the evidence ends up scattered across simulation folders. Reqi records acceptance criteria and verification evidence against each requirement in the same register, so the early simulation result for REQ-DR-041 sits next to the later cold-chamber result. See requirements traceability software for how that link is kept, and compliance reporting for the live view of which requirements are verified, by system and phase.

Frequently asked questions

What is shift-left verification?

Shift-left verification means checking requirements and design behaviour earlier in the lifecycle, usually against executable models and simulations, instead of waiting for integration and system test. On the V-model it runs the verification links of the right arm while the project is still on the left arm.

Why is it called shift-left?

If a project is drawn as a timeline running left to right, testing traditionally sits at the right-hand end. Moving it earlier moves it left. Larry Smith used the term in a 2001 Dr. Dobb's Journal article on software testing, and systems engineering adopted it for model-based verification.

Do you need a digital twin to shift left?

No. Before hardware exists, shift-left relies on executable models and simulation. A digital twin needs a physical system to mirror, so it helps most on later variants and upgrades, where measured data from systems in service makes the early simulations more realistic.

Does shift-left replace physical testing?

No. Models only cover what they include, and safety-critical industries still require evidence from the real system. Shift-left changes the result of physical testing: most faults have already been found and fixed, so the test confirms the design instead of discovering problems with it.

Related guides