From frontier to field

01A ladder from idea to operation

Every instrument and method now trusted in the field was once an idea in a laboratory. The path between the two is long, and most ideas leave it somewhere along the way.

Technology readiness levels give that path a shared vocabulary. They are a measurement system for the maturity of a technology, with each project assessed against the parameters of each level¹.

There are nine levels. At the lowest, scientific research is beginning; at the highest, the technology has been proven in a successful mission¹.

The scale has spread far beyond space. Its implementation is a well-established standard in many agencies and industries, and the European Space Agency uses the scale defined in an international standard².

European research programmes adopted the same ladder in plain terms, from technology validated in a laboratory to an actual system proven in its operational environment³.

For observing systems, the ladder is a useful discipline. It asks, at every step, what has actually been shown, where it was shown, and what remains to be proven before people depend on it.

It also protects good ideas from bad timing. A technique pushed into service before its evidence is ready can fail in public and set back a whole field, while one that climbs steadily earns lasting trust.

The ladder is a guide, not a promise of success. Two techniques at the same level can carry very different risks, so the level should always be read together with the evidence behind it.

02Evidence at every step

Each rung of the ladder asks for a particular kind of evidence. Early on, analysis and laboratory results are enough; later, only performance in the real environment will do.

The middle levels separate validation in a relevant environment from demonstration there, and then from a system prototype demonstrated in an operational environment³.

Exit criteria make each step concrete. One widely used set asks for documented test performance demonstrating agreement with analytical predictions, and a documented definition of the relevant environment⁴.

Software follows the same logic. End-to-end software is tested in a relevant environment against predicted performance, and prototypes are shown on full-scale realistic problems before they are trusted⁴.

The step from the laboratory to a relevant environment is where most surprises appear. Dust, heat, cold, vibration, interference and power cuts expose weaknesses that a bench never shows.

When a test fails, the honest move is to step back down the ladder, fix the cause and climb again. A level reached once and then lost should not be reported as reached.

Readiness also belongs to a whole system, not a single part. A sensor at a high level is only as ready as the power, enclosure, link and software around it, so each of those needs its own evidence too.

03Test beds and intercomparisons

The clearest test for a new instrument is to stand it beside a trusted one and let both measure the same thing for a long time.

Observing communities have formalised this. Comparing sensors, instruments and equipment used operationally is an important means of determining their performance characteristics under field conditions, with laboratory tests added where required⁵.

The results of such intercomparisons are published with guidance for improving observations and conclusions useful to instrument developers⁵.

A test bed for a new technique works the same way. It hosts the candidate next to references, records both through seasons and weather, and publishes the differences openly.

Guidance on instruments and methods of observation then turns lessons from many comparisons into shared practice that every operator can follow⁶.

The value of a test bed grows with time and with diversity. A technique that works in one climate may fail in another, so the strongest evidence comes from several sites over several seasons.

Good test beds are also open. When results and raw records are shared, other teams can check the analysis, reuse the data and avoid repeating the same trials at their own expense.

04Running new and old side by side

Even after testing, a new method should not replace an old one overnight. Running both in parallel lets users see the differences on real cases before they depend on the new one.

Weather forecasting offers a recent example. A forecasting system based on machine learning entered operations on 25 February 2025, running side by side with the established physics-based system⁷.

It outperforms physics-based models for many measures, including tropical cyclone tracks, with gains of up to 20 per cent⁷.

Both systems start from the same initial conditions, built from observations, which keeps the comparison fair and keeps observation at the centre of both⁷.

Parallel running also reveals where a new method is weaker. Users learn which cases to trust it for, and which still call for the old method or a human check.

For field instruments the same approach applies. A new sensor reports alongside the old one for a full cycle of seasons, and the old one is retired only when the record shows the change is understood.

The overlap is also what protects long records. A climate series that switches instruments without an overlap may show a jump that has nothing to do with the climate and everything to do with the change of method.

05Learning systems need extra care

Techniques based on machine learning add a twist to the ladder. They can perform well in testing and still fail when the world drifts away from the data they learned from.

Risk management for AI asks that trustworthiness be considered across the design, development, use and evaluation of AI products, services and systems⁸.

In practice that means testing a model on data from places and seasons it has not seen, and stating plainly where it was not tested.

It also means monitoring after deployment. A model in the field should run in shadow beside the current method first, then be promoted, and keep reporting how its outputs compare with independent observations.

When it drifts, the response should be planned in advance: who decides to roll it back, what replaces it, and how the affected outputs are marked for users.

These steps are not a brake on new ideas. They are what lets people adopt a new technique with confidence, rather than waiting for someone else to take the risk first.

Data matters as much as the model. Record which observations trained each version, how they were checked and which gaps they had, so that a later failure can be traced to its cause.

06Leaving the lab with a plan

A technique is ready for the field when its evidence answers the questions the field will ask: how it behaves in bad weather, how it fails, how it is maintained and how its outputs are checked.

A field pilot should look like a real site, with real power, links, crews and weather, not a protected corner of a campus.

Write the exit criteria before the pilot starts. Decide what results would justify the next step, what would send the work back, and who will judge.

Publish what was learned, including what did not work. Shared results from many pilots raise the whole field faster than private success stories.

Plan the handover from the start. The people who will run the technique every day should help design the pilot, learn its quirks and decide with the research team when it is ready for them.

Nexus Observatory Inc. designs frontier research tracks around this path, carrying techniques from test beds to field pilots with the evidence recorded at each step.