The role of the data integrator and the future of practitioners

In the field of meteorology, recent discussion has centered on whether current and evolving technologies have the capacity to push human intervention to the margins of weather forecast and warning functions. This is a loaded question, and one that has consequences for R2O and its role in improving weather services. However, the consideration and debate surrounding the transition of weather predictions from a predominantly human role to a computer capability must include a discussion of how the intersection of boundaries internal to the field and its organizations, and technology, brought the weather enterprise to its current operating state.

The United States National Weather Service has long maintained a workforce of operational meteorologists that are responsible for interrogating meteorological data and formulating predictions intended for public dissemination. Over time, the amount of meteorological data has grown. There are more observations, more precise observations, and more frequent observations. They come from a variety of sources. Numerical weather prediction models assimilate a lot of this data and resulting output is available to operational meteorologists to provide guidance in the forecast process. These meteorologists have the challenging task of assessing whether a given model is representative of current conditions, and if it is not, how the future forecast may change.

But there are multiple models, and courtesy of ensemble frameworks, multiple solutions. Improvements to the model dynamics and physics, coupled with technological advancements that have enabled faster computing, have resulted in the resolution of the atmospheric state at increasingly smaller scales. Some models run every hour, others every six or twelve hours, and produce output from days to weeks ahead.

Despite advances in model skills, these models will not encumber the benefit of the human forecaster universally in an instance, precisely because weather and weather predictions happen on different spatial and temporal scales, and at variable spatial and temporal resolutions. Predictability is a function of time, space, and complexity. Meteorology as a science limits practitioners and mathematical methods alike to understand more about what will generally happen over a large area within the next hour during calm weather than what will specifically happen at a point location two weeks from the present.

Furthermore, numerical weather prediction models do not incorporate subjective weather reports from people that can allude to impacts that are important to the public or an industry. Even before social media enabled the public to submit photos, videos, and written descriptions of interesting weather events, pilots could submit reports of turbulence to alert others of rough air ahead, and signal to meteorologists the ultimate chaos in the atmosphere. This is valuable information, but it is easily lost in a sea of data. Computers cannot easily quantify or verify reports like this, but practitioners are able to.

Trusting numerical weather predictions without condition or consideration is perilous, but technology can and should assist practitioners, and in a big way. However, in meteorology, a chasm is opening between human forecasting methods and computer forecasting algorithms, exposing their lack of interoperability. To date, the weather enterprise has asked the practitioner to perform the function of the “data integrator”, consuming all of the meteorological data and making sense of it. Operational meteorologists have long served as the arbiter between the observations and the model output, adjusting the forecast provided to customers and the public accordingly. Unfortunately, as the amount of data continues to increase, numerical methods leveraging vast networks will undoubtedly have to play a role in integrating data. But it does not mean that computers will suddenly assume the function of the “data integrator”, nor is this necessarily a role that numerical weather prediction models are positioned to play. Instead, computers should serve as an intermediary to combine diverse, redundant, and complementary data.

The intent of integration is not to have computers produce a forecast or warning decision. Instead, deliverables of integrated data from numerical methods should compel the practitioner to refine their emphasis of the data germane to the situation. This is going to be a challenge because, to date, there has only been limited effort in developing numerical techniques in meteorology outside of numerical weather prediction. To further complicate matters, it is likely that the practitioners’ mental model has not yet been mapped for all situations. Unless we understand the thought process behind a certain function that a practitioner currently serves, it is difficult to achieve a deliverable that contributes a package of meaningful information and nothing more. R2O can provide a process to conduct this analysis of the practitioners to enhance the science and new methods.

Lastly, we must not forget that practitioners still exhibit a marked advantage over computers in interpreting complex spatial patterns. If you need evidence of this, look no further than the CAPTCHA codes you enter when signing into certain secure web pages. While that may change in the distance future, the atmosphere’s unique structures will keep operational meteorologists employed for quite some time, not only because adequately trained practitioners can resolve features in spatial data, but because they understand the likely impact of such features on the sensible weather, and can convey that to those they serve.

To summarize, R2O should (1) enable practitioners to focus on types of data that are not easily converted for a computer to interpret, or where humans have an advantage in interpretation and (2) develop methods that suitably inform practitioners with necessary information that is not duplicative or extraneous. Use technology and numerical methods to integrate intermediary data to a manageable amount for the practitioner. Focusing on areas where the practitioner can add communicative and interpretative value will ensure that the broader organizations within an enterprise are best applying the technological and human resources available.