Considering Scope: Enterprise Architectures and Agility

In today’s era of “big data” permeating the meteorological community, it is easy to find examples of enterprise architectures that process, store, and serve weather information to the internal and external users. Implemented with the intent to further a strategy, and organize and homogenize processes, these enterprise architectures are multi-faceted in the purposes that they further. For example, a single instance of enterprise architecture must, at least, serve certain business or community requirements; meet information technology standards; integrate data models; provide access and customization for core users and related but diverse partners; and connect with applications.

In a previous post, an ideal enterprise-grade platform for R2O was postulated. But are enterprise architectures always the right solution? Can they be “too big to fail”? And what are the implications for R2O?

Proponents of enterprise architectures will revel about organizational, process, and information technology efficiencies that they afford. For some situations where the requirements are focused, such efficiencies can be realized. For example, the enterprise architecture for a system that serves as a data archive for observations of a certain type can be sufficiently isolated. However, if the architecture is designed to accommodate a wide range of functions, what is gained in efficiency is lost in complexity. Suppose the aforementioned data archive architecture and system is expanded to include observations of all types, from satellite imagery to aircraft reports, and now must support real-time users with delivery formats that meet unique specifications.

Envision that architecture. It could incorporate the best of the existing storage and dissemination models for each data type. It may be ideal for the members of the community that need all meteorological data from a single interface. The hardware may be scalable to adapt to the number of users and amount of data that must be served. It seems attainable.

Now, consider a different perspective: the operations, maintenance, and evolution. This expanded architecture requires several different data models, as meteorological data is stored different formats. Sometimes the same data must be cast into different formats, as there are different applications. The upstream collection systems are diverse and the paths to the archive vary. These systems and paths continually vary with upgrades over time.

The amount of heterogeneity within the enterprise architecture contravenes its agility. The scope multiplies with each new user, application, and data model. The degree of the multiplier is based on the uniqueness of an item relative to the others. Eventually the growing scope presents a challenge that becomes insurmountable and splinter projects form to circumvent the behemoth architecture.

This is real and it happens, especially with a bureaucracy characteristic of the public sector that enshrines and follows processes indiscriminately. If innovation requires agility, innovation may not be sustained under the umbrella of enterprise architecture. More importantly, it does not need to.

Building architectures for projects of small enough scope, such that the elements are isolated and confined to a subset of purposes and users, can provide benefits in development flexibility in contrast to the larger enterprise architecture because it does not have to be “everything to everyone”. The evolving projects internal to smaller architectures are also in a safer position to fail than if they were incorporated into the enterprise architecture. That does not mean that architectures of limited scope can be desultory. To the contrary, they must be well planned to fit a specific and unmet need. Enterprise architectures are best at supporting low risk solutions in any aspect of its purposes (to attain high availability, for instance, in the case of information technology systems) because it is difficult to isolate risk for integrated projects.

In R2O, it is hard to conceive an enterprise architecture that is expansive in reach and diverse in utility such that all of the partners involved in a transition cycle are united and satisfied with the solution. For example, the federal government and its university collaborators do not share the same information technology systems. The operational weather visualization software at the National Weather Service is not widespread beyond the agency, and support is not provided to all stakeholders. While R2O efforts must try to adapt to implemented technical systems where they already exist, especially in the short term, this should not deter questioning whether the best solutions are in place, especially when looking toward the future.

Innovation and collaboration that are invigorated through R2O activities can establish new directions and generate important growth. At a minimum, they can fulfill specialized needs. In ideal cases, they can provide a new proof of concept that may be beneficial at an enterprise level, whether that is a new business service or technical enhancement. While the risks are high, the costs are low, especially early on. But this is probably acceptable with today’s “fail fast” modus operandi. Enterprise architectures require careful coordination amongst many stakeholders, especially as they evolve. A few people can start a new architecture of limited scope and retool it without much external disruption during the early crafting stages. R2O efforts must be mindful of these differences and select the nature of architecture that is most suitable for a project. In many cases, that means thinking small in scope, and big in data and ideas.