The transmission and distribution system of the future will be more sophisticated than the smart grid envisioned today. What will shape the nature of the system and what will the requirements of its architecture be?



The Porsche 918 plug-in hybrid unveiled at the 2010 Geneva Motor Show Photo courtesy of Porsche

Jon Arnold & Larry Cochrane, Microsoft Corporation, USA

The worldwide power and utility industry is rethinking the structure, engineering and fundamental mission of the world’s utility systems. New forces are combining to encourage energy market participants to improve the grid’s sophistication, complexity and automation. These include climate change, rising demands for energy and the addition of new market participants such as plug-in hybrid electric vehicles (PHEVs).

Indeed, industry visionaries believe tomorrow’s smarter grid will not only need to accommodate flows of power but also information about its consumption and pricing. These flows will pass bidirectionally between utilities and their consumers, especially as consumers are themselves becoming producers of electricity.

Once tomorrow’s transmission and distribution system is enabled with better communication flows, it could very well become much more self-healing. Millions of new electricity grid-flow monitors would join end-users’ advanced meters in sending terabytes or petabytes of data to control systems for management and optimization. Utilities will likely need to develop entirely new business models in response to the influx of distributed generation sources. This is likely to be the new smart energy ecosystem, one that must become even more advanced and complicated than the smart grid envisioned today.


Emergence of a smart grid


A grid composed of new or improved grid-connected devices will enable the smart energy ecosystem to offer many new capabilities that respond to and drive changing consumer behaviour and attitudes toward energy.

For instance, the smart energy ecosystem will likely need to accept power from solar arrays on the rooftops of commercial buildings and private homes. It will also need to incorporate the varying but strong power coming from wind farms. When millions of individuals’ own PHEVs, a smart energy ecosystem will conceivably allow them to buy electricity from the grid in non-peak hours late at night. If the utility ever needs additional power during peaking events, it might draw on the stored power in the very same fleet.

Indeed, utilities are already deploying many devices that contain the microprocessors and two-way communication that can enable a wide variety of capabilities not possible before, including collection of more information, local decision-making and co-ordination.


Changing demands


The new smart energy economy will cause utilities and market participants to engage in a variety of new relationships and adopt business models that will evolve as the shape of the smart grid becomes more evident and opportunities present themselves.

In fact, we believe these new relationships and changing models will be among the more interesting outcomes as the smart energy ecosystem evolves. This new business environment will make it imperative to have a readily understood architecture in place to aid the capture of those opportunities as they arise.

The increasing diversity of energy resources will be one notable driver of new business models. For example, while wind, solar and other forms of distributed energy generation are becoming common and more cost-effective, they differ greatly from conventional plants in their operational, economic and control characteristics. Consider the operational complexity when a utility combines such variable generation sources with demand response, where the unused energy – sometimes referred to as a ‘negawatt’ – can be considered an energy resource if demand can be controlled.

This new and very diversified generation model of renewables, distributed generation and ‘negawattage’ transforms the electricity grid.

The grid’s operating model changes from one in which power flows one way from a small set of generation plants to one in which a two-way flow occurs with a large mixture of small, medium and large energy resources, many of which would have much more diverse operational characteristics. An extreme example of this dynamic is the use of PHEV batteries as storage for the grid.


New paradigms


The evolution of a smart energy ecosystem will require new computing paradigms. These include computing technologies, advances in storage, advances in communications technology, scale and participation of unreliable entities. For example, multiple cores in processors will become common-place. Applications will need to transition to multicore, multiprocessor, multithreaded design. Inexpensive, low-power, massively-parallel computing will dominate infrastructures and drive application design. Even while preserving existing investment through co-existence, application disaggregation will be necessary to capitalize on new hardware platforms.

In addition, wireless and hardwired communication capacities continue to expand, but the communication can be unreliable, either at certain times or in geographic locations commonly called ‘cell holes’.

Solutions will need to be flexible and resilient to momentary loss or interruptions of communication. As a result, autonomous operation will need to be a constant consideration.

The scale of interconnected smart energy systems will grow to new levels with the addition of the active participation of end-use customers, or loads, and a multitude of tiny new devices. Tight coupling of erratic autonomous participants will be proven unreliable, so systems will need to be designed to be flexible and adaptive to autonomous behaviour. The true measure of success will be building a working system out of small, autonomous, independent and unreliable devices and participants.

As a result, mastership cannot be guaranteed for some parts of the smart energy system. The system will require a design that expects the same computing problem to be addressed in multiple locations. For example, microgrids and integrated control centres may both calculate the energy balancing of a given distribution segment. In the case of microgrids, the solution can support effective operation of these grids in the event of loss of control centre communications. In the case of control centres, the solution can be co-ordinated between all neighbouring feeders.

Real-time energy management systems, whether at the transmission or distribution levels, will continue to have rigorous performance and reliability constraints.

Systems must be designed to be adaptive and resilient to autonomous, independent, potentially unexpected or non-responsive behaviour of the new participants – whether at scale, as in the case of end-use residential customers, or in bulk, as in large-scale renewable energy sources.




The incredible diversity of energy generation and delivery systems make it impossible to coherently offer a single, detailed view of one particular architectural framework that will work in every single instance.

Operation of a smart energy ecosystem will be shared between central and distributed generators. Control of distributed generators could be aggregated to form microgrids or ‘virtual’ power plants to facilitate their integration both in the physical system and in the market Source: European Technology Platforms Smartgrids

A smart energy reference architecture is needed to address prevailing smart energy systems and issues in enough detail to be useful, but without so much detail as to be untenable.

The National Institute of Standards and Technology (NIST) appears to be addressing the problem of too many standards – including international standards. The NIST is a global effort, despite being driven from the United States, that will accelerate the development and deployment of smart grid solutions worldwide in conjunction with the proposed reference architecture.

This smart energy reference architecture should be viewed as a bridge between NIST standards and specific technologies. It seeks to provide in one place a level of understanding about those technologies that exist in dozens of sources. The architecture requires five pillars: performance-oriented infrastructure, holistic life-user experience, energy network optimization, partner-enabling rich application platform, and interoperability.

A performance-oriented infrastructure includes those features necessary to make a reference architecture complete and appropriate to business needs. These include:

  • Economic features: The infrastructure must provide cost-effective means to deploy and integrate functionality
  • Deployment features: Components have to be flexible in how and where they can be deployed
  • Location-agnostic features: Services are designed so that they can be deployed on-premise or in the cloud
  • Always-connected features: Users and software components have access to platforms and services wherever they are located
  • Manageability features: Infrastructure components can be efficiently deployed, managed and monitored
  • Transferability features: Functionality and information can be migrated easily from one version of underlying infrastructure components to another with minimal interruption or intervention
  • Secure features: Deployed components, functionality and associated information are protected from unauthorized access or attacks
  • High performing and scalable features: Support for more users, larger models and increased transaction volumes can be accommodated through increasing hardware performance (scale-up) or the linear addition of hardware and network resources (scale-out)
  • Virtualization features: Components can be deployed in a manner that optimizes the use of hardware resources
  • Highly available and self-healing features: Support for transition to new equipment in the event of equipment failure
  • Disaster recovery and backup features: Capability to move to a new platform or facility or recovery from a natural disaster or terrorist event and the backup of results to facilitate the transition.


A holistic life-user experience enables all participants to view the smart energy ecosystem from the perspective of other participants. This equates to ensuring that the host company understands how customers experience the world and how technology fits in to that experience.

A technology architecture that facilitates the smart energy ecosystem will necessarily comprise:

  • A rich, integrated technology user experience for home, car, control centre and field workers
  • Browser-based collaboration using rich clients rendered appropriately across a multitude of devices
  • Supporting functionality for collaboration and mashups
  • A unified communications infrastructure in which the nature of the underlying communication infrastructures are transparent to users. The smart energy reference architecture permits an energy network to connect smart devices.


An optimized energy network incorporates:

  • Flexible communications: Deployments can leverage various communications paths and technologies and are easily reconfigured, minimizing the time required to make new information available to users
  • Smart connected devices: Intelligence is added to devices and they are connected to the communications network, enabling both intelligent autonomous operation and visibility of the network’s operation
  • Desktop, server, embedded and mobile operating systems: Operating systems can be effectively employed by leveraging the right type at the right level, for the right role, with the right performance
  • Application architecture: This is the architecture for applications infrastructure and services for commonly used capabilities so developers can focus on domain-specific functionality, optimizing speed to market and the reliability of solutions.


No single vendor will be able to provide all of the application functionality needed to implement the smart energy ecosystem.

The reference architecture seeks to offer a rich platform that makes it easy for partners to develop and deploy their applications. Notable aspects of the applications platform include services for:

  • Analytics: Rich statistical and analysis packages for data mining, discovery and reporting for diverse information consumers
  • Collaboration: Tools, services and applications enabling interaction between users and equipment
  • Complex event processing: Stream processing engines that can detect and filter events
  • Integration: Messaging and database technology for linking together workflow, processes and data optimization
  • Service bus: Services and components for communicating device and equipment data
  • Storage: Repositories for capturing and enabling the analysis of utility operational and business data
  • Workflow: Services for managing the automation of applications and business processes.


By providing these services to developers, the applications platform will enable them to concentrate on using their expertise to solve domain-specific problems. This would leave the platform to provide the common capabilities needed across many vertical domains. As a result, multiple vendors can provide competitive platform-consistent products and services, giving customers better offerings and more choices that are easy to leverage.

The smart energy reference architecture must enable interoperability so that the ecosystem can develop in a cost-effective manner. Otherwise, the vision for the ecosystem will be unrealistic and go unfulfilled. New solutions must work with previous utility technology systems in order to protect those investments. Pragmatic approaches to integration will need to be considered and the reference architecture should be flexible to allow new components to be deployed without customised integration. Interoperability considerations include:

  • Standards: A defined, consistent, industry-wide interface to allow new component deployment
  • Published interfaces: Transparently publicized for open industry use, even if a standard is not available, and also satisfying important interoperability needs
  • Information models: Consistent ontology for referring to equipment and assets to enable exchange of information throughout the enterprise and the value chain
  • User interfaces: Consistent content and behaviour in presentation of information and interaction with the user
  • Components: Well-defined sets of functionality packaged for developer and integrator reuse
  • Message formats: Key construct of service-oriented architecture, defining format and content, enabling services to exchange messages using the defined format, for example a publish–subscribe pattern
  • Interface definitions: All the elements of an interface so that applications can be independently developed to leverage the interface
  • Communication protocols: format, content and exchange mechanism so applications can be written to transfer information using the definition.
  • Security: definition of the security implementation including authentication, authorization, identity lifecycle management, certificates, claims and threat models – to enable secure interoperable design and deployment.



User experience


In addition to the reference architecture having a codified approach, the overall framework must identify several goals and characteristics. Interfaces must provide users with access to information and services appropriate for their organization and roles. Such a user experience will depend on the availability of secure, location-independent access to functionality.

The user interface should also allow for a composable front-end that provides consistency in how data are displayed, but does not lock an enterprise into using yet another standalone portal that does not integrate with one the enterprise already owns.

Beyond these basic requirements is a need for richness, efficiency, quality and consistency of the user experience that depends on information technology systems that enable visualization, analysis, business intelligence and reporting. New, advanced sensors will expand the capabilities of the smart energy ecosystem with increased integration with the Web. These include global positioning systems (GPS), phasor measurement units (PMUs), interval meter readings and centralized remedial action schemes (C-RASs).

For instance, by leveraging technologies such as GPS, it is now possible for devices to take measurements with a very precise view of time. This makes it possible to measure phase angles at locations on the grid using PMUs and to take grid-wide measurement snapshots. Interval meter readings will enable more accurate load models.

Together, these technologies provide new opportunities for improvements in network analysis, monitoring and control. This offers improvements in grid stability and security and facilitates better grid utilization. Other core components of the smart energy ecosystem technology architecture will be the Web technologies, integration standards and related products that now offer increased collaboration at many levels. These technologies provide opportunities for more pragmatic, lower-cost implementations and will overcome previous cost barriers to integration.

Advanced metering infrastructure (AMI) is yet another important enabler that some people often consider synonymous with smart grids. Its two-way communication capabilities mean AMI has created many new opportunities. For example, measuring usage can be more timely. This provides opportunities for new pricing options beyond billing that is based on total monthly consumption.

AMI allows automatic detection and confirmation of outages, with automatic verification of restoration. It helps detect customer-level issues about power quality, such as momentary outages and voltage levels. It also provides a gateway to home area networks, which let home devices react to pricing and load control signals as needed to implement demand-response programs, as is the case with those devices that use a standard called ZigBee.

AMI also allows the management of schedules for local energy consumption. Here the user can minimize costs based on their preferences and the utility can balance loads and achieve better utilization of the distribution networks. Communication networks should be considered a primary component in any architectural blueprint, because of the role they play as enablers of the smart energy ecosystem. The field networks now used to communicate with AMI devices are typically private, often using proprietary protocols or those specific to the utility industry.

Alternatively, broadband Internet services offer a communication infrastructure that is open and cost-effective with higher bandwidth. Broadband has the added benefit of already being widely deployed. The recent commitment by the Federal Communications Commission in the US to ‘net neutrality’ removes the biggest remaining broadband concern. As long as security is addressed upfront, metering and home area network communications infrastructures allow new families of devices to be added to the set of monitored and controllable devices on the grid. These include: smart thermostats; smart appliances; PHEVs, which can be in different states for charging, storage and discharging; ZigBee2 Smart Energy profile devices; HomePlug devices, which can distribute broadband or HDTV around the home on the power socket wires; IPSO devices, which can communicate using the Internet Protocol on private networks or over the Internet; residential solar and wind; and building automation.

A new generation of field and home devices that have the ability to make local decisions using two-way communication capabilities will allow customers to better monitor, control and schedule energy consumption, as well as respond to demand response events and pricing signals. Utilities or independent service providers could use these devices to extend their operational capabilities by facilitating registration of the devices in energy programmes that let the power provider adjust schedules to provide more efficient and balanced operation of distribution networks.




The transition of the power and utilities business to the new smart energy ecosystem may well be the most significant change to shape the industry since its inception.

New processes, such as end-use loads dynamically participating in the ecosystem in a meaningful way, and new data requirements will significantly change the landscape. For example, there is a 2880-fold increase in moving from a system that samples a customer billing once per month to one that takes samples at 15 minute intervals over a 30-day month.

Smart metering, electric cars, renewable generation, new communications, new business models and a host of new industry players will all shape the future. The outlook can be a daunting challenge for anyone in the power and utilities computing arena. The proposed smart energy reference architecture provides a preparatory map for navigating these challenges and establishing enough agility to overcome the challenges that are unforeseen today.


About the authors


Jon Arnold is managing director and Larry Cochrane is utilities industry technology strategist, Worldwide Power and Utilities, Microsoft Corporation.


More Powe Engineering International Issue Articles


Powe Engineering International Archives


View Power Generation Articles on