When it comes to the volatile world of trading power, utilities have an inherent advantage: knowledge. The discipline of producing and delivering power is part of a tradition that spans decades. Now that competition has overtaken the wholesale movement of power in the USA, and is on its way to determining retail delivery as well, utilities can leverage their best asset and put it to use for profit enhancement. Knowledge is a valuable commodity when it comes to electricity; the question is how to capitalize on that knowledge in the new competitive landscape.
New technology will enable organizations to use publicly available data and internal, proprietary knowledge and use both to their advantage by applying leading-edge econometric techniques. Sometimes referred to as “knowledge applications”, these modern tools are highly suited to the study of arbitrage opportunities that depend heavily on a wide diversity of historical and current data. Unlike traditional applications for financial trading, new tools are emerging that combine the extensive physical modelling of electricity production and consumption with statistical modelling of future behaviour.
Figure 1. Software architecture for full benefits of the ASP model
The benefit of such technology is that it can be offered on-line. Today, internal resources are challenged to match the pace of market change. With web-based outsourcing techniques, the benefits of packaged solutions and internal custom development can be combined – namely, rapid deployment of solutions with targeted, proprietary distinction.
Is it a forecast?
It is ironic that some of the most important knowledge within a utility is often trapped inside the organization. Take, for example, load forecasting. Because of the inelastic nature of electricity supply and the highly elastic nature of electricity demand, it stands to reason that utilities have focused on the challenge of load forecasting and have amassed a tremendous body of knowledge about seasonal variability in demand, the impact of weather, and so on.
In a competitive world, such internal knowledge is of limited benefit. This is because such knowledge is supported by a rich depth of internal data, and such data is not widely available. Suddenly, the highly detailed view within a utility is taken away when the utility looks outside its boundaries. Since other utilities view such internal data as a competitive advantage, it cannot be accessed. This results in a situation where a utility must make the best use of the data available, and piece together a view of the world that is coarse and limited, compared to an internal view that is fine-grained and complete.
Some of the techniques that utilities have employed for years are helpful in creating the best possible view of the world at large. But a level of inventiveness and resourcefulness is required, and the econometric challenges are significant. The end result is a view of the world that accounts for new levels of uncertainty, creating guideposts in the relatively uncharted territory of electricity as a true market commodity.
In some international markets, the available data may be quite abundant. But in the USA, a fierce competitive attitude has resulted in restricted available data, and that trend is getting worse. This means that the aggregation of data can be quite like the hunt for an elusive animal.
The world has recently witnessed the partial collapse of the wholesale market in California. One outcome of this collapse was the closure of the California Power Exchange, which had been responsible for publishing next-hour and day-ahead prices. This collapse has led to the temporary loss of significant data; data that was used to provide a clear picture of the world.
Some historical data is available, and some current data can be gathered. The challenge that many organizations face is not only finding and collecting the data, but filtering it, cleaning it, making it complete, integrating it, and making it suitable for modelling and research. Some vendors have arisen to take on this challenge. Where such efforts typically fall short is in the flexibility of modifying such data and adding to it. The solution required is that of outsourced data management.
Figure 2. Data required for modelling load, price, and price indicators
Figure 2 shows how data can be aggregated for the purpose of modelling load, price, and price indicators. Many data sources are public, meaning they are at least accessible. An external vendor is often a good choice for managing the collection and care of disparate public data. Some data sources are private or proprietary – available for a fee or strictly for use by the company that owns it.
Outsourced data management allows selective use of public and private data sources, with none of the overheads of traditional IT expenses and slowness, or the inflexibility of packaged data. This kind of outsourcing is one function of an Application Service Provider (ASP).
Central data warehouse
AcuPower from e-Acumen is one example of a service that aggregates a wide range of public data while allowing individual subscribers to edit the information and add internal data to it. By maintaining a central data warehouse and allowing secure, browser-based access to it, AcuPower can minimize deployment time, maintenance, upgrades and data management. Data can be edited and secure, selective use of internal data or other paid-for data services is allowed at the same time.
For example, supply data is extremely challenging to gather and maintain. In addition to the vast amount of data related to generating units and their operating characteristics, current information on outages is difficult to come by. Through an arrangement with Industrial Information Resources (IIR), AcuPower can offer daily outage updates to subscribers who have also subscribed to IIR’s outage reporting service. This data is entered into the AcuPower data warehouse and made available to the full palette of modelling techniques within AcuPower.
Weather forecasts are a key input variable for load and price forecasts. AcuPower collects the National Weather Service forecast every morning, and also provides an option for a more accurate forecast called KiloWeather. KiloWeather is a consensus of numerous weather forecasts that is proven to be more accurate than the National Weather Service and most of the component forecasts, most of the time. This gives traders the advantage of knowing the results of NWS-based forecasting and more accurate results at the same time.
Once the challenge of data aggregation is addressed, the key to competitive advantage lies in the modelling techniques and application structure used to mine the data for insights on trading. A more unusual use of the ASP framework is the delivery of a complete application solution in an outsourced manner. At this level, the benefits of outsourcing begin to multiply.
First and foremost, the econometric techniques evolving at the forefront of the new market for power are rarely found inside a trading organization. Some of these techniques are fresh from the halls of academia and industry think tanks. The best have evolved from a blend of research and real-world challenges, taking the enduring qualities of past techniques and adapting them with new insights to the unique challenges of power. A good example is the simulation of electricity spot prices, now famous for the “long tails” of their probability distributions. Based on academic as well as industry credentials, the team at e-Acumen has innovated new techniques for modelling electricity spot prices accurately. E-Acumen believe that an ASP is an ideal vehicle for delivering such innovation.
This is in part due to a related benefit of the ASP model. If the software has been properly designed, applications can be upgraded and enhanced frequently and quickly, with no break in productivity for the customer. Remember, customers access a single version of the software in a central location. So the latest econometric techniques can be brought to market flexibly and quickly. One concept core to the notion of trading is that no single technique is best in all cases. Different market situations require different solutions, and two benefits of the ASP model provide great comfort to the trader: the techniques at hand are the best available, and that any technique can be modified by individual insight and expertise.
AcuPower has remained true to these benefits by employing a modular architecture with a blend of different modelling techniques, and by allowing full customization of the data and application. This is the tradition of advanced web sites offering database-backed personalization.
E-Acumen has already noted that an ASP allows a seamless blend of public and private data. The next step is allowing complete modification of modelling assumptions and data, creation of new group defaults, alteration of any default, and saving of unique scenarios and results based on any of these changes. The result is a highly flexible application framework that can generate “default” results but also allows customization on demand. For most traders, there is no single “answer”. Instead, there is a range of possible answers with different confidence levels, based on assumptions moulded by market reality and individual insights.
Most energy trading floors realize that traders need the best possible support. So they stock up on information services, tools, and analysts. The analyst is a key driver of competitive advantage, since the digestion and use of diverse information and tools require time and expertise. AcuPower is designed to help the analyst derive useful information as quickly as possible, and provide a platform that can be further customized by traders.
Figure 3. Workflow on the trading floor using multi-user decision support system
One modelling technique in AcuPower involves the simulation of production cost using a fundamental approach. For such an approach to be effective, supply must be moulded in detail, and load forecasting must be as precise as possible. The process typically starts with adjustment of the defaults provided in AcuPower. On the data side, this tends to involve the data and parameters that govern supply stacks. A generating unit has numerous operating characteristics, some of which come from current market data. If an analyst has access to IIR’s outage data, outages will appear within the screen for outage adjustments. But additional outages often crop up during the day, and the outage picture is never complete.
Similarly, AcuPower’s estimation of fuel prices can be modified according to internal forward curves for gas prices, for example. This can be done manually, one area at a time, or all prices can be uploaded according to a pre-defined format for automatic inclusion. Alternatively, the trading floor can subscribe to pricing services that can be “turned on” within AcuPower.
For load forecasting, two modelling techniques account for the diversity of available data. At a Utility Planning Area level, where data is the scarcest, extensive pre-modelling with historical data produces accurate results with current weather forecasts. Load growth and temperature values can be adjusted based on the insights of the analyst or trader. For areas with current data, a dynamic forecasting engine generates fresh regressions on a regular basis throughout the day, providing the most accurate load forecast possible. Temperature sensitivities can be applied, and current actual temperatures and loads have a direct bearing on the forecast results. Scenarios can be created with alternate weather sources and different temperature adjustments. Price forecasting utilizes the same dynamic engine and similar data inputs for the best possible estimation of prices.
Once adjustments are made at a general level, the analyst can save the new defaults as the trading floor’s defaults. This can follow a process of generating numerous load/supply overlays, and comparing outcomes like marginal unit cost and price distributions to market expectations. In some cases, the overlay results are delivered directly to traders. In other cases, the traders can use AcuPower with the new group defaults and perform further scenario analysis and examination of price distributions and trends. The analyst has effectively transferred a wealth of market inputs and expertise to the trading floor, with complete transparency of assumptions and flexibility for individual variations and insights.
The trading floor is a hectic place, and power traders require extensive knowledge in order to make money. This is one example of how a knowledge application can increase the knowledge of a trader, and provide a base for the organization to improve its profit potential.
Knowledge is the most valuable commodity when it comes to trading power. Traders armed with the best data and tools can mean the difference between significant losses and significant gains. While utilities have a wealth of knowledge within their organizations, harnessing that knowledge in a volatile market requires new technology and new approaches.
Outsourcing data management and application development via an ASP is an emerging solution, even for large organizations with significant internal resources. A properly designed ASP can leverage internal, proprietary data and knowledge while providing the best possible modelling techniques and external data. Speed is of paramount importance in reacting to rapid market changes, and the proper blend of technology, data and expertise can accelerate the use of information in enhancing the bottom line.