In restructured power systems, markets can be designed in a way to optimize operational efficiencies and give optimal investment signals. While the precise market structure and accompanying instruments will vary from one jurisdiction to the next, they generally fall into three market types: long, medium, and short-term markets. In addition, there are emerging markets on ancillary services and distributed energy resources.

There are generally two types of long-term markets: the first for capacity (not always in place) and the second for long-term contracts for the off-take of electricity at a predefined price.

Capacity markets are typically mechanisms where a system operator procures or imposes capacity requirements (in MW) to ensure system adequacy in the future. This is often done with longer lead-periods from three to four years allowing enough time for projects to be built. They are often implemented through capacity auctions, which can enable a level-playing field for generators, demand response, and interconnections if appropriate participation guidelines are in place. The capacity does not have to actually produce electricity – it only has to be available in case of need.

The second type of long-term markets, or long-term contracts for off-take of electricity, include power purchase agreements or feed-in tariffs. The contract duration can vary between 10 and 35 years for long-lived investments such as nuclear power plants. Such agreements can be bilateral contracts between a utility and an independent power producer, or even a large demand user and an independent producer. Very often, however, they involve government intervention aimed at promoting new investment, either via an obligation or a regulated price. These long-term contracts can also be the result of procurement mechanisms, such as auctions. Ideally these contracts should be financial rather than physical, thus providing with the option to optimize dispatch closer to real-time.

Medium-term markets allow price risk to be better managed by suppliers and consumers. In well-functioning markets, most energy should be traded before the short-term markets, from a few months in advance up to three or four years.

The medium-term market may be a formal, organised spot market with future and forward standard products, or it may be informal, with variable quantities traded bilaterally over the counter by traders or retailers. In liquid European markets, roughly 90% of energy is traded on these medium-term markets.

Short-term markets (day ahead and intraday) play a key role in mobilising the flexibility of the power system, and how they are designed has effects on the cost-effectiveness of integration of renewables. These markets allow all actors to re-optimize and re-balance their portfolio close to real-time with minimal VRE forecast errors. These markets are also essential for the integration of power systems over large market areas. Prices constitute the references against which other medium- and long-term prices are set, and they motivate participants both in the short and long run.

Ancillary markets, in particular balancing markets, are applied to remove residual system imbalances in a least-cost manner. It is key that in the normal market segment there are clear balance responsibilities to market actors. It is also key that the balancing market can tap into a wide set of flexibility options (including demand response, storage, aggregation, VRE flexibility) to minimize societal costs.

Distributed energy resources include not only distributed generation, such as roof-top PV or small-scale bioenergy, but also distributed battery storage, flexible loads and even local grid control.

Around the world, local energy markets are emerging as an alternative for system operators looking to avoid network constraints, and even for consumers to optimize their energy consumption through microgrids. This shift implies matching the pace of technology deployment with the provision of the appropriate regulatory and institutional frameworks to enable new innovative models of system operation.

As with many other sectors of the economy, digitalization is reshaping the way we operate and plan power systems. Generally, digitalization can be used to:

• Improve system operation
• Provide predictive maintenance and asset optimization
• Enable end-consumer solutions: metering, billing, settlements, and demand response

The emergence of a greater number of grid-connected devices and new power system needs is driving the need for the deployment of data exchange platforms. Generally, such platforms aim to reduce transaction costs and remove entry barriers for new actors.

In order to facilitate competitiveness in the power system, the design of these platforms should balance priorities of full neutrality, non-discriminatory access to data and privacy concerns. Here, pinning down the right level of access requires close coordination with industry stakeholders to determine who should access to what data at what level of detail.

To this end, having a clear governance framework is key not only to ensure the right coordination scheme between TSOs, DNOs, and market players, but also to ensure that the right equipment is installed.

Moreover, these platforms vary in their application. For example, data platforms for end-consumer solutions may only aim to optimize the process and costs of metering, billing and settlement (as in the Danish DataHub), while grid-level platforms may be used to increase visibility of the network and enable the deployment of additional flexibility resources (as envisioned in the recommendations of the UK’s Energy Data Taskforce).

The rapid emergence of new devices and business models is increasingly driving the need for device ID platforms and raises interoperability concerns, particularly when transactions take place across multiple grid levels or across power systems.

Large-scale integration of VRE generation in power systems comes with key challenges: variability and uncertainty. Power system operators have historically been able to cope with these challenges, but the complexity of the problem has been magnified by the entrance of new technologies in both supply and demand.

Variability refers to the swings in electricity generation or demand. Power systems have historically been operated to meet significant (yet predictable) swings in demand. However increasing penetration of VRE generation means that system operations are now determined by the net load, which is the system load minus the output from VRE generators.

In systems with very high shares of renewables this can lead to steep ramps when VRE rapidly changes their output due to sudden changes in weather conditions. The key driver for increased variability in the net-load is the mismatch between the system’s load profile and locally available VRE generation profile. In some cases, the electrification of new end-uses, such as transport or heating, may be seen as an opportunity to reduce net-load variability through the introduction of demand management programmes.

Uncertainty relates to the increased difficulty of reliably predicting VRE resource availability at specific points of the day. Historically, power systems managed this by scheduling significant amounts of spinning reserves to back up VRE generation, leading to high fuel and availability costs.

Experience from power systems across the world has shown that the impact of uncertainty can be reduced by improving market design with gate-closures closer to real-time and requiring the use of improved forecasting techniques. For example, Ontario’s IESO is currently implementing a market reform to improve system operations in the face of high VRE penetration. This includes the introduction of a day-ahead market to complement its existing real-time market. 

High shares of VRE can create operational challenges, particularly short-term flexibility related to power system stability on the sub-second timescale. System inertia, a property derived from synchronous generators, acts to mitigate the rate of change of frequency following a contingency event in the power system.

VRE do not directly contribute to providing inertia to the power system since they are connected to the grid via power electronic converter devices, hence the term “non-synchronous”. VRE generators do not have a direct, electro-mechanical coupling to the grid, which makes them different to traditional, synchronous generators. As VRE displaces thermal generation, system inertia will be reduced, causing the system to be less stable. Currently, a few countries/regions with relatively small power system, such as Ireland and South Australia, are in the phase where the inertia is becoming a key issue.

One of the reliability measures used to maintain sufficient synchronous inertia is the establishment of a maximum limit of system non-synchronous system penetration (SNSP), which indicates the maximum share of non-synchronous generation (i.e. VRE), at any instance, that does not pose a security risk to the power system.

A SNSP limit is system specific and has to be established based on thorough analysis and testing. Ireland and Northern Ireland managed to raise the SNSP limit from 50% in 2015 to 65% with a target of 75% by 2020.

An option to overcome this issue is the delivery of synthetic inertia (or referred to as inertia-based fast frequency response) by VRE sources. First experiences are covered in pilot trials, and clear technical requirements and compliance procedures are needed to take these up in grid codes.

An alternative option is to provide additional inertia by means of other assets such as synchronous condensers installed by the transmission operator. These can also help to mitigate other technical issues related to higher shares of VRE such as loss of system strength. The case of South Australia is a good example of this.

A complementary option is one based on smarter measurement of inertia to allow system operators to recognise the real-time capability of the system in dealing with possible contingency. Japan and Great Britain are front-runners in the introduction of real-time inertia monitoring systems to overcome challenges associated with high VRE penetration.