📕 Make your first submission with ease by following starter kits.

        Control Track Starter Kit

        Forecasting Track Starter Kit 

📲 Join the conversation with other participants on CityLearn Discord Server.

👥 Challenge is more fun with friends, find your teammates.

💬 Got a query or suggest? Share your feedback.

🕵️ Introduction

Buildings are responsible for 30% of greenhouse gas emissions. At the same time buildings are taking a more active role in the power system by providing benefits to the electrical grid. As such, buildings are an unexplored opportunity to address climate change.

Energy storage systems that store electricity e.g., batteries or thermal energy such as domestic hot water systems can reduce peak loads on the grid by shifting loads to different times. Likewise, the use of heat pumps to satisfy space cooling and heating loads is an added opportunity for flexibility as they can be controlled to operate within the margins of occupant comfort while drawing minimal energy from the grid. Solar photovoltaic (PV) generation can reduce the overall demand on the grid by providing buildings with self-sufficiency during the day as well as an opportunities to store free and clean energy in the energy storage systems for times when there is no sunlight.

However, these distributed energy resources need to be carefully managed simultaneously in many buildings to unlock their full energy efficiency potential and reduce the cost to homeowners without violating comfort constraints. Rule-based control (RBC) is a popular control solution for building energy management. It makes use of simple if-else statements in its decision making process e.g., "if outdoor dry-bulb temperature is less than 20C and hour is 10 PM, charge battery by 5% of its capacity". However, RBC is not able to generalize to new buildings as it typically needs to be manually tuned to cater to building-specific characteristics. Also, in a multi-control task environment, RBC is unable to adapt and adjust to the interactions amongst the different tasks.

On the other hand, advanced control systems such as model predictive control (MPC) and reinforcement learning control (RLC) are well-suited for automated building energy management in a multi-task environment, while adapting to individual characteristics of occupants and buildings. Comparatively, RBC and RLC are inexpensive to implement as they have a lower entry bar for domain knowledge of the controlled systems. However this inexpensiveness leads to RBC and vanilla model-free RLC performing sub-optimally compared to MPC as they are not modeled after the system under control though, model-based RLC includes a model of the controlled system. Nevertheless, model-free RLC is a data-driven solution and as more training data become available, it implicitly learns the model of the building or system under control thus, achieves comparable performance as MPC as model-based RLC. One of the strengths of RLC is its ability to adapt to disturbances in the building it controls as the thermal or occupant dynamics change. Nevertheless, combinations of these three control solutions have been found to provide remarkable results.

The performance quality of RLC is dependent on the quality of forecasted system states such as hourly to day ahead forecasts of end-use loads, net electricity consumption, carbon emissions and solar generation that inform the controllers' decision making. Thus, this makes building energy management both a forecast problem as much as a control problem.

Also, the application of distributed energy resources and advanced control in the face of power outages to provide grid-resilience remains a developing research area where these control solutions must learn to operate during periods of curtailed supply or extreme cases of blackouts. Thus, the value of advanced control and their adoption to real world buildings hinges on their ability to demonstrate grid-resiliency.

The CityLearn Challenge makes use of the CityLearn Gym environment as an opportunity to compete in investigating the potential of artificial intelligence (AI) and distributed control systems to tackle multiple problems within the built-environment domain. It is designed to attract a multidisciplinary participation audience including researchers, industry experts, sustainability enthusiasts and AI hobbyists as a means of crowd-sourcing solutions to these problems.

The CityLearn Challenge 2023 addresses this multi-faceted nature of advanced control in buildings by blending the challenges of control algorithm design, forecast quality and grid-resilience. The CityLearn Challenge 2023 presents a control track as done in previous challenges as well as introduces an independent forecast track where, both tracks are run in parallel and utilize the same dataset.

In the control track, participants will develop energy management agent(s) and an optional custom reward function (in RLC solutions) to manage electrical and domestic hot water energy storage systems, and heat pump power in a synthetic single-family neighborhood under normal grid-operation and power outages. Whereas, in the forecast problem, participants will design regression models to predict the 48-hour-ahead end-use load profiles for each building in the neighborhood as well as the neighborhood-level 48-hour-ahead solar generation and carbon intensity profiles.

📑 Problem Statement

The CityLearn Challenge 2023 is a two-track challenge where either track is independent of, but may inform design choices in the other track. Both tracks make use of the same dataset of a synthetic single-family neighborhood and are run in parallel. The track-specific problem statements are described below:

Forecast Track

In the forecast track, participants are to develop regression prediction models for forecasting building loads, grid carbon intensity and solar generation. The forecasts should be provided at each time step for the following 48 time steps. The variables to be predicted are:  

 Building-Level

1. Cooling Load (kWh)
2. DHW Load (kWh)
3. Equipment Electric Power (kWh) (a.k.a. non-shiftable load)

Where each variable is forecasted for each building.

Neighborhood-Level

1. Carbon Intensity (kgCO2e/kWh)
2. Solar Generation (W/kW)

Where each variable is forecasted for the entire neighborhood. Unlike the Control Track, this forecast track does not include power outages in the environment.

Control Track

In the control track, participants are to develop their own single-agent or multi-agent reinforcement learning control (RLC) policy and optional custom reward function __OR__ a model predictive control (MPC) policy for electrical (battery) and domestic hot water storage systems, and heat pump control in the buildings with the goal of maintaining thermal comfort, reducing carbon emissions, increasing energy efficiency and providing resiliency in the event of power outages. 

A single-agent setup will mean one policy is used to control all building resources whereas, multi-agent setup will mean each building's set of resources is controlled using a unique policy that may cooperate or compete with other policies.

In Phase II, the environment will be updated to include stochastic power outages based on the Reliability Metrics of U.S. Distribution System where the control agent must adequately manage the available distributed energy resources to maintain comfort and energy demand.

Online evaluation during the course of Phase II will be reflective of the average scores from using three different private random seeds to generate stochastic power outage signals in the environment. These three seeds will be kept constant throughout Phase II. At the end of Phase II, three new private seeds will be used to select winners on a private leaderboard.

💾 Dataset

The challenge makes use of the open-source End-Use Load Profiles for the U.S. Building Stock dataset to generate a six-building single-family neighborhood in an undisclosed U.S. location. Each building is equipped with a heat pump to meet space cooling loads, an electric heater to meet domestic hot water (DHW) heating loads, a DHW energy storage system to shift DHW heating loads, a battery for electricity storage and photovoltaic (PV) system for self-generation. See the table below for a summary of the building metadata:

Building	Availability	Geometry	Vintage	Area (ft2)	Heat pump (kW)	Heater (kW)	DHW storage (kWh)	Battery (kWh)	PV (kW)
1	Public (Phase I & II)		1980s	1690	4.1	4.9	2.3	4.0	2.4
2	Public (Phase I & II)		1980s	1690	2.3	3.7	1.7	4.0	1.2
3	Public (Phase I & II)		1970s	1690	2.8	6.3	2.8	3.3	2.4
4	Private (Phase II)		?	?	?	?	?	?	?
5	Private (Phase II)		?	?	?	?	?	?	?
6	Private (Phase II)		?	?	?	?	?	?	?

This six-building dataset is split into Phase I (warm-up) and Phase II (evaluation) sets where each set is based on simulation data from a unique calendar year. During the course of the competition, participants will be provided with the dataset of 3/6 buildings to familiarize with the problem, train their agent(s) on and make submissions. This is the Phase I, a.k.a. warm-up, dataset. The warm-up dataset covers a one-month period and is automatically downloaded when the starter kit is forked. The leaderboard during Phase I will also reflect an evaluation that uses this 3/6 building set.

Subsequently, at the beginning of Phase II, the same 3/6 buildings but in a different 3-month period calendar year from Phase I are used for online evaluation. This updated dataset is kept hidden from participants but they will be able to see how their submissions perform with this hidden dataset via the Phase II public leaderboard.

At the end of Phase II, submissions will be halted and the complete 6/6 building data set for the 3-month period calendar year is used for evaluation. Evaluation on this dataset is kept in a private leaderboard that is visible to only the challenge organizers.

Hence, submissions in Phase II must:

1. Generalize to unknown occupant behaviors and building thermal dynamics that energy use.
2. Be designed such that they an be transferred to more buildings than trained on.

See this post on the discussion board for more information on changes in Phase II.

The dataset contains the following files:

1. schema.json: JSON model of the CityLearn environment that is loaded at runtime. Participants are encouraged to manipulate this file to assess the impact of parameters on the environment, policy, and reward function design. However, only the as-provided version of schema.json will be used for evaluation.
2. weather.csv: Actual meteorological weather data for the buildings' location used to provide observation values to the environment.
3. carbon_intensity.csv: Actual emission rate from grid mix (kgCO2e/kWh).
4. pricing.csv: Time-of-Use (TOU) electricity rate ($/kWh).
5. Building_1...n.csv: Each building's time series data including, temporal (calendar) variables, end-use demand, solar generation and indoor environment variables.
6. Building_1...n.pth: Each building's temperature dynamics LSTM model parameters.

Refer to the dataset and schema files.

📅 Timeline

Phase I (Warm Up Round, Aug 21 - Sep 18, 2023)

This phase provides participants with an opportunity to familiarize themselves with the competition, CityLearn environment and raise issues bordering on the problem statement, source code, dataset quality and documentation to be addressed by the organizers.A solution example will also be provided so that participants can test the submission process and see their submissions show up on the leaderboard. The submissions and leaderboard in this phase are not taken into account during Phase II and selection of winners in Phase III.

Phase II (Evaluation Round, Sep 19 - NOV 15, 2023)

This is the competition round. Participants will be able to see how each of their submissions rank against each other and how their latest submission ranks against other participants’ submissions in a public leaderboard. There are also changes made to the environment and online evaluation dataset in this phase dataset and problem statement. At the end of this phase, new submissions will be halted and existing submissions are evaluated against another different dataset but the scores and rankings are kept private and visible to only the challenge organizers.

Phase III (Review Round, Nov 16 - Nov 30, 2023)

During this phase, winners will be selected and announced. Also, the organizers will develop an executive summary of the competition that documents the preparation, winning solutions, challenges faced and lessons learned.

🏆 Prizes

This challenge has Leaderboard Prizes and Co-authorship Prizes

Leaderboard Prizes

Top three teams or participants on the Phase II leaderboard of both tracks will receive the following prizes:

Control Track

• 🥇 1st Prize: 1000 USD
• 🥈 2nd Prize: 800 USD
• 🥉 3rd Prize: 700 USD

Forecast Track

• 🥇 1st Prize: 1000 USD
• 🥈 2nd Prize: 800 USD
• 🥉 3rd Prize: 700 USD

Co-Authorship

In addition to the cash prizes, the top three teams or participants from both tracks will be invited to co-author a summary manuscript at the end of the competition. At the organizer's discretion, honorable mentions may be included for academically interesting approaches, such as those using exceptionally little computing or minimal domain knowledge. Honorable mentions will be invited to contribute a shorter section to the paper and have their names included inline.

Further Optional Reading

The following publications, articles webpages and tutorials are recommended but not required reads to get started with the challenge:

• Grid-Interactive Efficient Buildings

  • Grid-interactive Efficient Buildings Technical Report Series

• Building Control

  • All you need to know about model predictive control for buildings
  • Ten questions concerning reinforcement learning for building energy management

• Building Load Forecasting

  • Smart Meter Data Analytics: Practical Use-Cases and Best Practices of Machine Learning Applications for Energy Data in the Residential Sector
  • CCAI Summer School Tutorial: Building Load Forecasting with ML

• Grid Resiliency

  • Energy Resilience

• Dataset

  • End-Use Load Profiles for the U.S. Building Stock: Methodology and Results of Model Calibration, Validation, and Uncertainty Quantification

• CityLearn

  • CityLearn documentation
  • CityLearn: A Tutorial on Reinforcement Learning Control for Grid-Interactive Efficient Buildings and Communities

• Previous The CityLearn Challenges

  • The CityLearn Challenge 2022
    • Winning solutions
      • 1st Position Solution
      • 2nd Position Solution
      • 3rd Position Solution
  • The CityLearn Challenge 2021
  • The CityLearn Challenge 2020

📱 Contact

📲 Join the conversation with other participants on CityLearn Discord Server.

👥 Challenge is more fun with friends, find your teammates.

💬 Got a query or suggest? Share your feedback.

🤝 Sponsors

CCAI	MERL	Amazon
🌎 Hosting	🏆 Prize	🖥️ Compute credits