Selection of a representative subset of global climate models that captures the profile of regional changes for integrated climate impacts assessment
© The Author(s). 2017
Received: 15 August 2016
Accepted: 5 February 2017
Published: 4 March 2017
We present the Representative Temperature and Precipitation (T&P) GCM Subsetting Approach developed within the Agricultural Model Intercomparison and Improvement Project (AgMIP) to select a practical subset of global climate models (GCMs) for regional integrated assessment of climate impacts when resource limitations do not permit the full ensemble of GCMs to be evaluated given the need to also focus on impacts sector and economics models. Subsetting inherently leads to a loss of information but can free up resources to explore important uncertainties in the integrated assessment that would otherwise be prohibitive. The Representative T&P GCM Subsetting Approach identifies five individual GCMs that capture a profile of the full ensemble of temperature and precipitation change within the growing season while maintaining information about the probability that basic classes of climate changes (relatively cool/wet, cool/dry, middle, hot/wet, and hot/dry) are projected in the full GCM ensemble. We demonstrate the selection methodology for maize impacts in Ames, Iowa, and discuss limitations and situations when additional information may be required to select representative GCMs. We then classify 29 GCMs over all land areas to identify regions and seasons with characteristic diagonal skewness related to surface moisture as well as extreme skewness connected to snow-albedo feedbacks and GCM uncertainty. Finally, we employ this basic approach to recognize that GCM projections demonstrate coherence across space, time, and greenhouse gas concentration pathway. The Representative T&P GCM Subsetting Approach provides a quantitative basis for the determination of useful GCM subsets, provides a practical and coherent approach where previous assessments selected solely on availability of scenarios, and may be extended for application to a range of scales and sectoral impacts.
KeywordsClimate change GCMs Climate model subset Representative T&P GCM subset approach Climate impacts AgMIP Integrated assessment
Societal applications of climate change information are driven by the needs of stakeholders deciding upon particular adaptation, mitigation, policy, or risk management strategies. Model-based projections of climate changes and related uncertainties form a core component of these climate impact applications, however uncertainty may also be introduced by sector models (e.g., crop, livestock, health, ecosystem, fisheries, energy, or water resource models), biophysical or socioeconomic scenarios (e.g., greenhouse gas concentrations; regional development pathways), and the economic models upon which many decisions are evaluated. Connecting climate models, sector impact models, and economics models results in an integrated assessment framework capable of exploring the societal ramifications of climate impacts as well as opportunities to build resilience through interventions under a risk management framework. The Agricultural Model Intercomparison and Improvement Project (AgMIP) was developed to facilitate best practices and collaborative development of agricultural models with the aim of informing stakeholder decisions across a variety of regions, scales, models, and scenarios (Rosenzweig et al. 2013, 2015, 2016). One goal of AgMIP is to provide guidance on the creation of climate change scenarios for agricultural experts unfamiliar with the climate modeling community.
Impact assessments across sectors have too often been influenced by selection bias, leading to inconsistencies across studies and confusion among policymakers. White et al. (2011) revealed large differences across crop modeling studies in the number and types of global climate models (GCMs) used, with many using a small number of models and GCM selection often influenced by the availability of output. The first phase of the Inter-Sectoral Impacts Model Intercomparison Project (ISI-MIP; Warszawski et al. 2014) utilized a common set of 5 GCMs for all sectoral impacts assessments owing to the need for consistency across regions and sectors. The ramifications of choosing this subset, based in part upon which GCMs available at the time, continues to be explored (McSweeney and Jones 2016). Early AgMIP regional integrated assessments also used 5 GCMs selected according to prominence in publications, length of participation in CMIP, spatial resolution, and historical monsoon patterns (Ruane et al. 2015b).
A common limitation in AgMIP and related studies stems from the overwhelming number of possible combinations of individual elements within an integrated assessment framework, leading to a prohibitive number of simulations. For example, ongoing AgMIP work across sub-Saharan Africa and South Asia seeks to utilize information from a set of 29 GCMs, 3 time periods, 2 greenhouse gas concentration pathways, 2 climate scenario generation methodologies, 3 adaptation scenarios, 2 agricultural development pathways, at least 2 crop models, and 40 or more households representing the distribution of impacts for a given region. Together, this would be an impractical number of simulations to conduct and analyze (at least 29x3x2x2x3x2x2x40 = 167,040) for each crop species examined, so representative subsets are required wherever possible.
Subsetting GCMs necessarily leads to a loss of information, which begs the question: what types of information are most important to maintain? In the end this is a subjective decision that is best decided in discussion between researchers and stakeholders, however it must be justified (Knutti et al. 2010a) and physical and statistical relationships provide useful guidance.
The most prominent information for assessing sectoral climate impacts in a given region remain projected temperature change and precipitation change (the concentration of greenhouse gases such as carbon dioxide can be taken directly from driving scenarios). These quantities are indicative of large-scale changes in energy and water cycles in a region, and many other climate variables are closely correlated with these quantities. Even an increasing emphasis on extreme events requires the context of long-term shifts in mean temperature and precipitation, which also form the basis of intuitive climate scenario generation methodologies such as the “delta approach” wherein mean monthly temperature and precipitation changes are imposed upon historical observations (Wilby et al. 2004). The delta approach was the most common climate scenario generation methodology used in the White et al. (2011) review of agricultural impact models. Many impact sector models also respond strongly to mean temperature and rainfall shifts, allowing for the development of simple but effective emulators (e.g., Ruane et al. 2014; McDermid et al. 2015a; Howden and Crimp 2005).
Characteristics of the overall distribution of projections from the full GCM ensemble are priority information to maintain, as multi-model ensembles generally outperform individual models when validated across many observations (though not necessarily for any single observation). This has been illustrated in many studies within the climate modeling community (Reichler and Kim 2008; IPCC 2010; Flato et al. 2013) and for impacts models in various sectors including agriculture (Asseng et al. 2013; Bassu et al. 2014; Martre et al. 2015; Li et al. 2015). The overall spread and skewness of projections is also a worthwhile indication of model-based uncertainty, confidence, and the range of plausible outcomes.
To reduce the number of GCMs required to sample climate change uncertainty and thereby free up resources for other elements of a regional integrated assessment,
To focus the GCM selection and assessment process on the season of interest,
To ensure that the assessment framework exposes the system to major classes of change without averaging to an extent that temporal and spatial patterns are not physical,
To avoid the selection of extreme outliers that may skew results, and
To maintain information about overall uncertainty in GCM projections.
The Representative T&P GCM Subsetting Approach builds upon the work of Semenov and Stratonovich (2015), who suggested the use of “climate sensitivity indices” to subset GCMs based upon annual-mean temperature and precipitation changes in aggregated regions around the world. This methodology was developed independently from a similar approach recently described by Lutz et al. (2016) that focuses on describing the limits of an envelope of potential temperature and precipitation changes and the largest shifts in extremes. As described below, we add additional guidance on classes of regional climate change, the maintenance of uncertainty information from the broader ensemble, sector specific seasonal focus, a connection to anticipated scenario generation methods, and coherence across space, time, and greenhouse gas scenario. We also provide a regional demonstration, show how this framework may be applied at global scales, and explore how the perspective of GCM subset priorities enables statistical and physical analysis of the ensemble of GCM projections.
It is important to emphasize that selecting a subset of GCMs to conserve computational and analytic resources is distinct from efforts to develop unequal weights for GCMs in order to better capture the signal of future climate change (e.g., Giorgi and Mearns 2003; Tebaldi et al. 2005; see reviews in Knutti 2010; IPCC 2010; Flato et al. 2013). The development of unequal weights is particularly appealing given that CMIP represents an ensemble of opportunity rather than an ensemble designed to systematically capture uncertainty around a true projection. Weighting approaches allow models with substantial biases to be lowered in emphasis, although biases in the historical period do not necessarily reflect biases in climate response to anthropogenic forcings and it can be difficult to determine which metrics should form the basis of weighting. Knutti et al. (2010b) called for the use of multi-model ensembles but indicate that equal weighting is preferred until weighting methods can be more fully developed and validated.
Materials and methods
The Representative T&P GCM Subsetting Appraoch is built upon climate change projections provided by state-of-the-art climate and earth system models such as those submitted to the Coupled Model Intercomparison Project (CMIP; Taylor et al. 2012; Eyring et al. 2016) in support of the Intergovernmental Panel on Climate Change (IPCC). These projections are then combined with a gridded historical climate dataset in order to overcome common biases in GCM precipitation climatology. Meteorological observations for the Ames, Iowa, example utilized below were drawn from the Iowa Environmental Mesonet maintained by Iowa State University.
Future climate projections
Summary of 29 CMIP5 GCMs that form the ensemble of climate projections used in this study
2x [CO2] Eq. climate Sens. (°C)
Commonwealth Scientific and Industrial Research Organization (CSIRO) and Bureau of Meteorology (BOM), Australia
1.25° × 1.875°
Beijing Climate Center, China Meteorological Administration
~2.8° × 2.8°
College of Global Change and Earth Systems Science, Beijing Normal University (BNU)
~2.8° × 2.8°
Canadian Centre for Climate Modelling & Analysis
~2.8° × 2.8°
US National Center for Atmospheric Research (NCAR)
~0.9° × 1.25°
US National Science Foundation (NSF), US Department of Energy (DOE), and the US National Centre for Atmospheric Research (NCAR)
~0.9° × 1.25°
Euro-Mediterranean Center on Climate Change
~0.75° × 0.75°
Euro-Mediterranean Center on Climate Change
~1.9° × 1.875°
France National Centre for Meteorological Research
~1.4° × 1.4°
Queensland Climate Change Centre of Excellence and Commonwealth Scientific and Industrial Research Organization (CSIRO)
~1.9° × 1.875°
Chinese Academy of Sciences
~2.8° × 2.8°
NOAA/Geophysical Fluid Dynamic Laboratory (GFDL)
2.0° × 2.5°
NOAA/Geophysical Fluid Dynamic Laboratory (GFDL)
~2.0° × 2.5°
NOAA/Geophysical Fluid Dynamic Laboratory (GFDL)
~2.0° × 2.5°
National Aeronautics and Space Association Goddard Institute for Space Studies (NASA GISS)
2° × 2.5°
National Aeronautics and Space Association Goddard Institute for Space Studies (NASA GISS)
2° × 2.5°
UK Meteorological Office - Hadley Centre
1.25° × 1.875°
UK Meteorological Office - Hadley Centre
1.25° × 1.875°
UK Meteorological Office - Hadley Centre
1.25° × 1.875°
Russian Institute for Numerical Mathematics (INM)
1.5° × 2°
Institute Pierre Simon Laplace (IPSL)
~1.9° × 3.75°
Institute Pierre Simon Laplace (IPSL)
~1.3° × 2.5°
Institute Pierre Simon Laplace (IPSL)
~1.9° × 3.75°
University of Tokyo, Japanese National Institute for Environmental Studies (NIES), and Japan Agency for Marine-Earth Science and Technology (JAMSTEC)
~1.4° × ~1.4°
University of Tokyo, Japanese National Institute for Environmental Studies (NIES), and Japan Agency for Marine-Earth Science and Technology (JAMSTEC)
~2.8° × ~2.8°
Max Planck Institute (MPI) for Meteorology (low resolution)
~1.9° × 1.875°
Max Planck Institute (MPI) for Meteorology (mixed resolution)
~1.9° × 1.875°
Japanese Meteorological Research Institute (MRI)
~1.1° × 1.125°
Norwegian Climate Centre
~1.9° × 2.5°
GCM projections of temperature change are calculated as absolute differences (future-current) while projections of future precipitation are calculated as percentages of the current period (future/current * 100%). When averaging GCM projections of seasonal precipitation changes over multiple months, historical climate information over the “current” period (1980–2010) is needed to recognize that percentage changes in wet months impact total season precipitation more than changes in dry months. To demonstrate similar characteristics of GCM projections, we create simple climate scenarios by imposing the temperature changes (an additive factor) and precipitation changes (a multiplicative factor) upon historical observations as is done in the “Delta” approach among many other statistical methods (Wilby et al. 2004; Ruane et al. 2015b). Future time periods are defined as Near-term (2010–2039), Mid-Century (2040–2069), and “End-of-Century” (2070–2099) as in Ruane et al. (2015b).
Historical climate data
Local observations of meteorological variables are the ideal source of historical climate information, however these are missing or incomplete in many regions and gridded applications require spatial consistency. This study therefore uses AgMERRA (Ruane et al. 2015a), a hybrid of the Modern-Era Retrospective-analysis for Research and Applications (MERRA; Rienecker et al. 2011) and various gridded and satellite products in order to fill in gaps and remove common biases needed for global agricultural modeling. AgMERRA compares favorably with similar climate forcing datasets and reanalyses, although substantial uncertainty remains in regions with limited observational coverage (much of the tropics and many developing countries; Ruane et al. 2015a). AgMERRA provides a global, daily, 0.25° × 0.25° gridded climate dataset spanning 1980–2010 containing maximum and minimum temperatures, precipitation, solar radiation, wind speed, and relative humidity at the maximum temperature time of day. AgMERRA has been used in numerous AgMIP activities (Elliott et al. 2015; Ruane et al. 2015b).
The representative T&P GCM subsetting approach
To demonstrate the Representative T&P GCM Subsetting Approach, we consider an integrated assessment using crop and economic models to assess the impacts of climate change on maize production systems in Ames, Iowa, USA (93°45′W, 42°1′N). The framework of this hypothetical study includes the use of multiple climate and crop models, as well as a range of climate, agricultural development, and adaptation scenarios utilizing a regional economic model. This example is typical of many applications within AgMIP and similar projects which aim to provide stakeholders with information about potential challenges and vulnerabilities in order to aid in investment and risk management decisions (Antle et al. 2015).
Defining growing seasons
As crop model simulations that drive this analysis consider only the maize growing season, GCMs must also be selected based upon their projections of the relevant months. This seasonal focus highlights the differences between GCMs that would affect the overall outcome of the integrated assessment while avoiding the influence of potential biases in months that are never simulated. Here, we utilize local growing season as defined in the harmonized simulations of AgMIP’s Global Gridded Crop Model Intercomparison (GGCMI; Elliott et al. 2015), which derived planting windows from Sacks et al. (2010) and Portmann et al. (2008, 2010). We round planting and harvesting dates to whole months, with months included only if crops were in the ground for the majority of days (this would prevent over-representation of a month if planting occurred on its last day, for example). For Ames, Iowa, the average maize season corresponds to planting on May 13th and harvesting on September 27th, so GCMs will be selected based upon their projections of the May–September period (MJJAS). In other cases it may be desirable to utilize only a subset of the growing season, particularly if anomalies in a given month are shown to have a particularly strong impact on crop development. As maize is often grown in the summertime for mid-latitude farms and in rainy seasons within the tropics, maize season results presented in the Results section are consistent in representing the challenges faced by maize farmers despite including results from different months in different locations on the same map. Soil moisture in crop models is often initialized shortly before planting, so we do not include preceding months.
Integrated assessments that link together the economics of multiple growing seasons in a given farming system often benefit from the use of a consistent set of GCMs for all crops. In these situations it is likely desirable to analyze a growing season that covers all months when crops are in the ground, as detailed in the Discussion section below.
Delineating climate change quadrants
To identify fundamental classes of projected climate change for a region during a specific time period and RCP, we characterize an individual model’s projected, location-specific temperature and precipitation changes in terms of its deviation from the ensemble median. A GCM’s projections will therefore be relatively cool or hot and relatively wet or dry. In this demonstration we use the climate information from the GCM output’s native grid boxes, although a similar procedure could subset an ensemble of downscaled products and may be advisable where there is substantial disagreement between native projections and downscaled analyses (Christensen et al. 2007). It is important to underscore that these classifications are relative, as the ensemble median is likely different than today’s conditions. This is particularly important in the climate change context, as nearly all GCMs project long-term warming trends across the world (alternatively, “relatively cool” and “relatively hot” may be classified as “warm” and “hot”). A given GCM with low levels of overall warming may be classified as “relatively cool” over a given region if it is cooler than the median of all GCMs. Likewise, a given GCM exhibiting a slight long-term drying trend may still be “relatively wet” if the full GCM ensemble median is projecting a more extreme drying trend. With a GCM’s projection classified as either relatively hot/cool or relatively wet/dry, this makes four basic quadrants of change: “cool/wet”, “cool/dry”, “hot/wet”, “hot/dry”.
A fifth, “Middle”, classification is also introduced in order to capture the models that represent the nexus of these quadrants around the median of the ensemble. Using the ensemble standard deviation (σ) of growing season temperature and precipitation changes as a representation of this model spread, we capture these models in a fifth quadrant including all models whose temperature and precipitation changes are within ± (ρ*σ) of the median, where ρ is a standard deviation factor designed as a simple measure of spread in order to capture approximately 1/5th (20%) of all GCM projections. This targets each quadrant to contain approximately the same number of models to minimize the number of GCMs that each quadrant’s selected model has to represent (the selection of this GCM is described in the next section). The use of standard deviations to define the bounds of the “Middle” quadrant does not hinge on an assumption that GCM projections are characterized by a Gaussian distribution of individual projections for both temperature and precipitation changes. Although a Gaussian form is most common, bimodality and distributions more precisely represented by other statistical forms do occur (Tebaldi et al. 2005). The middle quadrants in in this study are defined using ρ = 0.5 as it is a close fit to the Mid-Century optimal values for RCP8.5 and RCP4.5, as described in the Patterns of quadrant weights section below. When communicating the relevance of this quadrant to stakeholders it is important to emphasize that this middle quadrant is not an invitation to select only a single GCM. While this quadrant captures the center of the distribution, on average it is not substantially more likely than any of the other quadrants.
Choosing a representative model for each quadrant
It is not necessarily desirable that the model representing a given quadrant be centered within the quadrant, rather our desire is to pick a model that is in the center of the GCMs that fall in this quadrant as its role is to represent the types of change projected by these models. The selection of a representative model is therefore aided by calculating the quadrant center of mass for temperature and precipitation changes; achieved by marking the mean temperature and precipitation change for all of the GCMs that fall into a given quadrant (dots in Fig. 1). The models that fall into the middle quadrant of Fig. 1, for example, are centered slightly cooler and wetter than the median of the entire GCM ensemble. This should be reflected in the selection of a representative GCM for the middle quadrant.
Although the quadrant centers of mass are the best representations of GCMs within a given quadrant, these growing season averages do not contain sufficient information for many impacts assessments. To build the types of scenarios needed for more complex assessments, information is needed on the daily time series, frequency of extreme events, number of rainy days, covariation of meteorological variables, sub-seasonal variability, and other factors that cannot be gleaned from this simple averaging. Multi-model average projections can also wash out spatial and temporal patterns to a point where they are not physically plausible. By selecting a single model we end up with physically-consistent climate information from a simulation that resulted in the type of climate changes projected by GCMs in this quadrant. Drawbacks of this reliance upon a single model’s physics are discussed in the Discussion section.
Uncertainty in model projections,
GCM biases related to major patterns of atmospheric circulation
The sensitivity of a given system or assessment.
Model uncertainty may justify selecting a model other than that which is closest to the quadrant center of mass if an examination of the GCM spread reveals that the center of mass was drawn far away from most GCMs by an extreme outlier. For example, if a single model increases precipitation by 300% while the precipitation changes of the other models in a quadrant are limited to 20% gains or less, the model closest to the center of mass will likely be the wettest of the non-outliers. In this case it may make more sense to drop the outlier as a candidate and select a model close to the center of the remaining models.
Particular care must be taken in assessment regions governed by major atmospheric circulation patterns or seasonal cycles (IPCC 2010; Knutti et al. 2010a). The most prominent examples are monsoons (e.g., in South Asia, East Asia, Southwest North America, or West Africa), the migration of the Inter-Tropical Convergence Zone (ITCZ), and the seasonal cycle of sea-ice extent; each of which can dominate seasonal precipitation and temperature patterns for a given area. In these regions it is possible that the GCM that is closest to the quadrant center of mass misses the onset or exit of a rainy season (as captured in observational products such as AgMERRA), leading to climate changes for a given month that are based upon dry conditions when observations reveal periodic rainfall.
To illustrate a potential motivation for subjective selection based upon major atmospheric circulation patterns, imagine a quadrant where GCM A and GCM B both project nearly identical seasonal temperature and precipitation changes but GCM A is slightly closer to the quadrant center of mass. If GCM B has a climatology that generally matches the observed seasonal cycle of rainfall but GCM A reveals a delayed rainy season and thus a dry month when observations contain substantial rain, two likely biases may manifest in GCM A that would not be found in GCM B. First, rainfall changes in excess of ±50% are possible with even a small change in rainfall, which could have dramatic effects when imposed upon wetter conditions. Second, temperature changes are likely biased toward greater increase as excess energy that should drive latent heat exchanges is instead pushed into sensible heat due to moisture limitations (resulting in a higher Bowen Ratio in GCM A than GCM B). A comparison between observed rainfall seasonality and GCM rainfall seasonality is thus required before selecting the representative GCM for a given quadrant, with results potentially justifying selecting GCM B despite GCM A being the closest GCM to the quadrant center of mass. Within a given quadrant the approach is therefore similar to the Reliability Ensemble Approach (Giorgi and Mearns 2003; Xu et al. 2010) in its reliance on a historical period comparison, although the focus is on selecting a leading representative GCM rather than on reducing the impact of a GCM through weighting.
A final subjective consideration may be necessary in situations where the two GCMs closest to the quadrant center of mass are likely to produce substantially different results owing to the sensitivity of the system being assessed. To illustrate this point, imagine that the hot and dry quadrant contained GCM C and GCM D, each of which is approximately the same temperature and precipitation change away from the center of mass but in opposite directions (e.g., GCM C is 1 °C warmer and 10% drier than the quadrant center of mass; GCM D is 1 °C cooler and 10% wetter than the quadrant center of mass). In this example either GCM could be selected, but GCM C is more highly recommended due to its greater exploration of the sensitivity of the system to the hot and dry changes that its quadrant represents. This necessarily leads to slightly more extreme responses (particularly where systems are non-linear), but this information may be conveyed to stakeholders who can respond accordingly.
Quantifying the weights of representative models for each quadrant
Uncertainty in the ensemble of GCM projections is quantified using the probability that projections fall into any given quadrant. This follows an implicit assumption that models are exchangeable and equally likely manifestations of future climate, thus individual models are not differentially weighted (Knutti et al. 2010a). A quadrant weighting factor (W quadrant ) may therefore be calculated by dividing the number of models that falls into a given quadrant and by the total number of models in the ensemble (W quadrant = N quadrant /N Total ). These factors may then be passed on to later phases of the integrated assessment to represent model-based probability and confidence, allowing eventual results to be aggregated using a set of quadrant weight in factors that sum to 1. As the goal is to represent the entire ensemble of GCMs through selected GCMs, studies examining multiple time period/RCP combinations do not require the same subset of GCMs to be used for each combination. Rather, selecting the most representative GCMs and noting corresponding weights maintains more information about the overall ensemble.
In addition to carrying information about the probability of particular classes of climate change, quadrant weights also convey information about physical mechanisms prevalent in projected climate changes for a given region and season. The latter is recognizable as strong deviations from the expected weights if the 29 members of the GCM projection ensemble were distributed evenly across all five quadrants (20% of models in each quadrant).
“Diagonal skewness” is defined as existing when there are substantially more GCMs falling into a particular diagonal orientation of three quadrants than the 17.4 (60%) GCMs that would be expected. Specifically, site projections are considered to exhibit “hot/wet vs. cool/dry skewness” when at least 22 (75%) of the GCMs fall in either the hot/wet, middle, or cool/dry quadrants. Conversely, site projections feature “hot/dry vs. cool/wet skewness” when at least 22 (75%) of the GCMs fall in either the hot/dry, middle, or cool/wet quadrants.
“Extreme skewness” is defined as existing when there are substantially more (or fewer) GCMs falling into the middle quadrant than the 5.8 (20%) GCMs that would be expected. Specifically, site projections are considered “very extreme” when 3 (10.3%) or less of the 29 GCMs fall within the middle quadrant. This is approximately half of the expected value the middle quadrant, and also means that 26 (89.7%) or more of the GCMs fall into the cool/wet, cool/dry, hot/wet, or hot/dry quadrants. Site projections are considered “Non-extreme” when 9 (31%) or more models fall within the middle quadrant, which is 1.5 times the number expected.
The Ames, Iowa, maize-growing season example demonstrates the approach and reveals probabilistic information about climate projections for the area (Fig. 1). During the May-September maize-growing season the 29 CMIP5 GCMs project median temperature (T) rises of 3.24 °C and 5% precipitation (P) increases. These changes form the criteria for defining GCM projections as being relatively hot (ΔT > +3.24 °C), cool (ΔT < +3.24 °C), wet (P > +5% change), or dry (P < +5% change). A GCM projecting increases in rainfall of 0–5% would therefore still be considered “relatively dry” in comparison to the 29-member GCM ensemble despite not being drier than the historical conditions (there is only one model projecting an increase in Ames precipitation that would still be considered “relatively dry”). All GCMs (including both the “relatively hot” and “relatively cool”) project temperatures warmer than the current period.
The middle quadrant is defined using a range centered upon the median of one standard deviation of projected temperature (0.75 °C) and precipitation changes (11%). GCMs that are within one half standard deviation in both the temperature and precipitation change dimensions are therefore considered to be in the “middle” of the projection ensemble.
With the five quadrants defined using ensemble statistics, we find that there are 8 GCMs classified as “cool/wet” for the Ames maize-growing season (green triangles in Fig. 1), 3 are “cool/dry” (blue), 4 are “hot/wet” (yellow), 9 are “hot/dry” (red), and 5 are “middle” (black). Using the center of mass for projected changes in each quadrant (noted as a dot in Fig. 1) as a target temperature and precipitation change that best represents each quadrant, we can select five individual models to represent the broader ensemble at this location and growing season (triangles highlighted with gray edges in Fig. 1). None of these models demonstrate mean biases in the historical period that would justify elimination, and biases in the baseline period are not associated with projected changes (Additional file 1: Figure S1).
In this way the Representative T&P GCM Subsetting Approach encourages the system to be tested against the types of changes that could happen while also providing guidance about the changes that are more likely to happen according to the ensemble of GCM projections.
Patterns of change
The process by which GCMs are classified into five quadrants using the median and standard deviation of temperature and precipitation change projections also yields interesting information about the GCMs and the simulated climate system. While there is an element of subjectivity in the guided selection of representative GCMs for each quadrant, the delineation of quadrants is an objective process which can be repeated for all regions. Below, we analyze the classification (described in the Delineating climate change quadrants section) of all 29 CMIP5 GCMs listed in Table 1 for each 0.25° × 0.25° grid box of AgMERRA over both the entire year and over the maize-growing season as defined in the AgMIP GGCMI (grid boxes without a planting date are omitted).
Median and standard deviations of projections
The standard deviation of temperature changes across the 29 GCMs (Fig. 3e-f) show the largest values at high latitudes, over the Himalaya Mountains, and in the Amazon Basin. These are due in large part to model differences in ice-albedo feedback, the resolution of complex topography, and atmosphere-biosphere interactions, respectively (Flato et al. 2013). The standard deviation of temperature also tends to scale with median temperature increase. Precipitation change is remarkably consistent (Fig. 3g) except over the arid portions of North Africa and Western Asia where even small changes can lead to large percentage shifts in some GCMs. These patterns are largely replicated in the maize growing season (Fig. 3h), although standard deviations are slightly higher when fewer months are averaged together as individual models’ monthly biases tend to be larger than their annual bias.
Patterns of quadrant weights
Horizontal variation in quadrant weights reveals that small-scale deviations (within a couple degrees latitude and longitude) are typically small in comparison to emergent large-scale patterns that reveal a disproportionate number of GCMs in a given quadrant. Large-scale patterns highlight meridional variation (e.g., a reduction in hot and dry models at high latitudes for annual changes; Fig. 4i), major mountain chains and tundra (e.g., an increase in the number of middle quadrant GCMs over the Himalayas for annual changes; Fig. 4e), and semi-arid zones (e.g., common deviations over Australia and Southwestern Africa in many quadrants). This indicates that the probabilities captured by quadrant weights are not likely to vary tremendously over a region less than a couple degrees latitude or longitude across (which is typical of many impact studies) unless there is a major shift in aridity or orography. The physical mechanisms behind large scale patterns and resulting skewness are highlighted in the next section.
Percentage of 29 CMIP5 GCMs that fall into relative temperature and precipitation change quadrants for Mid-Century (2040–2069) RCP8.5
Annual changes over all land areas
Maize-season changes over all maize-growing areas
Relatively Cool and Wet
Relatively Cool and Dry
Relatively Hot and Wet
Relatively Hot and Dry
Patterns of skewness
The increase in maize skewness likely comes from the co-location of maize-growing areas with frost-free climates and seasons containing moderate amounts of rainfall. Due to temperature constraints maize is typically not grown at high-latitudes and is not in the ground during the wintertime storms that deliver a large portion of seasonal rainfall to the dry, primarily mid-latitude regions. Maize is therefore concentrated where soil moisture is variable and prevailing weather patterns associate rainfall with cool and wet air masses (typical of a mid-latitude summer) rather than warm and wet air masses (as are often found in wintertime or polar cyclones). Increased temperatures and energy are projected to raise global evapotranspiration demand, however some maize-growing regions will not be able to keep up with the elevated demand posed by the warmest GCMs and will therefore shift toward drier conditions. This in turn raises B, increasing sensible heat (and local temperatures) and leading to hot/dry vs. cool/wet skew such as that we saw over the Amazon for annual changes. The maize season maps do not include the winter season and high-latitude areas constrained by surface moisture and energy, leading to more instances where the warmest models increase evapotranspiration and drive wetter conditions (hot/wet vs. cool/dry skew). Figure 5c illustrates hot/wet vs. cool/dry skewness using an annual grid box in Massachusetts that is dominated by wintertime precipitation, while Fig. 5d uses the Ames maize-growing season from the gridded analysis to demonstrate hot/dry vs. cool/wet skewness.
The middle quadrant of both the globally-averaged annual and maize change distributions contain similar weights (19.1 and 18.8%, respectively for the Mid-Century RCP8.5), but some regions do exhibit extreme skew (Figs. 6c, d). Non-extreme skew is common in northern Siberia, where median temperature and precipitation changes are both among the highest on the planet. Temperature changes, in particular, have a very high standard deviation owing to model differences in climate sensitivity and local factors like the snow-ice albedo feedback. Non-extreme skew therefore is a description of the spread of GCM projections rather than the extremity of the GCM projections themselves. In fact non-extreme skew is found in places where standard deviation of GCM projections is so large due to outliers that the middle quadrant expands to capture many of the other GCMs. Figure 5e displays an example of this non-extreme skew near Lake Tamyr, Siberia, where the middle quadrant contains a disproportionate number of GCMs. Non-extreme skew is also seen where variations in the GCMs’ resolution of orography cause slightly larger standard deviations of temperature and precipitation change, including the highly productive plains and foothills just south of the Himalayas. Regions displaying very extreme skew (e.g., Fig. 5f) tend to be patchy but are most prevalent in Africa and Eastern Europe. Very extreme skew occurs when the GCMs tend to separate evenly into each of the outer quadrants while leaving very few models in the center of the distribution. In these regions GCMs show patterns of temperature and rainfall responses that are bimodal or disjointed in another way (potentially due to the acceleration of temperature or precipitation shifts beyond a given threshold or associated with a particular regime shift).
Re-constructing a GCM’s change patterns using quadrants
The global maps of GCM quadrant classifications also enable large-scale analysis through a recombination of regional integrated assessment results that were conducted using representative GCMs. For example, the first phase of AgMIP regional integrated assessments in sub-Saharan Africa and South Asia focused upon distributed regions linked together through the use of 5 common GCMs for all sites (Ruane et al. 2015b; Kihara et al. 2015; McDermid et al., 2015b). This guaranteed consistency in continent-wide analysis but also incorrectly assumed that the 5 selected GCMs adequately sampled projected climate changes for each region. In future phases of this work the Representative GCM approach will be used to capture the key classes of climate change within each region, providing more useful regional information. Results from these analyses will then enable continent-wide analyses linked either by quadrant (e.g., impact of relatively cool/wet climate scenarios on Africa) or by GCM (e.g., impact of HadGEM2-ES over Africa). For the latter analysis the impact pattern of the HadGEM2-ES can be constructed by identifying the type of change projected by that GCM over a given region and then utilizing the corresponding representative GCM from the regional integrated analysis. Through this relatively inexpensive approach one could examine the larger-scale patterns of change (which affect international trade, for example) across many GCMs while maintaining practically representative subsets for analysis within each region.
Benefits of representative T&P GCM subsetting approach
Practical in the number of resources required (reducing the CMIP5 GCM ensemble by ~1/6th to free up resources for other elements of a regional integrated assessment);
Defined according to changes in the season and variables that most affect the system of interest;
Characteristic of the major classes of climate change (relatively cool/wet, cool/dry, hot/wet, hot/dry, and middle) projected by the GCM ensemble;
Cognizant of potential outliers that may be evidence of substantial biases;
Connected to weights that denote the ensemble-based probability of GCM projections that are similar to each GCM within the subset;
Generalizable through standard definitions;
Capable of utilizing local observations or being built upon gridded climate products (such as AgMERRA);
Transferable from region to region and sector to sector;
Able to be recombined for larger-scale analyses;
Generally stable across alike geographical areas, time periods, and RCPs; and
Relatively simple to communicate to stakeholders.
Limitations and areas for continuing development
By definition, any subset of GCMs is a reduction in information compared to the full ensemble and therefore subsetting should only be undertaken when limited resources are required elsewhere in an integrated assessment process. A clear understanding of assumptions and limitations is therefore necessary when subsetting is necessary.
The Representative T&P GCM Subsetting Approach’s reliance on mean temperature and precipitation change projections will not account for differences in other climatic properties that may affect the system of interest (climate models have many degrees of freedom). Agricultural outcomes, for example, may be particularly sensitive to changes in the number of rainy days or the sub-seasonal breakdown of temperature increases (heat waves are particularly damaging during key plant development stages). Ruane et al. (2013) found that scenarios featuring only growing season mean temperature and precipitation changes slightly reduce the range of simulated yield changes compared to scenarios containing additional information about monthly temperature and rainfall changes. That representative models are similar to other models within their quadrant on variables other than growing season mean temperature and precipitation is implicitly assumed but worthy of further study. It is likely that close relationships exist (e.g., mean rainfall with relative humidity, cloud cover, or the number of rainy days) but that other variables may be more independent (e.g., interannual variability, the frequency of extreme events). The subset approach only selects GCMs for further analysis, and therefore the overall importance of variable changes other than the seasonal mean depends on the choice of a scenario generation approach that accounts for these changes as well as the sensitivity of the integrated assessment models to these changes.
Although the Representative T&P GCM Subsetting Approach can be generalized and applied objectively, subjective considerations related to the selection of specific representative GCMs (described in the Choosing a representative model for each quadrant section) may be very important in some regions. As an illustration, consider a case in which GCM E falls nicely near the center of mass as defined by other GCMs within the hot/dry quadrant but each of the other GCMs lies quite far from that center of mass. If GCM E contains substantial bias in comparison to historical observations of seasonal temperature or precipitation, an additional objective criterion is needed to decide at which point we accept GCM E rather than selecting the next-best GCM and therefore relying on a GCM that is less representative of the mean growing season climate changes.
Systems with multiple crops require special attention in defining a growing season in which to evaluate projected changes in temperature and precipitation and select representative GCMs. The Rice-Wheat systems of the Punjab in Pakistan, for example, are famously productive in utilizing monsoon rains during the wet (Kharif; May-October) season and harnessing the power of irrigation in the dry (Rabi; November-April) season (Ahmad et al. 2015). The economics of this rice-wheat system is better assessed using a single driving climate model rather than attempting to stitch together rice impacts simulated with one GCM and wheat impacts simulated by another. In this situation it is better to define the growing season as encompassing all months in which rice and wheat are cultivated, however investigation of the individual seasons is informative. Projections of temperature and precipitation change for the individual Kharif and Rabi seasons may distinguish models that are consistently within a given quadrants from those with different behaviors from season to season; information which could be incorporated into the selection of representative GCMs or passed on to the final risk assessment.
Objective rules for dealing with outliers would also benefit those seeking to automate the GCM selection process. These are most prevalent in the precipitation change dimension, and therefore several approaches merit further study. A first approach would be to cap precipitation changes (below 25% or above 200% of present day rainfall for any given month, for example) in order to limit their influence on the quadrant center of mass and the overall standard deviation of the GCM distribution. An alternate approach would be to fully eliminate GCMs where precipitation changes exceeded given thresholds, utilizing the remaining models within a quadrant to form a center of mass and select a representative GCM. This is often necessary in arid and monsoon-driven regions, as mismatches between the GCM and observed rainfall can lead to implausibly large percentage increases for months at the beginning or end of distinct rainy seasons (Additional file 1: Figure S3 presents the number of models that exceed a high threshold for each region). In both approaches the outlier could still be considered in the weighting of the quadrant, although examples are possible where large biases in a wet month that is dry within a GCM actually shift the growing season from a dry to a wet classification. Strong connections between a GCM’s baseline temperature and precipitation biases with that GCM’s eventual relative change quadrant may also suggest that dynamical or thermodynamical biases are affecting projections for a region (potentially due to sea ice/snow cover, monsoon circulations, or soil moisture anomalies), indicating the need to disqualify particularly egregious models (Additional file 1: Figure S1 shows that this is not the case for the rainfed maize season in Ames).
Any objective approach must also be careful to recognize that some extreme regional changes are quite plausible and grounded in strong model physics. These may be of utmost interest to stakeholders and should not be too quickly dismissed. For example, a maize farmer in Iowa may be more interested in the probability that his crop fails (an event at the distributional tail) than the expected future yield (a value in the center of the distribution). A stakeholder that is interested in simply stress-testing a system may therefore be more interested in selecting representative GCMs that bound the projected impacts than in establishing a probability of occurrence.
Very rarely (<2.5% of grid boxes for annual; <1% for maize) diagonal or extreme skew in a given location is so dramatic that an entire quadrant will be devoid of any GCM. For these locations it would be consistent to either ignore this quadrant (if its weight is 0% than it is not worth simulating) or define an additional quadrant to separate GCMs within the most heavily weighted quadrant.
The methods and analyses above were based upon an effort to select a 5-GCM subset from a broader 29-GCM CMIP5 ensemble, but different resource levels or systems of focus may call for larger or smaller subsets. In terms of selecting a larger number of GCMs, the next logical step would be to select 9 GCMs representing a 3x3 matrix of temperature and precipitation changes (allowing a quadrant for “hot/middle” changes, for example). Alternatively, a third dimension of analysis could be added, for example the standard deviation of temperatures (to emphasize extreme events) or end-of-season mean rainfall (to emphasize the important grain-filling stage of crops). This would also select 9 GCMs, as combinations of relatively high or low values for each of the three dimensions defines 8 quadrants in addition to the middle quadrant. While there is an appeal in using more complex statistics such as 99th percentile rainfall levels or the number of consecutive dry days (see, e.g., Peterson 2005), these values are not well simulated in many GCMs and there is resulting risk of determining selection on areas of GCM weakness. If an even-numbered subset is desired to avoid focus on a central scenario, the next reasonable level would be 8 GCMs, forming outer (cool/wet, cool/dry, hot/wet, hot/dry) and inner (middle/cool, middle/hot, middle/wet, middle/dry) quadrants. Alternatively, assessments of farm systems that are carefully irrigated to manage water stress may only be interested in selecting 3 representative GCMs for high, middle, and low temperature changes (likely closely related to their climate sensitivities). Irrigated farms may also prefer to use solar radiation change as an alternative dimension for quadrant analysis.
Other forms of selecting GCMs to represent classes of change are also possible, including the definition of alternative quadrants or more complex cluster methodologies potentially incorporating more variables. As sub-setting always loses information contained in the complete ensemble, many of the same challenges addressed above remain, including the potential that individual GCMs do not fall nicely into clusters or quadrants in some regions. More complex methods may become additionally problematic, however, if they are too complex for dissemination among partners from the variety of disciplines often included in integrated assessments. The temperature and precipitation quadrant approach described in this study is bolstered by its ease of generalization and communication with stakeholders.
On the global scale a quadrant approach defined by mean temperature and precipitation change is hindered by the globally-closed hydrologic cycle, which tends to respond to increasing temperatures by increasing the overall levels of both evapotranspiration and precipitation. As integrated assessments are dependent on the resolution of regional impacts, mean precipitation changes are not a sufficient metric as GCMs balance precipitation increases in one region with compensating drying in other areas. A better metric to gauge hydrologic impacts at the global scale is therefore the mean absolute percentage change of precipitation, which is indicative of the way that climate change strengthens and alters the geographical patterns within the water cycle. Global average temperature changes are closely related to the GCMs’ climate sensitivities listed in Table 1. In the process of selecting GCMs for a global study it would also be useful to examine the annual quadrant classifications in important regions of interest, as presented in Additional file 1: Figure S2. Further validation exercises that explore the extent and ramifications of regional temperature and precipitation biases within the GCMs would also be quite useful for a wide number of climate applications (Ruane et al. 2016).
Representative GCM selection for a national or regional-level integrated assessment is challenging in that it is unlikely that a single set of GCMs can represent the classes of change for all points within a large domain. Sub-regional assessments may be recombined for larger-scale evaluation as described in the Re-constructing a GCM’s change patterns using quadrants section, but in cases where economic analyses span the broader region it is important that inputs are consistent. In this situation an initial first step would be to perform quadrant analysis at a distributed network of sites within the domain and then look for particular GCMs that are suitably consistent representatives of a given quadrant. The final selection of GCMs may prioritize accurate representation of the GCM ensemble over key sub-regions (e.g., a country’s breadbasket in an agricultural study) and can also note which sub-regions may be missing a representative for a quadrant within the selected GCMs.
Summary and next steps
The Representative T&P GCM Subsetting Approach provides a practical way to reduce computational and analytical resources in integrated assessments of climate change impacts. Although information is lost in any subsetting of GCMs, this efficient approach captures the basic combinations of important climate change factors and their relative probabilities in order to enable stakeholder risk management. The core of the approach involves the analysis of major types of climate changes likely to affect a given sector (illustrated above for agriculture), with the goal of selecting GCMs that represent each major type of change and are associated with probabilistic information related to the broader ensemble. In some cases this analysis may lead to further stakeholder inquiry as to the extremes possible within a given quadrant, which could form the basis for continuing study.
The process of classifying GCMs relative to the wider ensemble of projected temperature and precipitation changes for a given region also provides useful insight into the sensitivity of these variables and the coherence of regional patterns across space, time, and greenhouse gas scenario. GCMs generally demonstrate noteworthy consistency, with some regions also demonstrating various forms of skewness in the full ensemble that are indicative of climatic processes or model uncertainty. As public attention following accords at the 21st Conference of Parties in Paris shifts increasingly toward the implementation of mitigation and adaptation strategies rooted in climate applications research, this type of analysis may help in the selection of GCM subsets covering the range of regional changes needed to increase resilience.
The authors benefited from multiple conversations with members of the AgMIP community who helped communicate the need for consistent and informative climate subsets. Dr. Ruane’s research was funded under the NASA Earth Sciences Division (WBS: 509496.02.08.04.24 and 281945.02.03.03.96). Dr. McDermid’s contributions were supported by the United Kingdom Department for International Development (DFID; Program Code 202108). We thank Olivier Crespo, Jack Simmons, and Shari Lifson for their support and advice. We acknowledge the World Climate Research Programme’s Working Group on Coupled Modelling, which is responsible for CMIP, and we thank the climate modeling groups (listed in Table 1 of this paper) for producing and making available their model output. For CMIP the U.S. Department of Energy’s Program for Climate Model Diagnosis and Intercomparison provides coordinating support and led development of software infrastructure in partnership with the Global Organization for Earth System Science Portals.
ACR developed the idea, conducted work and led manuscript writing. SM helped develop the idea and contributed to the manuscript writing. Both authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
- Ahmad A, et al (2015) Impact of Climate Change on the Rice–Wheat Cropping System of Pakistan. In: Rosenzweig C, Hillel, D (eds) Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP). C. Rosenzweig, and D. Hillel, Eds., ICP Series on Climate Change Impacts, Adaptation, and Mitigation Vol. 3 Part 2. Imperial College Press, 219–258, doi:https://doi.org/10.1142/9781783265640_0019
- Antle JM, Valdivia RO, Boote K, Janssen S, Jones JW, Porter CH, Rosenzweig C, Ruane AC, Thorburn PJ (2015) AgMIP’s transdisciplinary agricultural systems approach to regional integrated assessment of climate impacts, vulnerability, and adaptation. In: Rosenzweig C, Hillel, D (eds) Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP). ICP Series on Climate Change Impacts, Adaptation, and Mitigation Vol. 3. Part 1, Imperial College Press, 27–44, doi:https://doi.org/10.1142/9781783265640_0002
- Asseng S, Ewert F, Rosenzweig C, Jones JW, Hatfield JL, Ruane AC, Boote KJ, Thorburn PJ, Rötter RP, Cammarano D, Brisson N, Basso B, Martre P, Aggarwal PK, Angulo C, Bertuzzi P, Biernath C, Challinor AJ, Doltra J, Gayler S, Goldberg R, Grant R, Heng L, Hooker J, Hunt LA, Ingwersen J, Izaurralde RC, Kersebaum KC, Müller C, Naresh Kumar S, Nendel C, O’Leary G, Olesen JE, Osborne TM, Palosuo T, Priesack E, Ripoche D, Semenov MA, Shcherbak I, Steduto P, Stöckle C, Stratonovitch P, Streck T, Supit I, Tao F, Travasso M, Waha K, Wallach D, White JW, Williams JR, Wolf K (2013) Uncertainty in simulating wheat yields under climate change. Nature Clim Change 3:827–832. doi:https://doi.org/10.1038/nclimate1916 View ArticleGoogle Scholar
- Bassu S, Brisson N, Durand J-L, Boote K, Lizaso J, Jones JW, Rosenzweig C, Ruane ACR, Adam M, Baron C, Basso B, Biernath C, Boogaard H, Conijn S, Corbeels M, Deryng D, De Sanctis G, Gayler S, Grassini P, Hatfield J, Hoek S, Izaurralde C, Jongschaap R, Kemanian AR, Kersebaum KC, Kim SH, Kumar NS, Makowski D, Müller C, Nendel C, Priesack E, Pravia MV, Sau F, Shcherbak I, Tao F, Teixeira E, Timlin D, Waha K (2014) How do various maize crop models vary in their responses to climate change factors? Glob. Change Biol 20(7):2301–2320. doi:https://doi.org/10.1111/gcb.12520 View ArticleGoogle Scholar
- Christensen JH, Hewitson B, Busuioc A, Chen A, Gao X, Held I, Jones R, Kolli RK, Kwon W-T, Laprise R, Magana Rueda V, Mearns L, Menendez CG, Raisanen J, Rinke A, Sarr A, Whetton P (2007) Regional climate projections. In: Solomon S, Qin D, Manning M, Chen Z, Marquis M, Averyt KB, Tignor M, Miller HL (eds) Climate change 2007: the physical science basis. Contribution of Working Group I to the fourth assessment report of the intergovernmental panel on climate change. Cambridge University Press, Cambridge, United Kingdom, and New York, pp 847–845Google Scholar
- Collins M, Knutti R, Arblaster J, Dufresne J-L, Fichefet T, Friedlingstein P, Gao X, Gutowski WJ, Johns T, Krinner G, Shongwe M, Tebaldi C, Weaver AJ, Wehner M (2013) Long-term climate change: projections, commitments and irreversibility. In: Stocker TF, Qin D, Plattner GK, Tignor M, Allen SK, Boschung J, Nauels A, Xia Y, Bex V, Midgley PM (eds) Climate change 2013: the physical science basis. Contribution of Working Group I to the fifth assessment report of the intergovernmental panel on climate change. Cambridge University Press, Cambridge, United Kingdom and New YorkGoogle Scholar
- Elliott J, Müller C, Deryng D, Chryssanthacopoulos J, Boote KJ, Büchner M, Foster I, Glotter M, Heinke J, Iizumi T, Izaurralde RC, Mueller ND, Ray DK, Rosenzweig C, Ruane AC, Sheffield J (2015) The global gridded crop model intercomparison: data and modeling protocols for phase 1 (v1.0). Geosci Model Dev 8:261–277. doi:https://doi.org/10.5194/gmd-8-261-2015 View ArticleGoogle Scholar
- Eyring V, Bony S, Meehl GA, Senior CA, Stevens B, Stouffer RJ, Taylor KE (2016) Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization. Geosci Model Dev 9:1937–1958. doi:https://doi.org/10.5194/gmd-9-1937-2016 View ArticleGoogle Scholar
- Flato G, Marotzke J, Abiodun B, Braconnot P, Chou SC, Collins W, Cox P, Driouech F, Emori S, Eyring V, Forest C, Gleckler P, Guilyardi E, Jakob C, Kattsov V, Reason C, Rummukainen M (2013) Evaluation of climate models. In: Stocker TF, Qin D, Plattner GK, Tignor M, Allen SK, Boschung J, Nauels A, Xia Y, Bex V, Midgley PM (eds) Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, 741–882, doi:https://doi.org/10.1017/CBO9781107415324.020
- Giorgi F, Mearns LO (2003) Probability of regional climate change based on the Reliability Ensemble Averaging (REA) method. Geophys Res Lett 30(12):1629. doi:https://doi.org/10.1029/2003GL017130 View ArticleGoogle Scholar
- Gleckler PJ, Taylor KE, Doutriaux C (2008) Performance metrics for climate models. J Geophys Res 113(D6):D06104, doi:https://doi.org/10.1029/2007JD008972
- Howden M, Crimp S (2005) Assessing dangerous climate change impacts on Australia’s wheat industry. MODSIM 2005 International Congress on Modeling and Simulation. Modelling and Simulation Society of Australia and New Zealand., pp 170–176. ISBN 0-9758400-2-9Google Scholar
- IPCC (2010) Meeting report of the intergovernmental panel on climate change expert meeting on assessing and combining multi model climate projections. In: Stocker TF, Qin D, Plattner G-K, Tignor M, Midgley PM (eds) IPCC Working Group I Technical Support Unit. University of Bern, Bern, p 117. ISBN 978-92-9169-129-6Google Scholar
- Kihara J, MacCarthy DS, Bationo A, Koala S, Hickman J, Koo J, Vanya C, Adiku S, Beletse Y, Masikate P, Rao KPC, Mutter CZ, Rosenzweig C, Jones JW (2015) Perspectives on Climate Effects on Agriculture: The International Efforts of AgMIP in Sub-Saharan Africa. In: Rosenzweig C, Hillel, D (eds) Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP). ICP Series on Climate Change Impacts, Adaptation, and Mitigation Vol. 3 Part 2. Imperial College Press, p. 3–23. doi:https://doi.org/10.1142/9781783265640_0013
- Knutti R (2010) The end of model democracy? Clim Change 102(3):395–404. doi:https://doi.org/10.1007/s10584-010-9800-2 View ArticleGoogle Scholar
- Knutti R (2014) IPCC Working Group I AR5 snapshot: the historical, rcp4.5, and rcp8.5 experiments. WDCC at DKRZ., doi:https://doi.org/10.1594/WDCC/ETHhi; doi:https://doi.org/10.1594/WDCC/ETHr4; doi:https://doi.org/10.1594/WDCC/ETHr8 Google Scholar
- Knutti R, Abramowitz G, Collins M, Eyring V, Gleckler PJ, Hewitson B, Mearns L (2010a) Good practice guidance paper on assessing and combining multi model climate projections. In: Stocker TF, Qin D, Plattner G-K, Tignor M, Midgley PM (eds) Meeting report of the intergovernmental panel on climate change expert meeting on assessing and combining multi model climate projections. IPCC Working Group I Technical Support Unit. University of Bern, BernGoogle Scholar
- Knutti R, Furrer R, Tebaldi C, Cermak K, Meehl GA (2010b) Challenges in combining projections from multiple models. J Clim 23:2739–2756. doi:https://doi.org/10.1175/2009JCLI3361.1
- Li T, Hasegawa T, Yin X, Zhu Y, Boote K, Adam M, Bregaglio S, Buis S, Confalonieri R, Fumoto T, Gaydon D, Marcaida M III, Nakagawa H, Oriol P, Ruane AC, Ruget F, Singh B, Singh U, Tang L, Tao F, Wilkens P, Yoshida H, Zhang Z, Bouman B (2015) Uncertainties in predicting rice yield by current crop models under a wide range of climatic conditions. Glob Change Biol 21(3):1328–1341. doi:https://doi.org/10.1111/gcb.12758 View ArticleGoogle Scholar
- Lutz AF, ter Maat HW, Biemans H, Shresth AB, Wester P, Immerzeel WW (2016) Selecting representative climate models for climate change impact studies: an advanced envelope-based selection approach. Int J Climatol 36:3988–4005. doi:https://doi.org/10.1002/joc.4608 View ArticleGoogle Scholar
- Martre P, Wallach D, Asseng S, Ewert F, Jones JW, Rötter RP, Boote KJ, Ruane AC, Thorburn PJ, Cammarano D, Hatfield JL, Rosenzweig C, Aggarwal PK, Angulo C, Basso B, Bertuzzi P, Biernath C, Brisson N, Challinor AJ, Doltra J, Gayler S, Goldberg R, GrantRF HL, Hooker J, Hunt LA, Ingwersen J, Izaurralde RC, Kersebaum KC, Müller C, Kumar SN, Nendel C, O’Leary G, Olesen JE, Osborne TM, Palosuo T, Priesack E, Ripoche D, Semenov MA, Shcherbak I, Steduto P, Stöckle CO, Stratonovitch P, Streck T, Supit I, Tao F, Travasso M, Waha K, White JW, Wolf J (2015) Multimodel ensembles of wheat growth: many models are better than one. Glob Change Biol 21(2):911–925. doi:https://doi.org/10.1111/gcb.12768 View ArticleGoogle Scholar
- McDermid, SP, Ruane AC, Rosenzweig C, Hudson NI, Morales MD, Agalawatte P, Ahmad S, Ahuja LR, Amien I, Anapalli SS, Anothai J, Asseng S, Biggs J, Bert F, Bertuzzi P, Bhatia V, Bindi M, Broad I, Cammarano D, Carretero R, Chattha AA, Chung U, Debats S, Deligios P, De Sanctis G, Dhliwayo T, Dumont B, Estes L, Ewert F, Ferrise R, Gaiser T, Garcia G, Gbegbelegbe S, Geethalakshmi V, Gerardeaux E, Goldberg R, Grant B, Guevara E, Hickman J, Hoffmann H, Huang H, Hussain J, Justino FB, Karunaratne AS, Koehler A-K, Kouakou PK, Kumar SN, Lakshmanan A, Lieffering M, Lin X, Luo Q, Magrin G, Mancini M, Marin FR, Marta AD, Masutomi Y, Mavromatis T, McLean G, Meira S, Mohanty M, Moriondo M, Nasim W, Negm N, Orlando F, Orlandini S, Ozturk I, Pinto HMS, PodestaG, Qi Z, Ramarohetra J, Rahman HH, Raynal H, Rodriguez G, Rötter R, Sharda V, Shuo L, Smith W, Snow V, Soltani A, Srinivas K, Sultan B, Swain DK, Tao F, Tesfaye K, Travasso MI, Trombi G, Topaj A, Vanuytrecht E, Viscarra FE, Wajid SA, Wang E, Wang H, Wang J, Wijekoon E, Byun-Woo L, Xiaoguang Y, Young BH, Yun JI, Zhao Z, Zubair L (2015a) The AgMIP Coordinated Climate-Crop Modeling Project (C3MP): Methods and protocols. In: Rosenzweig C, Hillel, D, (Ed), Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP) Integrated Crop and Economic Assessments, Part 1. ICP Series on Climate Change Impacts, Adaptation, and Mitigation Vol. 3. Part 1, Imperial College Press, p 191–220, doi:https://doi.org/10.1142/9781783265640_0008.
- McDermid SP, Dileepkumar G, Dakshina Murthy KM, Nedumaran S, Singh P , Srinivasa C, Gangwar B, Subash N, Ahmad A, Zubair L, Nissanka SP (2015b) Integrated Assessments of the Impacts of Climate Change on Agriculture: An Overview of AgMIP Regional Research in South Asia. In: Rosenzweig C, Hillel, D, (Eds), Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP). ICP Series on Climate Change Impacts, Adaptation, and Mitigation Vol. 3. The Agriculutral Model Intercomparison and Improvement Project Integrated Crop and Economic Assessments, Part 2 (pp. 201–217). Imperial College Press. doi:https://doi.org/10.1142/9781783265640_0018
- McSweeney CF, Jones RG (2016) How representative is the spread of climate projections from the 5 CMIP5 GCMs used in ISI-MIP? Climate Services 1, doi:https://doi.org/10.1016/j.cliser.2016.02.001
- Moss RH, Edmonds JA, Hibbard KA, Manning MR, Rose SK, van Vuuren DP, Carter TR, Emori S, Kainuma M, Kram T, Meehl GA, Mitchell JFB, Nakicenovic N, Riahi K, Smith SJ, Stouffer RJ, Thomson AM, Weyant JP, Wilbanks TJ (2010) The next generation of scenarios for climate change research and assessment. Nature 463(7282):747–756. doi:https://doi.org/10.1038/nature08823 View ArticleGoogle Scholar
- Peterson TC (2005) Climate change indices. WMO Bull 54(2):83–86Google Scholar
- Portmann F et al (2008) Global data set of monthly growing areas of 26 irrigated crops. Institute of Physical Geography, University of Frankfurt, Frankfurt am Main, p 400Google Scholar
- Portmann FT, Siebert S, Döll P (2010) MIRCA2000-Global monthly irrigated and rainfed crop areas around the year 2000: a new high-resolution data set for agricultural and hydrological modeling. Global Biogeochem Cycles 24:Gb1011View ArticleGoogle Scholar
- Reichler T, Kim J (2008) How well do coupled models simulate today’s climate? Bull Am Met Soc 89:303–311View ArticleGoogle Scholar
- Rienecker MM et al (2011) MERRA: NASA’s modern-era retrospective analysis for research and applications. J Climate 24:3624–3648. doi:https://doi.org/10.1175/JCLI-D-11-00015.1 View ArticleGoogle Scholar
- Rosenzweig C, Jones JW, Hatfield JL, Ruane AC, Boote KJ, Thorburn P, Antle JM, Nelson GC, Porter C, Janssen S, Asseng S, Basso B, Ewert F, Wallach D, Baigorria G, Winter JM (2013) The Agricultural Model Intercomparison and Improvement Project (AgMIP): protocols and pilot studies. Ag For Meteor 170:166–182. doi:https://doi.org/10.1016/j.agrformet.2012.09.011
- Rosenzweig C, Jones JW, Hatfield JL, Antle JM, Ruane AC, Mutter CZ (2015): The Agricultural Model Intercomparison and Improvement Project: Phase I activities by a global community of science. In: Rosenzweig C, Hillel, D, (Eds), Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP). ICP Series on Climate Change Impacts, Adaptation, and Mitigation Vol. 3. Part 1, Imperial College Press, p 3–24. doi:https://doi.org/10.1142/9781783265640_0001.
- Rosenzweig C, Antle J, Elliott J (2016) Assessing Impacts of Climate Change on Food Security Worldwide. Eos, 97, doi:https://doi.org/10.1029/2016EO047387. Published on 9 March 2016.
- Ruane AC, Roads JO (2008) Dominant balances and exchanges of the atmospheric water cycle in the Reanalysis-2 at diurnal, annual, and intraseasonal time scales. J Climate 21(16):3951–3966. doi:https://doi.org/10.1175/2007JCLI2015.1 View ArticleGoogle Scholar
- Ruane AC, Cecil LD, Horton RM, Gordón R, McCollum R, Brown D, Killough B, Goldberg R, Greeley AP, Rosenzweig C (2013) Climate change impact uncertainties for maize in Panama: farm information, climate projections, and yield sensitivities. Agr Forest Meteorol 170:132–145. doi:https://doi.org/10.1016/j.agrformet.2011.10.015 View ArticleGoogle Scholar
- Ruane AC, McDermid S, Rosenzweig C, Baigorria GA, Jones JW, Romero CC, Cecil LD (2014) Carbon–Temperature–Water change analysis for peanut production under climate change: a prototype for the AgMIP Coordinated Climate-Crop Modeling Project (C3MP). Glob Chang Biol 20:394–407. doi:https://doi.org/10.1111/gcb.12412 View ArticleGoogle Scholar
- Ruane AC, Goldberg R, Chryssanthacopoulos J (2015a) Climate forcing datasets for agricultural modeling: Merged products for gap-filling and historical climate series estimation. Agr Forest Meteorol 200. doi:https://doi.org/10.1016/j.agrformet.2014.09.016
- Ruane AC, Winter JM, McDermid SP, Hudson NI (2015b) AgMIP climate datasets and scenarios for integrated assessment. In: Rosenzweig C, Hillel, D, (Eds), Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP). ICP Series on Climate Change Impacts, Adaptation, and Mitigation Vol. 3. Part 1, Imperial College Press, p 45–78, doi:https://doi.org/10.1142/9781783265640_0003.
- Ruane AC, Teichmann C, Arnell N, Carter TR, Ebi KL, Frieler K, Goodess CM, Hewitson B, Horton R, Kovats RS, Lotze HK, Mearns LO, Navarra A, Ojima DS, Riahi K, Rosenzweig C, Themessl M, Vincent K (2016) The Vulnerability, Impacts, Adaptation and Climate Services Advisory Board (VIACS AB v1.0) contribution to CMIP6. Geosci Model Dev 9:3493–3515. doi:https://doi.org/10.5194/gmd-9-3493-2016 View ArticleGoogle Scholar
- Sacks WJ et al (2010) Crop planting dates: an analysis of global patterns. Glob Ecol Biogeogr 19(5):607–620Google Scholar
- Semenov MA, Stratonovich P (2015) Adapting wheat ideotypes for climate change: accounting for uncertainties in CMIP5 climate projections. Clim Res Vol 65:123–139. doi:https://doi.org/10.3354/cr01297 View ArticleGoogle Scholar
- Sriver RL, Forest CE, Keller K (2015) Effects of initial conditions uncertainty on regional climate variability: an analysis using a low-resolution CESM ensemble. Geophys Res Lett 42:5468–5476. doi:https://doi.org/10.1002/2015GL064546 View ArticleGoogle Scholar
- Taylor KE, Stouffer RJ, Meehl GA (2012) An overview of CMIP5 and the experiment design. Bull Am Meteorol Soc 93(4):485–498, doi:https://doi.org/10.1175/BAMS-D-11-00094.1
- Tebaldi C, Smith RL, Nychka D, Mearns LO (2005) Quantifying uncertainty in projections of regional climate change: a Bayesian approach to the analysis of multimodel ensembles. J Clim 18:1524–1540View ArticleGoogle Scholar
- Trenberth K (2011) Changes in precipitation with climate change. Clim Res 47:123–138. doi:https://doi.org/10.3354/cr00953 View ArticleGoogle Scholar
- Warszawski L, Frieler K, Huber V, Piontek F, Serdeczny O, Schewe J (2014) The Inter-Sectoral Impact Model Intercomparison Project (ISI–MIP): project framework. P Natl Acad Sci USA 111:3228–3232. doi:https://doi.org/10.1073/pnas.1312330110 View ArticleGoogle Scholar
- White JW, Hoogenboom G, Kimball BA, Wall GW (2011) Methodologies for simulating impacts of climate change on crop production. Field Crop Res 124(3):357–368. doi:https://doi.org/10.1016/j.fcr.2011.07.001 View ArticleGoogle Scholar
- Wilby RL, Charles S, Zorita E, Timbal B, Whetton P, Mearns L (2004) Guidelines for use of climate scenarios developed from statistical downscaling methods., IPCC Supporting Material, available from the DDC of IPPC TGCIAGoogle Scholar
- Xu Y, Gao X, Giorgi F (2010) Upgrades to the reliability ensemble averaging method for producing probabilistic climate-change projections. Clim Res 1:61–81. doi:https://doi.org/10.3354/cr00835 View ArticleGoogle Scholar