Originally a guest post on September 14, 2013 at Climate Etc.

These comments constitute a response to erroneous statements and misrepresentations made in a report published by the Met Office in July 2013: “The recent pause in global warming (3): What are the implications for projections of future warming?” (the Report).

A significant part of the Met Office Report concerns what it refers to as a recent comprehensive study, being the 2013 Nature Geoscience paper “Energy budget constraints on climate response” (Otto et al). The Report contains several misrepresentations of the findings of the Otto et al study as well as a number of other misleading or erroneous statements.

The author team of Otto et al includes fourteen lead or coordinating lead authors of the forthcoming IPCC scientific report (AR5 WG1 report). The author of this memorandum, Nicholas Lewis, whilst being one of the authors of the Otto et al 2013 study, writes in a personal capacity, not as a representative of its author team.

The Met Office Report discusses in some detail estimates of transient climate response (TCR) and equilibrium/effective climate sensitivity (ECS). TCR, a measure of the rise in global surface temperature at the end of a 70 year period over which atmospheric CO2-equivalent concentrations grow at 1% per annum (and hence double), is closely linked to projections of human-caused warming several decades into the future. ECS measures the eventual surface  temperature increase from such a doubling once ocean temperatures have fully adjusted. ECS largely determines TCR.

The Met Office Report considers whether climate model estimates of TCR and ECS need to be revised in the light of recent observational evidence, in particular the relatively slow increase in global surface temperature over the last 15 to 20 years. It concludes that, broadly, such estimates – and hence climate model projections of future warming – do not need to be reduced.

One of the Met Office Report’s main conclusions is that “the upper ranges of TCR and ECS derived from extended observational records … are broadly consistent with the upper range from the latest generation of comprehensive climate models” (CMIP5 models). This is contradicted by Otto et al, which stated “Our results match those of other observation-based studies and suggest that the TCRs of some of the models in the CMIP5 ensemble with the strongest climate response to increases in atmospheric CO2 levels may be inconsistent with recent observations”.  Barely half CMIP5 models analysed (Forster et al. 2013) have TCRs below the 95% upper bound for TCR of 2.0°C given in Otto et al, using observational data for the latest decade.

These issues are of particular relevance to the Met Office. Both the TCR and ECS of its flagship HadGEM2-ES model, used for policy advice, are very near the top of the range for CMIP5 models (Forster et al. 2013) . Its TCR exceeds the 95% bound derived from CMIP5 models other than HadGEM2-ES. The HadGEM2-ES model’s TCR also exceeds the 95% bound derived from previous generation (CMIP3) models. The Met Office HadGEM2-ES model’s TCR of 2.5°C is not only well above the upper 95% bound of 2.0°C given in Otto et al, but also above the 2.3°C bound given in Gillett et al 2013 – the only other study cited in the Met Office Report that derived TCR from observational records. Indeed, the HadGEM2-ES TCR is nearly double the Otto et al best estimate for TCR of 1.3°C. As for ECS, the HadGEM2-ES model’s ECS of 4.6°C lies well beyond the upper 95% bounds given not only by Otto et al6 but also by Aldrin et al 2012 (3.5°C) and Lewis 2013 (3.0°C). HadGEM2’s ECS also exceeds the 4.5°C top of the IPCC’s ‘likely’ range, and the 95% bounds both from CMIP3 models and from CMIP5 models excluding HadGEM2-ES (Murphy et al. 2009).

The TCR of the Met Office’s previous generation climate model, HadCM3, is at the 95% upper bound for TCR of 2.0°C given in Otto et al. A perturbed physics ensemble study based on adjusting the parameters in HadCM3, now set out in two major published papers (Sexton et al 2012 and Harris et al 2013), represents the techniques and climate model on which the official UK Climate Projections (Murphy et al. 2009) produced by the Met Office were based. HadCM3’s high TCR might not matter if, as the Report claims, “uncertainty in the response of the climate system to CO2 forcing is comprehensively sampled” in the study. However, it is not. Despite thorough attempts to make it do so by varying its parameters, HadCM3 is unable to sample the region where, according to several recent observational studies, key characteristics of the real climate system are most probably located. It is therefore unsurprising that incorporating observational data barely alters the Harris et al 2013 prior central estimate for TCR.

Misrepresentations relating to transient climate response (TCR)

The Met Office Report refers to three methods of estimating TCR: from simulations made with climate models, from observations, and by combining climate model and observationally-derived values. It makes the contentious claim that none of these methods can be said to be superior to the others. In science, it is standard to test the validity of theoretical models by comparing their predictions to observational data. Accordingly, it seems possible to say that estimates derived purely from simulations by climate models, without combination with observationally-derived values, are likely to be inferior to those from the other two methods.

Figure 1 and Table 1

Figure 1 in the Report, which gives the Otto et al TCR 5–95% range as 0.7 to 2.5°C, most likely value 1.4°C, based on 1970–2009 data, is objectionable on a number of grounds:

1.  Figure 1 does not use the Otto et al primary TCR best estimate of 1.3°C and 5–95% range of 0.9 to 2.0°C, based on data for the decade 2000–09. Although caution is required in interpreting results for any short period, arguably – as stated in Otto et al – the estimate based on the most recent decade’s data is the most reliable since it has the strongest forcing and is much less affected by the 1991 eruption of Mount Pinatubo. Accordingly, showing in Figure 1 only the (wider) TCR estimated range based on 1970–2009 data for Otto et al for comparison with other estimates is misleading.

2. Classifying the Harris et al 2013 HadCM3-perturbed-physics-ensemble derived TCR estimates as based on model and observations gives a misleading impression of the relative influence of those two factors. Although highly sophisticated, the study would more appropriately be classified as primarily model based. As HadCM3’s parameters are perturbed, the resulting changes in ECS and aerosol forcing are closely linked. When significantly lower values for ECS – as suggested by recent observational studies – are obtained, HadCM3’s aerosol forcing takes on highly negative values. The observational data strongly contraindicate aerosol forcing being highly negative, so parameter combinations resulting in significantly reduced model ECS levels (and thus highly negative aerosol forcing) are heavily down-weighted. As a result, whatever the actual level of ECS, HadCM3-derived ECS estimates are bound to be high. See Box 1 [refer to full document Lewis UKMO]. The HadCM3-derived TCR estimates are very largely determined by its ECS estimates, so the same is true for TCR. Since the Harris et al 2013 results largely reflect the particular characteristics of the HadCM3 model, and are at variance with results from several recent fully observationally-constrained studies, no reliance should be placed upon them.

3. The central TCR estimate of 1.6°C given for Gillett et al 2013 is a mean, and is not comparable to the median estimates quoted for Otto et al and Harris et al 2013. Since TCR and ECS probability distributions are usually skewed, the median – the 50th percentile of the distribution – is a much preferable central estimate for TCR and ECS than the mean (the probability-weighted average value). Gillett et al 2013 used a standard detection-and-attribution method, giving regression coefficients which can then be multiplied by the model TCRs to produce observationally-based TCR estimates. The median TCR estimate based on individual model regression coefficients was 1.46°C.

4.  The multi-model CMIP3 and CMIP5 central TCR estimates given of 1.8°C are also means rather than medians. The median TCR for the set of CMIP3 models referenced is lower, at 1.6°C. (The mean–median difference is negligible for the set of CMIP5 models.)

Several of the TCR estimates given for Otto et al in Table 1 are wrong. The central TCR estimate based on the 2000s observational period is given as 1.4°C. It was actually 1.3°C. Several of the other TCR percentiles given are also wrong

A revised version of Figure 1 in the Met Office Report showing the impact of the revisions discussed above, and a better idea of the distribution of probability, is shown in Figure 1.

Figure 1: Estimates of TCR from the same sources as in Figure 1 of the Met Office Report, with estimates based on 2000-2009 data added for Otto et al.

The flasks span the 5–95 % uncertainty ranges for the estimates from each source; the black horizontal lines mark their 50th percentiles (medians). The width of the flask shows probability density, in each case for the shifted lognormal distribution corresponding to the applicable 5th, 50th and 95th percentiles. Since flasks from all sources therefore have equal area, probability is proportional to area between as well as within flasks.  Short white horizontal bars show the central estimate for each source given in Figure 1 of the Met Office Report. The red bar in the CMIP5 column shows the TCR of the Met Office HadGEM2-ES model. The colours denote type of source, with blue showing estimates based on observations, purple showing estimates based primarily on model but partly on observations, and salmon for model-only estimates.

Physical considerations

The Met Office Report states that “To reach the very low values [for TCR] quoted in Otto et al. (2013) would require negative feedbacks to be acting quite strongly to counteract the well understood physics of greenhouse gas radiative forcing, water vapour feedback and surface albedo feedback”. That is misleading. The strong negative lapse rate feedback is very closely linked to the water vapour feedback (they are sometimes combined into a single feedback) and has a similar level of understanding. Therefore, it should be included along with greenhouse gas radiative forcing, water vapour feedback and surface albedo feedback.

A multi-model study of feedbacks, Soden and Held 2006, showed a median ECS for the model ensemble of 1.8°C after combined water vapour/lapse rate and surface albedo feedbacks. The median 1.3°C and 1.4°C estimates for TCR in Otto et al both correspond to ECS estimates above 1.8°C, so are consistent with the Soden and Held 2006 findings without requiring any additional negative feedbacks to be acting. Moreover, real-world water vapour feedback may not be as strong as in typical climate model simulations. Although the basic physics of these feedbacks may be well understood, there remains substantial uncertainty as to their magnitudes. Furthermore, cloud feedbacks are highly uncertain.

Estimated warming at 2100

The Met Office Report miscalculates estimated warming at 2100 under the RCP8.5 scenario for all the studies shown, mainly by using inconsistent bases in the calculation. Figure 2 of the  Report also contains an additional misrepresentation of Otto et al’s results. Both issues can be illustrated using the values for Otto et al. The 5–95% range given in Figure 2 of the Report for Otto et al, of 1.7–6.2°C with a median value of 3.5°C, is evidently based on multiplying the study’s TCR estimates based on 1970–2009 data by 8.5 Wm-2 and dividing by 3.44 Wm-2, the CMIP5 mean F2x level in Forster et al 2013, as used in Otto et al. The RCP8.5 scenario was so named because the resulting forcing approaches 8.5 Wm-2 in 2100.  However, that was based on a different, higher, F2x basis. In fact, the RCP8.5 indicative forcings dataset (Meinshausen et al, 2011) gives a level of 8.34 Wm-2 in 2100, not 8.5 Wm-2, and it is based on a canonical F2x estimate of 3.71 Wm-2, not 3.44 Wm-2. So the RCP8.5 forcing in 2100 should be divided by 3.71 Wm-2, not 3.44 Wm-2, in order to estimate warming in 2100. Substituting that divisor in the calculation, and taking the indicative RCP8.5 mean forcing over 2095-2105 to even out the solar cycle, reduces the 3.5°C median 2100 warming estimate from Otto et al using 1970–2009 data to 3.1°C.

Moreover, the Met Office Report does not reveal that the warming at 2100 shown for Otto et al is based on its TCR estimates using data for 1970–2009 rather than its primary, arguably more reliable, TCR estimates based on data for the 2000s. Using those estimates instead, the median 2100 warming estimate from Otto et al under the RCP8.5 scenario becomes 2.9°C, with a 5–95% range of 2.0–4.4°C. Both the median estimate and the top of the 95% range are substantially lower than the figures of respectively 3.5°C and 6.2°C given in Figure 2 of the Report. Note that the statement in the caption thereto that medians are marked is incorrect for 3 out of 5 sources; it is means that are marked, except for Otto et al and Harris et al 2013.

Figure 2: Estimated warming at 2100 relative to pre-industrial levels under the RCP8.5 scenario.

Values are calculated as in Figure 2 of the Met Office Report but on a corrected basis and using the TCR values from Figure 1 above rather than from Figure 1 of the Report. The flasks span the 5–95 % uncertainty ranges for the estimates from each source; the black horizontal lines mark their 50th percentiles (medians). The width of the flask shows probability density, in each case for the shifted lognormal distribution corresponding to the applicable 5th, 50th and 95th percentiles. Since flasks from all sources therefore have equal area, probability is proportional to area between as well as within flasks. Short white horizontal bars show the central estimate for each source given in Figure 2 of the Report. The red bar in the CMIP5 column shows the Estimated warming at 2100 for the Met Office HadGEM2-ES model. The colours denote type of source, with blue showing estimates based on observations, purple showing estimates based primarily on model but partly on observations, and salmon for model-only estimates.

The use of inconsistent F2x bases and overstatement of RCP8.5 forcing at 2100 does not just affect the Otto et al study. All the estimated 2100 warming figures given for Gillett et al 2013, Harris et al 2013, CMIP3 and CMIP5 similarly need to be reduced by 10% in order to correct these errors. A revised version of Figure 2 in the Met Office Report showing the impact of these corrections is shown in Figure 2. As noted in the Report, RCP8.5 is the highest of all the RCP scenarios, which should be borne in mind when considering the projected Warming to 2100 levels in Figure 2.

The uncertainty ranges shown both in the Met Office Report Figure 2 and Figure 2 here are in any case exaggerated, since 25–30% of the decadal-mean radiative forcing projected under RCP8.5 for 2100 relative to pre-industrial levels has already occurred. Moreover, according to observational records, by 2012 the global mean temperature had increased by about 0.8°C. That is in line, scaling pro rata, with 25–30% of the medians for the Otto et al and Gillett et al projected increases to 2100. But the non-observational studies’ central warming projections need to be adjusted down to reflect their overestimates of the temperature increase to date.

Misrepresentations relating to equilibrium climate sensitivity (ECS)

Estimation of equilibrium climate sensitivity is dealt with in Section 4 of the Met Office Report. It repeats the misleading claim that “Positive feedbacks in the physical climate system, the largest of which is the water-vapour feedback, increase this number [ECS] to over 2°C”, and compounds this distortion by stating that “the fundamental physics of climate sensitivity, involving black body radiation and water vapour feedbacks … alone give a climate sensitivity of at least 2.0°C”.  As already pointed out, these claims ignore the negative lapse-rate feedback, which is intimately linked to the water-vapour feedback. After including lapse-rate feedback as well as water-vapour feedback, all the models analysed in Soden and Held 2006 had a climate sensitivity of below 2.0°C.

The Met Office Report also claims, concerning the Otto et al ECS estimate based on data for the 2000s, that “As for TCR the observations of most recent decade are not representative of the full observational record and so not expected to be representative of the longer term future.” That claim is not valid. Decadal-scale internal variability of global temperature was allowed for in Otto et al. Moreover, the slower increase in surface temperatures during the 2000s was, as one would expect, associated with a higher estimated rate of ocean heat uptake, which increases the ECS estimate. The Otto et al ECS estimates based on data for each of the 1980s, the 1990s and the 2000s, and for 1970-2009, measured in each case relative to estimates for 1860–79, were all very similar. Indeed, the Otto et al ECS estimate based on 2000–09 data was actually the highest of those four ECS estimates, as a result of the top-of-the-range ocean heat uptake estimate used.

Figure 5 of the Met Office Report only gives ECS 5-95% bounds for Otto et al and CMIP3 and CMIP5 model ensembles. It is surprising that no other recent peer-reviewed observationally-based estimated ranges for ECS were shown. Examples are Aldrin et al 2012, Lewis 2013 and Masters 2013.

It is questionable whether valid estimates of probabilistic ranges can be constructed from an ensemble of simulations by different climate models. But even assuming so, Figure 5 of the Report indicates it does not show, as stated, the 5-95% ECS range for CMIP5 models given by Forster et al 2013. That range is 1.9–4.5°C, not 2.1–4.6°C. Moreover, the CMIP3 and CMIP5 central estimates are means, whereas those for Otto et al are medians.

A revised version of Figure 5 in the Met Office Report that does include several recent observationally-based studies, and corrects the CMIP5 ECS 5-95% range, is shown in Figure 3. All central estimates are medians to provide consistency. The range from palaeoclimate estimates (1.2–5.2°C) has been omitted since it is not on a comparable basis to the remaining ranges and palaeoclimate estimates involve great uncertainties.

It is evident from Figure 3 that a variety of recent observationally-based  studies all produce median estimates for ECS that are very substantially below the CMIP3 and CMIP5 model-based estimates, indicating that ECS in the more sensitive CMIP3 and CMIP5 models, at least, is likely to be out of line with reality.

Figure 3: Estimates of ECS from the same sources as in Figure 5 of the Met Office Report, with those from Aldrin et al 2012, Lewis 20138 and Masters 2013 added and the Palaeo range omitted.

The flasks span the 5–95 % uncertainty ranges for the estimates from each source; the black horizontal lines mark their 50th percentiles (medians). The width of the flask shows probability density, in each case for the shifted lognormal distribution corresponding to the applicable 5th, 50th and 95th percentiles. Since flasks from all sources therefore have equal area, probability is proportional to area between as well as within flasks. Short white horizontal bars show the central estimate for each source given in Figure 5 of the Report. The red bar in the CMIP5 column shows the ECS of the Met Office HadGEM2-ES model. The colours denote type of source, with blue showing estimates based on observations and salmon for model-only estimates.

Conclusions

The Met Office Report contained a number of misrepresentations of Otto et al 2013, misleading statements and outright mistakes. Correcting these and providing a fairer analysis reveals that, although some of the recent observationally-based estimates of TCR and ECS considered have upper bounds to their uncertainty ranges that are comparable to those for model-based estimates, they all give lower best estimates than model-based best estimates. Observationally-based median estimates for TCR and ECS are often comparable to the bottom of model-based uncertainty ranges.

The statement in the Met Office Report that “the upper ranges of TCR and ECS derived from extended observational records … are broadly consistent with the upper range from the latest generation of comprehensive climate models” has been shown to be questionable. The TCR and ECS characteristics of the Met Office flagship HadGEM2-ES model are of particular interest in this regard. None of the 95% bounds for TCR estimates from the observationally-based sources cited in the Report exceed the TCR of HadGEM2-ES. And HadGEM2-ES has an ECS that exceeds not only the 95% bound from Otto et al but also that from two other recent observationally-based studies. Moreover, both the TCR and the ECS of HadGEM2-ES exceed the 95% bounds derived not only from CMIP3 models but also from CMIP5 models other than HadGEM2-ES.

It has also been shown that estimates based on perturbing parameters in the Met Office HadCM3 model, as exemplified by Harris et al 2013, do not sample combinations of ECS and aerosol forcing located in the region most supported by several recent observationally-based studies, and so cannot be regarded as properly reflecting observational evidence. That is of concern since these same techniques, using the HadCM3 model, represent the basis on which the official UK Climate Projections were constructed.

Nicholas Lewis

JC note:  The complete document with footnotes and references can be downloaded here [Lewis UKMO], including Box 1.