Taking it Online: Monitoring and Evaluating for U.S. Foreign Policy Outcomes Post COP26
December 7, 2021 ~
COP26 dominated the headlines across the globe last month and now that the dust has settled a bit more, let us consider what U.S. foreign policy outcomes resulted from this highly anticipated 26th Conference of the Parties. In my view, it is too soon to assess whether the climate summit was a success or a failure one month post summit. This question is largely dependent on where you sit and what metrics may have been established to indicate success or failure. The parties involved in formal negotiations are national governments; however, attendees from fossil fuel industries, trade associations and civil society organizations participated as observers. Each of these parties have their own interests and views as to the degree of success or failure.
Pledges and commitments to this, that and the other will be for naught if there isn’t transparency, accountability and learning and if there aren’t mechanisms in place for monitoring and evaluation (M&E) of these pledges and commitments. Fortunately, for climate change and follow up efforts at COP26, there is no need to reinvent the wheel and one place that we can turn to is work already done for the UN Sustainable Development Goals (SDGs) under previous Administrations to build and expand data collected and progress monitored for the United States. In fact, SDG #13 is devoted to climate action and lessons may be learned from how the United States tracked progress made towards the SDGs for COP26. While the data and information provided for climate action is dated having last been updated in July 2018, the United States should not hesitate to build on those lessons learned to build a better framework for goal setting, monitoring and reporting.
From a U.S. foreign policy perspective, there is potential to look to steps taken on Global Goals and progress made on specific indicators and targets to help evaluate the effectiveness of U.S. diplomacy, advocacy and assistance in achieving COP26 outcomes. Given the diverse efforts to track U.S. progress for the SDGs, one might ask if the efforts undertaken to collect data and track progress against goals is commensurate with the use made of the data. This has been difficult to answer. With time, the data could become more helpful in informing and refining U.S. as well as global policies and actions. In the near horizon, the United Nations and its agencies stand to benefit from tracking the trajectories in the SDG indicators to justify their existence. In the far horizon, the global community stands to benefit, but there are many barriers to overcome along the way, including existing institutional barriers, differing country level priorities, political and cultural differences, and technical capacity. Tracking the data to inform transparency and accountability efforts also has the potential to produce unexpected consequences, and even delegitimize some parties involved.
Using the SDG indicators to hold the United States and other countries accountable has been fraught with challenges. First, will shaming the U.S. in comparison to its peer nations produce useful action? In her book Scorecard Diplomacy (2017), political scientist Judith Kelley asserts that public grades are potent symbols that can evoke countries’ concerns about their reputations and motivate them to address problems. In reality, however, the shame may be heard or felt more by the developing economies and not developed economies like the United States.
Second, the credibility of the data used to measure SDG indicators is central to bolstering their use, but vulnerable. Monitoring the SDG indicators is needed to measure progress in meeting goals, but often there are many country-specific contextual factors that must be taken into account to make sense of the data on the conditions that are measured. The SDG indicators tend to be outcomes, not outputs, as they should be, but most are rather vaguely stated outcomes. The validity and reliability of the data collected undoubtedly vary across the United States, as well as across countries – making cross-national comparisons tricky. Even within the English language, there are issues of measurement reliability across disciplines, so one may imagine the reliability concerns of translation into multiple languages, where words do not fully capture concepts. In addition, within the U.S. federal system there is variation in the periodicity of reporting across jurisdictions that also affects the reliability of the data. Political will, as well as cross-organizational coordination and resources are needed to maintain the quality of the data. And trust in the credibility of the data and the judgements made in cross-national comparisons must be sustained.
As an example, below is a partial profile overview of the United States taken from the Sustainable Development Report for 2021 and provides a sense of how the United States compares to 164 other countries’ performance profiled in the report. Specific to SDG13 on climate action, the United States is designated as “Major challenges remain,” which translates to “Score stagnating or increasing at less than 50% of required rate.”
Third, the very important question of who is “at the table” needs to be addressed in both selecting indicators and interpreting their meaning. In the United States there has been great variation in terms of which federal agencies and which national, state and local level actors have been involved in selecting and monitoring the SDG indicators. While there may be some collaboration and coordination that takes place that is not visible to the public, there seems to have been a notable lack of interagency collaboration, as well as intergovernmental coordination, in selecting and monitoring the indicators. The sustainability of efforts, domestic and international, are also tied to the short-term horizon of term-bound, elected officials. While some prominent NGOs in the United States are using the SDG indictors to rate or rank local governments to mobilize resources to address deficiencies, they lack power to initiate governmental action.
At the national level in the U.S. government there has not been a central, focal authority with power or will to pressure federal agencies or states or local governments to make progress on specific goals. Under the Biden Administration, we are seeing a change to prioritize climate change and a whole of government approach that began on day 1 with the appointment of first Special Presidential Envoy for Climate John Kerry and first-ever Principal to sit on the National Security Council. In May, President Biden restored Mike Kuperberg as Executive Director of the U.S. Global Change Research Program (USGCRP), who is to lead efforts to produce the Fifth National Climate Assessment. By law, the U.S. government must complete a national climate assessment every four years and the most recent such report was finalized in 2018. Last month, as COP26 negotiations were going on, Denice Ross was named U.S. Chief Data Scientist. There is also hope for real progress in the context of Secretary of State Blinken’s announcement at the end of October to modernize American diplomacy. In his announcement, Blinken emphasized five pillars with the fourth pillar to modernize technological, communications and analytic capabilities. Together, these announcements reinforce the Biden Administration’s efforts to use evidence to inform policymaking and serve as critical steps toward carrying out monitoring and potentially evaluating for U.S. foreign policy outcomes.
Another important question that remains for the U.S. and other countries is how policymakers will use insights gained from tracking the SDG indicators to address the important issues they have framed. Governments at all levels could use these data proactively to inform the development of policies, but resourced enforcement and evaluation of these policies must also be part of the comprehensive package to truly address the complex issues of our time. The work already done on tracking the SDGs may lay the foundation for an improved framework for monitoring and evaluation with COP26. Future evaluation work could measure the efficacy of cross-national efforts to use the SDG and COP26 imperative to plan and execute collective action with both short- and long-term goals in mind.
Change and momentum, however, need not be solely within the domain of government. To achieve the very ambitious SDGs, including COP26, all levels of government, and all three sectors need to demonstrate commitment and take action to orchestrate collaborative joint action. (Fukuda-Parr, 2016). Additionally, the consistent and persistent day-to-day actions that we as individuals all take continues to hold potential and promise and will fundamentally affect long-term outcomes and transformational change.