What the Intelligence Community Doesn’t Know is Hurting the US

“The U.S. intelligence community must invest in understanding its own business model to make data-driven decisions for the future,” argue authors Katrina Mulligan, Cipher Brief Expert Matt Olsen and Alexandra Schmitt from the Center for American Progress.

Katrina Mulligan, Managing Director, Center for American Progress

Katrina Mulligan is the managing director for National Security and International Policy at the Center for American Progress. Previously, she served as an attorney adviser and director for preparedness and response in the National Security Division at the U.S. Department of Justice, where she represented the department on a wide range of National Security Council (NSC) policy committees. In this role, she provided legal and policy advice on issues including foreign influence and election interference, immigration and watch listing, preparedness and response, and efforts related to biological, chemical, radiological, and nuclear weapons of mass destruction. She also contributed on a number of national strategies and operational plans, including the 2017 National Security Strategy.

Matthew Olsen, Chief Trust & Security Officer, Uber and former Director, NCTC

Matthew Olsen is the Chief Trust and Security Officer at Uber.  Olsen spent over twenty years working for the U.S. government, most recently as the Director of the National Counterterrorism Center (NCTC).  Prior to his appointment to NCTC, Olsen served as the General Counsel for the National Security Agency and Special Counsel to the Director of the Federal Bureau of Investigation.

Alexandra Schmitt, Policy Analyst, Center for American Progress

Alexandra Schmitt is a policy analyst on the National Security and International Policy team at the Center for American Progress. Prior to joining American Progress, she served as a research assistant at Harvard University’s Belfer Center for Science and International Affairs and the Center for Public Leadership.

The U.S. intelligence community (IC) is made up of 17 different intelligence agencies and subordinate organizations that work both separately and together to conduct intelligence activities to support U.S. foreign policy and national security. Altogether, the IC is a $81.5 billion-dollar enterprise. Yet, unlike Fortune 500 companies a fraction of its size, the IC does not focus on understanding or measuring the value of what it produces. As the IC confronts the transformational moment of the post-9/11 era, it will need better insights about itself to inform how it adapts. The IC can get those insights by leveraging business analytics—already widely used in the corporate world—to transform the way it performs its mission.

Today’s IC lacks foundational mechanisms and data to effectively meet the needs of its customers. Despite what the IC may know about other things, it tracks almost no data about those who consume intelligence, including the president and his or her national security team; policymakers; law enforcement; the military; and the U.S. Congress. This shortcoming means that the IC cannot observe customer-related patterns and trends, provide insights into what its customers value, or inform business-related decisions such as which collection platforms will yield the highest return on investment. Lacking comprehensive business data to inform its decision-making, the IC makes costly decisions that affect U.S. security, global governance, and the civil liberties and privacy of Americans and people around the world. And it does so without a clear understanding of which of its tools offer the most value for policymakers and the largest return on investment for American taxpayers. Although the IC has invested in advanced analytic tradecraft and big data techniques for other purposes, the IC has made comparatively few and only isolated investments in understanding the needs and behavior of the customers it serves.

As a result, the IC is flying blind. Notwithstanding promising efforts within individual agency silos, the IC does not comprehensively collect basic business data about the intelligence it produces, who is using it, and for what purposes. At the enterprise level, the IC does not methodically track how many policymakers access or read a particular intelligence product—let alone derive more granular insights about where they stopped reading and what sources of information they search for next. Moreover, the IC cannot accurately quantify the number of times a particular piece of raw reporting is accessed by intelligence analysts or cited in finished analytic products. If, for example, an analyst authors the most widely read product in a year, that analyst and her supervisor would generally not know it. If another analyst—or an entire team of analysts—were regularly reporting on a matter that resulted in no readership at all, there would be few—if any—ways for their leaders to uncover that fact or use it to justify a change in staffing priorities.

Despite tremendous progress in harnessing technological advances to serve the IC’s mission, today’s analysts are still receiving outdated anecdotal feedback rather than detailed data and analysis of the intelligence products they produce. Briefer notes such as “read with interest” are often cited in analyst performance reviews as evidence of impact—sometimes the only evidence an analyst or their supervisor might receive regarding a product’s mission impact.

At the macro level, the problem is even more pronounced. The IC has no reliable quantitative data about how frequently intelligence reports from major collection platforms are used and by which customers. The result is stark. The IC regularly invests millions—or even billions—of dollars in its capacity to do something without a meaningful way to measure the impact of its purchase.

Equally important, the IC cannot quantify how it responds to changes in intelligence priorities or which intelligence queries fall short of satisfying customer needs. It also lacks insights about which type of intelligence collection—human intelligence (HUMINT), signals intelligence (SIGINT), or geospatial intelligence (GEOINT)—is most useful in answering a given intelligence question. As a result, the IC cannot make informed decisions about how best to allocate existing collection and analytic capacity. Without those basic insights, the IC cannot derive more complex understandings about where it is over-invested or underinvested; which portfolios would benefit from additional collection or analytic resources; or when a particular collection platform or analytic product is not generating value for anyone.

While the benefits of better IC business analytics are clear, so are the anticipated obstacles to adopting an enterprise-wide solution. What little IC business data exist are stored inside individual department and agency silos; are not collected pursuant to uniform data standards; and are not permitted to inform IC-wide decisions. Moreover, any new system developed to collect and store information about IC reports or its customers would become a new target for exploitation by malicious actors. Then there is the IC’s culture, which favors the status quo and where parochial interests work against efforts to integrate and unify the IC—particularly given that the U.S. director of national intelligence (DNI) often faces resistance from other agencies on efforts to better integrate the community. But perhaps most challenging is the fact that even the cleanest and most comprehensive IC business data can never tell the whole story. The nature of the IC’s work defies simple quantitative measures of value. To be effective, IC business data must be supplemented by qualitative methods and metrics must be developed in consultation with subject matter experts who understand the limits of what the data show.

The IC is at a critical moment that will require it to adapt and transform. Rapid advances in technology have led to an explosion in the volume, velocity, and variety of data that the IC can access. The volume of this data will only increase. As former DNI Dan Coats said last year, the IC “must find innovative ways to exploit and establish relevance” of its information. Former Principal Deputy Director of National Intelligence Sue Gordon recently posed an existential question about the IC:

[I]f data exists in ridiculous abundance, and everyone – not just the national security community – understands its value, why are clarity, wisdom, insight, and answers to our most vexing national security and private sector challenges so elusive? And if it’s a world where the threats are to and through data, why do we keep being surprised at our adversaries’ and competitors’ attacks when we don’t invest in security?

The answer to Gordon’s question lies largely in the IC’s lagging methods for understanding its own usefulness. Although the IC’s technical achievements in the past 20 years have transformed the intelligence business, the IC has surprisingly little understanding of which intelligence tools and products actually inform policymakers and at what cost or return on investment. That needs to change.

Looking ahead, the IC must invest in its own business data analytics—or business intelligence—to make a stronger business case for the tools and capabilities it requires. In doing so, the IC will need data-driven metrics to understand where its insights are most useful to its customers. Rather than assume that more data will enhance its mission, the IC will need to be able to accurately describe the value it derives from the data it already has. It will need to lift up the promising efforts now housed within a few individual agencies and find ways to scale those efforts to provide IC-wide insights that are not possible within IC agency silos. It will also need to invest in people to analyze and derive insights from IC business data and empower them to create a demand signal for more and better data they can use. Equally important, in an increasingly constrained resource environment, the IC will need to harness the power of its own metrics and business data to inform the allocation of existing collection and analytic resources. Such insights will also be critical to ensuring the IC’s exquisite capabilities are invested where they will have maximum impact.

Defining business analytics for the IC

The term “IC business data” refers to quantitative data that provide insights about the IC’s customers for the purpose of improving IC decision-making. IC business data include information about intelligence customer behavior such as page views, attention time within a report, user journey—how the customer finds the report or product and where they go next—and volume of reports or articles accessed per visit. They may also combine that customer data to allow for identification of consumer trends and tailoring of future products to consumer preferences.

“IC business data analytics” refers to the systemic analysis of quantitative IC business data on various aspects of customer behavior aimed at informing day-to-day operations and longer-term strategic decisions.

“IC customers” refers to authorized consumers of classified intelligence information. These consumers include the president and his or her national security team, homeland security and law enforcement consumers, Congress, and can also include systems that depend on intelligence information as an input to perform tasks for other purposes.

This issue brief explores how the IC can harness the power of business analytics to improve the performance of its mission. It explores the discipline of business analytics; explains why the IC is late to adopt modern business intelligence practices; and makes the case that the IC should immediately invest in developing its own sophisticated business analytics standards and methodology. It then anticipates some problems the IC is likely to encounter—problems ranging from technical interoperability, secrecy and security, and cultural obstacles—and proposes solutions before making recommendations for how the IC can leverage business analytics to improve its value and performance in achieving U.S. national security goals.

Understanding the power of Business Analytics

Business analytics is an emerging discipline. As recently as 2018, there was no single, widely known, authoritative definition of the term. Generally speaking, business analytics refers to the skills, technologies, and practices for systematically applying qualitative, quantitative, and statistical computational tools and methods to analyze data, gain insights, and inform and support decision-making, typically to drive business planning. Gartner, one of the expert commercial businesses in the field, suggests business analytics is how tools such as data mining, predictive analytics, applied analytics, and statistics are delivered as a usable application or tool suitable for a business user in a given industry.

Business analytics is critically important across industry sectors. While it is used in health care, information technology, financial services, and education, it is most visible in the retail sector, where loyalty programs, customer tracking, and market research create insights that are used to make decisions about products, promotions, price, and distribution management. Predictive models use previous customer interactions to predict future events and are used in sales forecasting and merchandise planning and allocation. In fact, business analytics—and its related tools, business intelligence, and data visualization—is viewed as so central to modern business decision-making that executive education courses offering business analytics certifications are on offer at business schools across the country, from the University of California, Berkeley, to New York University and Wharton. Corporations often have entire divisions and high-level executives devoted to business analytics. AT&T has a senior vice president for big data while eBay has a vice president for global customer optimization and data. Other corporations often have roles for chief analytics officers or similar titles.

Effective business analytics is crucial for companies’ effective growth and development. It provides key evidence-based insights for decision-makers that can inform business decisions and strategy. It provides a more direct channel to consumers and customers, enabling an effective feedback loop and the ability to track customer behavior and use of products. It also helps leaders make tough decisions about what works and what does not and can provide a data-driven rationale for cutting a business or expanding into new areas.

The IC needs Data to inform its Decisions

Given the intelligence community’s exquisite technological capabilities and its progress in establishing an integrated technology platform, many assume—wrongly—that the IC must already have the business data it needs to assess performance and inform enterprise-level investment and resource allocation decisions. It does not. There are a number of technological and cultural obstacles that account for this gap, including the federated history of the IC’s information technology systems, discussed below; the IC’s reluctance to draw insights about its performance from imperfect measures; and the limits of the IC’s ability to innovate around business processes while it has pursued other, mission-focused technology infrastructure reform.

Historically, the main obstacle to collecting useful business data about the IC’s reports and products was the federated operating model of the IC’s information technology (IT) infrastructure. This legacy model was the result both of older information technology and bespoke technological solutions designed to offer more protection for highly classified intelligence information. Today, despite incremental progress integrating this infrastructure, each of the IC’s 17 intelligence agencies still has its own IT infrastructure and its own systems, applications, and data standards. This federated model permits each department and agency to derive some insights from its intelligence products and applications, but only within its own information silos. There are many well-documented shortcomings with this approach, but less often discussed is that it prevents anyone in the IC from having access to a complete picture of the reach and impact of intelligence reports and products. As a consequence, at a time when the rest of the world is using data to derive new insights and to evaluate business decisions and strategy, the IC has fallen behind, unable to acquire even the most basic information about the number of times reports are accessed and by whom.

Lacking comprehensive data to inform assessments of impact, the IC continues to rely on outdated anecdotal feedback mechanisms, including written descriptions of the interest and amount of time spent by a policymaker briefed on a particular intelligence report. Such descriptions are highly prized within the intelligence community, particularly at the CIA. Formal mechanisms for providing this anecdotal feedback have even emerged, including most notably within the president’s daily briefing staff, who regularly provide written reports about the reactions of policymakers to particular intelligence reports.

To some, it may be unsurprising that the IC derives meaning from such subtlety; after all, the IC is trained to do so. But there are other institutional and cultural reasons why the IC has not embraced metrics and data as key indicators of value and impact. Most obviously, intelligence reports are not commodities. While sales numbers and advertising clicks may be strong indicators of impact in other sectors, the utility of such measures within the IC is less clear and may even be misleading. That is because the IC exists to serve a different mission: provide insights and information to inform national security policymaking. That mission can sometimes be served if only a single person sees a given report or if five years from when a report was written it is used by a “targeter” or analyst to piece together a larger puzzle. Because the intelligence community rightly considers impact as more than the number of clicks, such measures can never on their own be sufficient to determine the value or impact of a particular report. The nature of the intelligence mission often defies simple numeric measures, and the consequences of making decisions based on flawed metrics could be severe.

This, unfortunately, is why many good efforts to assess the value of IC collection and analysis have failed. Others, such as the Office of the Director of National Intelligence’s (ODNI) Collection Platform Value Assessment, have had modest success because they primarily measure subjective judgments about the value of collection platforms. These subjective judgments allow platform owners to find silver linings in otherwise bleak data.

However, in rejecting quantitative measures as inherently flawed, the IC is missing the opportunity to find meaningful insights from imperfect sources—something the IC does exceptionally well in other contexts. Quantitative data may not be a perfect proxy for the value of IC collection and analysis, but that does not mean it is not a critical input. No single source will provide all of the information the IC needs to a degree of accuracy it would trust, but it is possible to find context and trends within the data to make informed decisions even from imperfect information. Some departments and agencies are beginning to see the value of the insights that imperfect data can provide. A few ambitious efforts have cropped up in some agencies and shown tremendous promise. But there are limits to what those efforts will be able to achieve without the ability to look across the IC to see where each department or agency is adding unique or disproportionate value. An agency will also be unable to determine when—in light of what other agencies are contributing—its capacity is over- or underinvested.

The bottom line is that, at the enterprise level, the IC develops and provides no systematic, all-inclusive, quantitative data about intelligence collection and analysis to inform leadership decision-making. As a result, the IC does not know how foundational business data—even flawed, imperfect data—can be useful to it. Instead, the IC has created complex systems and processes to compensate for the data it lacks. In almost all cases, these systems allow departments and agencies to maintain a significant degree of independence. Since there are no standard measures of value and impact across agencies, there are few opportunities to challenge agency decisions or question budget justifications and even fewer opportunities to identify and make informed trade-offs across departments and agencies. For many, this system works to their advantage and there are few incentives to pursue significant changes. As a result, bureaucratic inertia has worked to thwart past efforts to develop the kind of data this paper argues are needed.

Driving Change: The ODNI should lead the IC’s Data Transformation

The ODNI is the only entity with the mission and authority to mandate, manage, and oversee the collection and integration of comprehensive business data from across the intelligence community. Under Intelligence Community Directive (ICD) 101, the ODNI has the lead for the development and implementation of IC policy and standards, which would be an essential mechanism for requiring uniform business data across the IC. The ODNI is also responsible for the intelligence planning, programming, budget, and evaluation process, which would need to be revised to ingest the insights comprehensive business data could provide. In addition, the ODNI is well positioned to analyze enterprise business data and provide useful insights that IC elements can use. While individual IC elements can and do seek to understand the limited business data they currently produce, they are unable to look at data across IC departments and agencies to derive broader insights.

Yet, while the ODNI is the only entity with the ability to develop and issue IC policy or mandate business data standards across the IC, the ODNI is a comparatively small agency with few meaningful levers of power. The few DNIs who have chosen to use them during the agency’s brief 15-year history have employed those levers in pursuit of other compelling goals—most notably, former DNI James Clapper’s push to move the IC to a common technology platform. A successful ODNI-led effort to develop and issue IC policy and data standards around IC business data would require personal commitment and attention from a respected DNI to overcome institutional headwinds. No DNI to date has been willing to champion such an effort, which is why past efforts initiated below the DNI’s level have failed.

Unlocking the Value in the IC’s Business Data

There are several compelling reasons for the IC to adopt a new, comprehensive approach to collecting and analyzing information about its products. First, it makes smart business sense. In the corporate world, business data underpin almost every product launch, messaging effort, and major acquisition. Businesses derive insights from how many people open an email or click on an advertisement, how they scroll on a webpage, what search terms they use, and more. The IC should be able to derive similar insights about its customers, even if they must be narrowly scoped for legal or other reasons. Without it, the IC will be unable to make data-driven decisions about what collection platforms it needs, when and where to realign its analytic capacity, and make trade-offs between IC disciplines. Smart data analytics can help the IC test its assumptions and identify clear gaps before investing millions or billions in taxpayer dollars on a platform that may not be worth the investment in terms of output.

Second, data analytics can make the IC more effective in its mission. A comprehensive understanding of how and by whom IC products are accessed would provide valuable insights on what IC customers need. This would further improve the IC-policymaker feedback loop, a perennial challenge. Data analytics could also allow for instant feedback on a product’s utility, concision, and analysis, providing key insights to intelligence analysts that they may otherwise never hear regarding the quality of their products. Instant feedback mechanisms directly from intelligence customers could be tested and deployed, providing insights that are not currently possible. This could lead to a better understanding of which products policymakers use and how they use them and thus inform better intelligence gathering and analysis.

Finally, using data analytics can help the IC as a whole and its component organizations understand where they are adding unique value. At the enterprise level, data analytics can help the IC improve its overall performance by informing the allocation of scarce resources, aligning collection capacity where it can have the greatest impact, and by making tradeoffs across intelligence disciplines instead of within them. Each IC element has exquisite capabilities not available anywhere else. It is to each element’s benefit if those capabilities can be put to use where they can have the maximum impact. Comprehensive IC business data would allow the ODNI and IC elements to better identify areas of unnecessary redundancy as well as the comparative advantages of different IC elements, offices, and units. By focusing the IC’s mission—and limited resources—where it can be most effective for policymakers, every IC element can play to its unique strengths.

Industry examples and lessons that could apply to the U.S Intelligence Community

Examples from other industries can demonstrate the utility of data analytics for the intelligence community:

Customer relationship management (CRM) software: CRM software is the backbone of business analytics for many businesses today. At their simplest level, CRMs manage customer data for businesses, allowing for tracking and management of customers, data insights, and the ability to integrate data into other applications. The business analytics enabled by CRM software tend to fall into three areas:

  • Data from the past: This is a constantly updated database about relationships with customers such as points of contact, previous communication and documents, and key history such as when contracts were signed, or payments made.
  • Data to inform the present: CRMs often integrate insights and data from websites and marketing, potential customer leads for response, and real-time metrics on sales or deliveries.
  • Data to predict the future: Predictive analytics are often applied to the data from the past and present to generate constantly updating predictions about pipeline, sales, or other metrics for the future that are critical for business planning.

Journalism and media outlets: When traditional newsrooms and business practices couldn’t keep up with the changing media landscape, media outlets turned to business data analytics. With readers increasingly consuming news stories online, business data analytics allows news outlets to track customer engagement and retention, as well as derive other key insights to inform future coverage and reporting. To build a data analytics program, news outlets turned to programmers and data analysts to track things such as “unique” versus “repeat” visitors, the length of engagement with a piece, how readers found the story, and whether the reader jumped to other news stories or services from the same publication. The systematic analysis of this quantitative data helped inform decisions by editors on what to cover in the future, where to direct additional resources, and how to target ad revenue. The Washington Post’s transformation from a legacy newspaper to an innovative digital platform following its purchase by Amazon’s Jeff Bezos highlights how data analytics can transform a business: The changes instituted helped double web traffic, transformed how journalists worked, and created opportunities to generate $100 million per year from its content management system. For a struggling industry, the revenue opportunities and changes in operations represented a lifeline that was replicated by other papers: In the second quarter of 2020, The New York Times earned more revenue from digital subscribers and online ads than from the physical paper for the first time in its history.

Health care: The network of hospitals, primary care facilities, pharmacies, and insurers that makes up the U.S. health care system is also using business data analytics to improve efficiency. Health care companies are creating technological solutions that allow interoperability across IT systems of different parts of the health care system, allowing a patient’s information to be shared safely and securely from a doctor to a surgeon to a pharmacist. The goal of these systems is for each patient’s health records to be both secure and usable in a way that enables better health outcomes, creates cost savings through efficiency and elimination of administrative waste, and improves the quality of care for the patient. Public-sector players, such as the U.S. Centers for Medicare and Medicaid Services, have invested in teams designed to manage and harness data gathered in order to conduct enterprise-wide analysis on how to better serve patient populations while finding cost savings and driving reform in the industry. The health care industry faces and has successfully dealt with many obstacles related to privacy, data security, and interoperability—similar to those the IC will face—but their example illustrates how some of those obstacles can be thoughtfully overcome.

Anticipated Obstacles to the IC’s full embrace of Business Analytics

While the benefits of better intelligence community data analytics are clear, so are the anticipated obstacles to adopting an enterprise-wide solution. They include:

  • Interoperability: While some agencies have sought to derive insights from the business data they do individually possess, there is currently no centralized mechanism for collecting business data across the IC and therefore no comprehensive approach to accessing or analyzing it. What little IC business data exist are stored inside individual department and agency silos, are not collected pursuant to uniform data standards, and are not permitted by agency leaders to inform IC-wide decisions. ODNI efforts to extract meaningful insights from such data are often rejected by departments and agencies that argue that flaws in the data make them unusable to inform decision-making. Those same voices often reject efforts to improve the quality of the data, creating a Catch-22 that serves to reinforce the status quo. However, broad implementation of the IC IT Enterprise (ICITE)—a new IT architecture that connects intelligence community agencies on one common platform—provides new opportunities to set and enforce uniform data standards for the purpose of creating a high-quality, tailored set of IC business data aligned with organizational priorities.
  • Customer behavior: The best and most insightful information about the value of IC products comes from the IC’s customers. Yet, the IC’s most important customers are also the government’s busiest officials. Any feedback tool, no matter how effortlessly intended its design, will have to contend with the challenge of human behavior. Past efforts to capture customer feedback have failed because customers did not take the time to provide it—a hurdle that any future efforts will likely encounter. To overcome this obstacle, the IC should learn what it can from private industry, piloting new and creative efforts to facilitate feedback with minimal friction.
  • Secrecy and security: Any new system or process for collecting and storing information about IC reports and customer behaviors inherently becomes a new target for exploitation by malicious actors. While metadata about IC customer behaviors may not be classified, it must certainly be protected. IC officials have legitimate reasons for concern over the security and accessibility of the business data they collect. There may also be legal issues to overcome regarding the collection of any information—even if it is only metadata—about IC customers, who are nearly all U.S. persons. Yet, those obstacles can and must be overcome, as they have been in other areas of the IC’s mission.
  • Culture: As previously noted, the absence of comprehensive IC business data favors the status quo—and some departments and agencies prefer it that way. Parochial interests have long worked against efforts to integrate and unify the intelligence community, as every past DNI can attest. An effort that aims specifically to aggregate actionable information about IC performance and put it directly into the hands of the DNI is almost certain to face strong objections from within and from outside the IC, some of them legitimate. Some agencies will reasonably fear that imperfect data will be used to make harmful decisions with negative mission impacts. A skeptical Congress may also resist more ODNI bureaucracy that does not contribute directly to core IC missions. These cultural barriers could significantly impede efforts to implement uniform data standards and accept the use of IC business data for strategic decision-making.
  • Data do not equal value: Perhaps most challenging, the IC has always resisted external efforts to assess the “value” of the intelligence it provides. It is often said that data never tell the full story, and it is true that quantitative analysis will always have to be supplemented by qualitative judgments in order to tell the full story about any given data metric or trend. Any IC business analytics program would need to carefully consider the difficulties involved in defining and measuring the “value” of an intelligence report or product and account for certain realities that are unique to the IC, such as a single report that answered a key intelligence question five years after it was collected or a single analytic product that informs the thinking of the one policymaker who needed it. An old report that was never accessed may answer a critical question years later, and in contrast, a product with overwhelming readership may never meaningfully contribute to a policy decision. To be effective, IC business data must never be understood as telling the whole story, and IC business data analytics must be supplemented by qualitative methods and creative metrics designed to address these and other scenarios.

Recommendations

Make systematic, high-quality business data a top priority for the IC. The DNI should champion an IC-wide effort to develop systematic, high-quality IC business data, making them a signature priority and helping overcome the bureaucratic hurdles that have thwarted past efforts. The DNI should view IC business data as foundational to understanding the IC’s performance, as a key lever for influencing department and agency behavior, and as a critical enabler for making prudent decision-making in a constrained resource environment. Prioritizing this effort at the DNI level should signal it as a high priority for the rest of the community.

The DNI should lift up existing efforts to develop and use IC business data to show senior leaders across the IC the power of the insights such data can provide and should establish attractive incentives to encourage buy-in. For example, the DNI could host a competition or forum that showcases individual agency efforts to harness IC business data and allow the winner to present to the IC Deputy Executive Committee or the IC Executive Committee.

Create a set of uniform metrics for IC business data. The DNI should direct the IC chief data officer to lead a whole-of-IC effort to identify a set of uniform metrics for IC business data. This major effort must involve senior level participation from all IC agencies and should draw from the lessons that have been learned by individual department and agency business data efforts. It will need to consider complex challenges, identifying useful metrics to account for intelligence “assists”—enabling tools or platforms that are several hops away from a finished report or product. The metrics team should be supported by an interagency lawyers’ group to consider legal issues as they arise.

Create a congressional demand signal for IC business data. Relevant congressional oversight committees should request briefings from IC leaders on how the IC is leveraging quantitative data that provide insights about its customers for the purpose of improving IC management and decision-making. Congress should also ensure that efforts to derive insights into IC customers comply with civil liberties and privacy laws.

To close current gaps, Congress should dedicate funding to establish an IC business analytics unit at the ODNI tasked with identifying and closing gaps in existing IC business data; improving the quality of quantitative business data across the IC; and providing regular reports to Congress about how IC business data are being used to inform IC decision-making.

Enlist the support of the National Security Council (NSC). The NSC, through the senior director for Intelligence Programs, should support ODNI efforts to improve IC business data collection and quality wherever possible. In addition to requesting regular updates, the NSC staff should resolve policy disputes through the interagency policy committee process. Support from the NSC should also create disincentives for IC agencies to seek different answers from the White House than they receive from the DNI.

Require the use of uniform IC business data standards across the IC. The DNI should develop and publish an intelligence community directive requiring the use of uniform IC business data standards common to all tools and platforms on the ICITE that would also be required to capture that data. The ICD should require departments and agencies to provide the ODNI with full, unrestricted access to the business data they collect.

Create a unit to derive insights from IC business data. The ODNI should create a unit tasked with deriving insights from IC business data. This unit should include developers, data analysts, data visualization experts, subject matter experts, and other staff deemed necessary to assist the mission and should draw on expertise from the private sector, particularly regarding best practices for harnessing and analyzing business data. It should develop new tools to capture, incorporate, and analyze IC business data, produce several formal product lines making their analysis useful to senior IC leaders, and should also respond to inquiries and taskings from other IC elements. Its formal product lines should include custom reports to the leadership of each IC department and agency with insights about their highest-value reports and products, broadly defined. This unit should be advised by a group of subject matter experts from across the IC to inform how the data are used and presented. IC business data should also be reviewed as part of the annual State of the Mission briefings conducted by the DNI with the heads of each IC element.

Create a standing interagency group of IC business data experts to innovate around intelligence customer experience solutions. The IC chief data officer should meet regularly with individual department and agency business analytics units to share insights and generate ideas about how to create different intelligence customer experience solutions based on what is learned. Examples—for illustrative purposes only—might include:

  • A policymaker who never accesses intelligence might get an email with links to key articles tailored to his or her portfolio or might be offered a daily intelligence briefer.
  • A policymaker who runs a string of search requests might receive immediate feedback requests to ascertain if they found what they were looking for.
  • Systems reliant on feeds of intelligence data might trigger evaluations of data quality and outreach about improvements.

Update the IC’s data strategy. The ODNI should update the 2017 Intelligence Community Information Environment Data Strategy to include IC business analytics as a key enabler of mission success.

Recalibrate the intelligence planning process to incorporate insights from business data analytics. Once IC business data are available and the data quality has been refined, the ODNI should recalibrate the Intelligence Program Budget Submission process so that insights derived from IC business data can inform the IC budget and acquisition processes.

Conclusion

Notwithstanding its current limitations, the U.S. intelligence community makes the best decisions it can based on imperfect information. But it cannot continue to leverage partial, anecdotal information to inform decisions about the IC’s future. Before the IC continues down a path to “store, process, exploit, and manage” the explosive growth of its data, it must first put in place foundational systems to collect basic business data about the intelligence it produces, who is using it, and for what purposes. Put another way, before the IC invests billions of dollars in new and existing tools and technologies, it should first invest in understanding its own business model—collecting and using data about the reach and impact of its work to make objective and data-driven decisions about where to invest scarce resources in the future.

Read the full report with endnotes at The Center for American Progress

Read more expert-driven national security analysis, insight and perspective in The Cipher Brief


Related Articles

Search

Close