Aiming to give scientists better access to the vast quantity of science data it collects, NASA has created a Science Discovery Engine that leverages generative AI to deliver contextual results. Credit: NASA When you generate and collect as much data as the US National Aeronautics and Space Administration (NASA) does, finding just the right data set for a research project can be a problem. With seven operating centers, nine research facilities, and more than 18,000 staff, the agency continually generates an overwhelming amount of data, which it stores in more than 30 science data repositories across five topical areas — astrophysics, heliophysics, biological science, physical science, earth science, and planetary science. Overall, the agency houses more than 88,000 datasets and 715,000 documents across 128 data sources. Its earth science data alone is expected to hit 250 petabytes by 2025. In light of such complexity, scientists need more than just domain expertise to navigate through it all. “It requires researchers to know which repository to go to and what that repository has,” says Kaylin Bugbee, NASA data scientist at Marshall Space Flight Center in Huntsville, Ala. “You have to be both science literate and data literate.” In 2019, NASA’s Science Mission Directorate (SMD) released a report based on a series of interviews with scientists that made it clear those scientists needed a centralized search capability to help them find the data they needed. The SMD’s mission is to engage with the US science community, sponsor scientific research, and use aircraft, balloon, and spaceflight programs for investigations in Earth orbit, in the Solar System, and beyond. Recognizing that giving scientists and researchers access to its data was fundamental to its purpose, SMD developed its Open Source Science Initiative (OSSI) as a result of that report in an effort to make publicly funded scientific research transparent, inclusive, accessible, and reproducible. The mission of the OSSI: a commitment to the open sharing of software, data, and knowledge (including algorithms, papers, documents, and ancillary information) as early as possible in the scientific process. “It really came from the scientists and scientific community, and it also aligns with our broader SMD priority of enabling interdisciplinary science,” Bugbee says. “That’s where new discoveries are made.” To facilitate that mission, the agency is now turning to a combination of neural nets and generative AI to put those vast amounts of data at scientists’ fingertips. Restoring order A key element of OSSI is the Science Discovery Engine (SDE), a centralized search and discovery capability for all of NASA’s open science data and information, powered by Sinequa’s enterprise search platform. “Until the SDE was created, you couldn’t go to a single place to search for our open data and documentation,” Bugbee says. “Now it serves as a single search capability for our open science data.” New York-based Sinequa, which got its start more than two decades ago with a semantic search engine, focuses on leveraging AI and large language models (LLMs) to deliver contextual search information. It has since integrated Microsoft’s Azure OpenAI Service with its own neural search capabilities to power the platform. Specifically, Sinequa’s neural search capability uses a combination of keyword and vector search to discover information, while its GPT summarizes the information gathered into rapidly digestible and reusable formats. It also allows scientists to use natural language to ask deeper questions and refine the search or the response. The SDE understands nearly 9,000 different scientific terms, with that number expected to grow as the AI learns. Bugbee and her interdisciplinary team, which includes scientists with expertise in data stewardship and informatics, as well as developers and AI and ML experts, worked closely with stakeholders to understand their needs, and also with NASA’s Office of the CIO and Sinequa to build a proof of concept. “They helped us set up the environment we needed,” she explains. “We had to have an open capability, so we had some special architectural needs.” Bugbee says one of her team’s biggest challenges in getting everything up and running was how dispersed content was across the NASA ecosystem. Her team spent about a year trying to understand the information landscape, the data, and the metadata schemas. “All of the contextual information that really brings richness to the data — things like code and GitHub, or algorithm documentation that describes how the data was developed — that kind of content is spread over a number of web pages and it’s been an effort to curate and identify where all those things reside,” she says. Cleared for launch Bugbee is no stranger to data management and data stewardship. She cut her teeth in the field working to improve metadata quality in Data.gov and on President Obama’s Climate Data Initiative. But working on the SDE really drove home the importance of good curation workflow: the processes for principled and controlled data creation, maintenance, and management. “If I could go back in time, I’d have a more robust curation workflow built in from the beginning,” she says. “We kind of used the out-of-the-box approach to start and it worked for a time, but to really get the results we wanted, we needed that curation workflow.” While the SDE is still in beta, Bugbee says her team has received a great deal of positive feedback from scientists to date, and the plan is to deliver a more fully operational system later this year. Already the team has implemented a new user interface that allows users to filter by topics before they begin their search. Related content brandpost Sponsored by Randstad How CIOs can use internal mobility and redeployment for successful digital transformation The role of your people is key a factor in the success of digital transformation. Find out how redeployment and talent transition experts, Randstad RiseSmart, can help organizations at all stages of the digital transformation journey. By Neal Weinberg May 28, 2024 5 mins Digital Transformation opinion Navigating the Cybersecurity Threat Landscape in the UAE: Strategies for CISOs By Andrea Benito May 28, 2024 4 mins feature 10 most difficult-to-fill IT roles — and how to address the gap With AI, cybersecurity, and data analytics talent in short supply, IT leaders are upskilling, recruiting from the business, highlighting culture, and relying on contractors to secure the talent they need. By Stephanie Overby May 28, 2024 11 mins Hiring Staff Management feature Sun Country enhances customer experience with IT The low-cost regional airline turned to IT to develop a self-service e-voucher program aimed at easing the hassles that come with flight delays and cancellations. By Paula Rooney May 28, 2024 6 mins CIO 100 Travel and Hospitality Industry Digital Transformation PODCASTS VIDEOS RESOURCES EVENTS SUBSCRIBE TO OUR NEWSLETTER From our editors straight to your inbox Get started by entering your email address below. Please enter a valid email address Subscribe