Neon Big Data Environment
The collection and comprehension of information is pivotal to our national security. Today, government agencies still struggle to digest enormous amounts of data and discover the crucially important, life-saving information that lies hidden within it. In addition to raw data taken straight from the source, emerging technology has led to the creation of machine-learning analytic modules that generate even more data in an effort to identify additional information within the raw data. To explore and analyze these massive, complex datasets, most analysts rely on multiple separate software tools that provide a disjointed user experience and often require expensive licenses.
Neon (along with its underlying library, NUCLEUS) was specifically made to help CACI teams overcome this challenge by streamlining the creation of robust, easy-to-use data visualization software that is customized for specific programs and facilitates the exploration and investigation of diverse, program-specific datasets in a single, unified, browser-based application.
Originating from CACI’s DARPA and IARPA work and now publicly available as free-and-open-source-software, Neon is already in use across multiple government programs to deliver critical insight into distinct datasets, including text, multimedia, geospatial, temporal, ontological, social, raw data, and machine learning analytic output.
CACI’s data scientists and software engineers have years of experience handling many terabytes of real-world data – raw and processed, static and streaming, structured and unstructured. Every day, they help CACI’s customers discover critical insight in their data by building new processing and analytic software, designing innovative and intuitive applications, and working closely with end-users to better understand their specific requirements and workflows.
CACI used these skills to create Neon and NUCLEUS, which together form a universal visualization software platform that enables CACI teams to rapidly prototype and develop user-friendly visualization interfaces made to meet the unique, mission-critical needs of individual customers.
CACI teams that use Neon and NUCLEUS can also take advantage of data visualization tools built by other teams, letting CACI developers quickly and easily share their innovative data visualizations, which others can then reuse as “off-the-shelf” components.
Neon and NUCLEUS give analysts and engineers access to the software they need to solve difficult problems involving big data, as well as a platform on which they can write or integrate their own custom tools.
Neon consists of three key elements:
The Neon dashboard is a front-end user interface that visualizes large, complex datasets in a flexible, browser-based environment. It offers a collection of different customizable visualization widgets (charts, feeds, graphs, maps, and more) that simultaneously display data from separate datastores (like SQL and Elasticsearch). Interaction with one visualization widget filters the data displayed by the others, allowing the user to discover crucial information within the datasets. Additionally, Neon has the capability to create and save personalized “dashboards” of widget configurations, import new data into existing datastores from uploaded text files, export data to CSV, and interact with external software modules (like machine-learning analytics) via REST.
NUCLEUS is a software library that provides search, filter, and data visualization components for JavaScript applications (including Neon). NUCLEUS lets CACI build program-specific user interfaces tailored to their customer’s unique data sources, technical infrastructures, and end-user workflows while leveraging common search, filter, and data visualization capabilities. NUCLEUS works with any JavaScript framework. With its simple “data in, filters out” design, developers can also integrate their own data visualizations with the NUCLEUS library, or enhance existing data visualizations with new features.
The NUCLEUS data server is a Java REST backend that handles direct connections to datastores along with additional data processing, clustering, and analysis. The data server’s architecture design also makes it easy to add new datastore adapters or data processing modules.