Decoding Complex AI Models: Purdue Researchers Transform Deep Learning Predictions into Topological Maps

The highly parameterized nature of complex prediction models makes describing and interpreting the prediction strategies difficult. Researchers have introduced a novel approach using topological data analysis (TDA), to solve the issue. These models, including machine learning, neural networks, and AI models, have become standard tools in various scientific fields but are often difficult to interpret due to their extensive parameterization.

The researchers from Purdue University recognized the need for a tool that could transform these intricate models into a more understandable format. They leveraged TDA to construct Reeb networks, providing a topological view that facilitates the interpretation of prediction strategies. The method was applied to various domains, showcasing its scalability across large datasets.

The proposed Reeb networks are essentially discretizations of topological structures, allowing for the visualization of prediction landscapes. Each node in the Reeb network represents a local simplification of the prediction space, computed as a cluster of data points with similar predictions. Nodes are connected based on shared data points, revealing informative relationships between predictions and training data.

One significant application of this approach is in detecting labeling errors in training data. The Reeb networks proved effective in identifying ambiguous regions or prediction boundaries, guiding further investigation into potential errors. The method also demonstrated utility in understanding generalization in image classification and inspecting predictions related to pathogenic mutations in the BRCA1 gene.

Comparisons were drawn with widely used visualization techniques such as tSNE and UMAP, highlighting the Reeb networks’ ability to provide more information about the boundaries between predictions and relationships between training data and predictions.

The construction of Reeb networks involves prerequisites such as a large set of data points with unknown labels, known relationships among data points, and a real-valued guide to each predicted value. The researchers employed a recursive splitting and merging procedure called GTDA (graph-based TDA) to build the Reeb net from the original data points and graph. The method is scalable, as demonstrated by its analysis of 1.3 million images in ImageNet.

In practical applications, the Reeb network framework was applied to a graph neural network predicting product types on Amazon based on reviews. It revealed key ambiguities in product categories, emphasizing the limitations of prediction accuracy and suggesting the need for label improvements. Similar insights were gained when applying the framework to a pretrained ResNet50 model on the Imagenet dataset, providing a visual taxonomy of images and uncovering ground truth labeling errors.

The researchers also showcased the application of Reeb networks in understanding predictions related to malignant gene mutations, particularly in the BRCA1 gene. The networks highlighted localized components in the DNA sequence and their mapping to secondary structures, aiding interpretation.

In conclusion, the researchers anticipate that topological inspection techniques, such as Reeb networks, will play a crucial role in translating complex prediction models into actionable human-level insights. The method’s ability to identify issues from labeling errors to protein structure suggests its broad applicability and potential as an early diagnostic tool for prediction models.

Check out the Paper and Github. All credit for this research goes to the researchers of this project. Also, don’t forget to join our 33k+ ML SubReddit, 41k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.

If you like our work, you will love our newsletter..

Pragati Jhunjhunwala is a consulting intern at MarktechPost. She is currently pursuing her B.Tech from the Indian Institute of Technology(IIT), Kharagpur. She is a tech enthusiast and has a keen interest in the scope of software and data science applications. She is always reading about the developments in different field of AI and ML.