New Release: Cobalt Version 0.3.9
Cobalt Version 0.3.9 is now available. You can install the latest version by running pip install --upgrade cobalt-ai
Features:
Group comparison in the UI now supports a choice of different statistical tests for numerical features.
In addition to the t-test, the Kolmogorov-Smirnov test and the Wilcoxon rank-sum test are supported, as well as a version of the t-test that uses permutation sampling to approximate the p-value instead of the t-distribution.
Workspace.get_group_neighbors() is a new method that finds a group af nearby neighbors of a given CobaltDataSubset. This neighborhood group can also be used as Group B in the group comparison UI.
The graph layout algorithm has been substantially improved and now presents cleaner, easier-to-read graphs. Some configuration options are available in cobalt.settings.
Evaluating LLM Hallucinations with BluelightAI Cobalt
Large language models are powerful tools for flexibly solving a wide variety of problems. But it’s surprisingly common for them to produce outputs that are untethered to reality. This phenomenon of hallucination is a major limiting factor for deploying LLMs in sensitive applications. If you want to deploy an LLM-based system in production, it’s important to understand the types of mistakes it may make, and use this knowledge to make decisions about what model to deploy and how to mitigate risks.
In this post, we’ll see how BluelightAI Cobalt can help understand a model’s tendency to hallucinate. In particular, we’ll identify certain types of inputs or questions where a model is more prone to make errors. Follow along in the Colab notebook!
How to Use BluelightAI Cobalt with Tabular Data
BluelightAI Cobalt is built to quickly give you deep insights into complex data. You may have seen examples where Cobalt quickly reveals something hidden in text or image data, leveraging the power of neural embedding models. But what about tabular data, the often-underappreciated workhorse of machine learning and data science tasks? Can Cobalt bring the power of TDA to understanding structured tabular datasets?
Yes! Using tabular data in Cobalt is easy and straightforward. We’ll show how to do this with a quick exploration of a simple tabular dataset from the UCI repository. This dataset consists of physiochemical data on around 6500 samples of different wines, together with quality ratings and a tag for whether the wine is red or white.