aiXplain v2.4.0: Metric Changes, Text Normalization, and Autosaving Pipelines

We are delighted to announce the release of our platform’s latest version 2.4.0 with changes to Benchmark, Design, Metrics, and more. You can check the list of all the changes in the release notes, or you can visit the platform and try them out for yourselves.

Comparing Apples to Apples

Benchmark is a tool that provides an in-depth analysis and comparison for the performance of AI models. You select the test dataset, models that you wish to evaluate, and the metrics you want to use for evaluation; then a comprehensive and accurate report is generated with actionable insights. However, text datasets come with all sorts of abnormalities in text that each model handles differently. For example, numbers, URLs, letter casing, and diacritics are some of the many abnormalities that will be processed differently by each model and the datasets’ ground truth doesn’t always cover all of these differences.

As we take pride in making Benchmark a reliable and accurate model performance evaluation tool, we added the option to select what text normalization configurations you would like for each metric. This means that you can pre-process the dataset and compare apples to apples since you’re stripping out the abnormalities which are handled differently by each model. This change in Benchmark comes with a change to Metrics which now treats and displays them similar to models, showcasing their price, supplier, input, and output on the asset card.

Ensuring Work Doesn’t Get Lost

aiXplain’s unique experience allows members to seamlessly switch between their different teams and experiences. As switching between experiences on the same team maintains the data persistence, for example you can simultaneously create a complex AI pipeline on Design and fine-tune an AI model in FineTune without losing progress on any of the experiences. However, this data persistence shouldn’t occur as you are switching between different teams. So we added warnings if you attempt to switch teams while having any unsaved work. Additionally, for building AI pipelines on Design we now added the autosave functionality that you can choose to toggle on or off. Autosave will only be possible after a pipeline is initially saved. Future iterations of this feature will allow auto-saving for incomplete (or draft) pipelines.

A Step Forward in Product Assistance

We are striving to make the onboarding experience for our new members as seamless as possible. As soon as you log into aiXplain, you will be welcomed by a guide that walks you through the essentials of aiXplain. These guides will be integrated with all the other tools and experiences as we get more feedback on them. This release also added UI changes to Benchmark and Design which makes the experiences more intuitive.

Our AI community will be pleased to hear that this release introduces SDK support for aiXplain’s pre-trained model fine-tuning tool, FineTune. This puts us closer to achieving more support to our community who has been enjoying our SDK so far. Future releases will highlight more crucial new features in the SDK that will answer the questions that we’re asked about getting models into aiXplain. No spoilers… Wink.