Multilingual benchmark evaluates how well AI interprets clinical text and health records in nine languages

Researchers at Mass General Brigham recently developed BRIDGE, a multilingual benchmark that evaluates how well large ...
VentureBeat

LiveBench is an open LLM benchmark that uses contamination-free test data and objective scoring

Want smarter insights in your inbox? Sign up for our weekly newsletters to get only what matters to enterprise AI, data, and security leaders. Subscribe Now A team of Abacus.AI, New York University, ...
Crypto Briefing

MIT’s MeMo framework boosts LLM performance by 26% without retraining

MIT's MeMo framework trains a compact memory model that boosts LLM performance by up to 26.73% without retraining, with major implications for crypto AI agents.
InfoWorld

33 LLM metrics to watch closely

Look to these key metrics and benchmarks to evaluate the performance, capability, reliability, and safety of your AI models ...
EDN

The hidden bottleneck in LLM inference and the impact on MLPerf benchmarking

Recent frontier LLM inference benchmarks have highlighted a recurring pattern. GPU-based systems deliver outstanding ...
EDN

MLPerf and the rise of latency-aware LLM benchmarking

Here is a sneak peek at the evolution of the MLPerf benchmark and how generative AI forced a radical shift in AI hardware ...
TechCrunch

This LLM framework takes a first stab at benchmarking Big AI’s compliance with the EU AI Act

While most countries’ lawmakers are still discussing how to put guardrails around artificial intelligence, the European Union is ahead of the pack, having passed a risk-based framework for regulating ...
Semiconductor Engineering

Benchmark and Evaluation Framework For Characterizing LLM Performance In Formal Verification (UC Berkeley, Nvidia)

A new technical paper titled “FVEval: Understanding Language Model Capabilities in Formal Verification of Digital Hardware” was published by researchers at UC Berkeley and NVIDIA. “The remarkable ...
MSN on MSN

Anthropic's Fable 5 LLM tops performance benchmarks, has cyber safety rails

Anthropic just changed the AI landscape with the release of Claude Fable 5. This is not just another minor update or a slightly faster version of what you are already using. It represents a massive ...

Researchers say they trained a foundation model from scratch for about $1,500

Sapient researchers trained a 1B reasoning model on just 40B tokens — scoring competitively with 2B-7B models at a fraction ...