SiMa.ai says Palette 3.0 makes LLMs on its chips up to 3.5x faster

Palette 3.0 uses speculative decoding to speed up language models on SiMa.ai’s Modalix chips. SiMa.ai’s own white paper shows the 3.5x peak came on one model, under favourable test conditions.


SiMa.ai says Palette 3.0 makes LLMs on its chips up to 3.5x faster
Image Credits Credit: SiMa.ai

SiMa.ai has released Palette 3.0, the next version of the software for its Modalix AI chips. The San Jose start-up says language models on the chip now generate text up to 3.5 times faster. Power draw stays broadly the same.

The gain comes from speculative decoding. A small, fast draft model proposes several tokens at once, and the main model checks them together. Palette 3.0 uses three such methods: EAGLE3, DFlash, and Gemma 4 multi-token prediction.

“What fundamentally changes for drone and robotics teams is starting from a working autonomy system instead of a bare board,” Manuel López Roldán, who leads agentic software and generative AI at SiMa.ai, told TNW.

Where the 3.5x comes from

The figure compares speculative decoding with standard decoding of the same model on the same chip. That comes from a SiMa.ai white paper published with the release. The best result came from Alibaba’s Qwen3.5 9B with DFlash. It went from 11.98 to 41.75 tokens per second, at about 7.5W either way.

Energy per token fell from 0.627 to 0.182 joules in that test. Across four models, the speed-up ranged from 2.21 to 3.49 times. SiMa.ai ran the tests on five fixed prompts, including a maths proof and a Python script. It says the goal was to measure gains under favourable conditions.

A second test was closer to a real product. An in-car assistant had to turn commands such as “I’m cold” into one correct function call. All 900 measured requests returned the right call. Speculative decoding made them 1.35 to 2.06 times faster. That gain is smaller, because the method does not speed up processing of the prompt.

What else changes

Palette 1.0 ran vision models on SiMa.ai’s first chip. Palette 2.0 added generative AI on Modalix in December 2025. In June, Palette Neat added an AI agent that writes and deploys applications from plain-language requests.

Palette 3.0 is about running models on the device, López Roldán said. It sends several models straight to the chip’s accelerator at once, with no background service in between. It can also run separate Docker containers on the chip. It also supports large mixture-of-experts models, such as OpenAI’s GPT-OSS-20B, on the 16GB Modalix module.

For drones, SiMa.ai offers a reference platform with segmentation of 1080p video at 30 frames a second and visual odometry in about 38 milliseconds, López Roldán said. Robot software such as ROS 2 and Nav2 is ready to install. Indian engineering firm Mistral Solutions is a partner on this release.

The code is open source on GitHub, with documentation in SiMa.ai’s Developer Center. All performance figures come from SiMa.ai’s own tests.

SiMa.ai raised $150m at a $1.45bn valuation in September. Other edge AI chipmakers have also drawn money this year. Analog Devices agreed to buy Alif Semiconductor, and Syntiant filed for an IPO.

Get the TNW newsletter

Get the most important tech news in your inbox each week.

Thanks! Check your inbox to confirm your subscription.

By subscribing you agree to our Privacy Statement.

Published
Back to top