Block's Dessa Learned to Talk Without Text

Block, the payments company led by Jack Dorsey, on Oct. 23 unveiled Dessa, a research-preview voice model that it says learned to hold a conversation from recorded human speech alone, with no text and no pretrained language model underneath. "You learned to talk by listening. So did Dessa," the company's announcement page says.

Key takeaways

• Dessa is a full-duplex, speech-to-speech model: it listens and talks at the same time, and audio goes in and audio comes out with no transcription step.
• Block says it trained the model from scratch on human conversation using a single NVIDIA GB200 NVL72 system, and that it runs locally on a MacBook Pro M4 Max or better.
• There are no published benchmarks, parameter counts, weights or pricing; early access goes first to a small group of AI safety researchers.

What is Block's Dessa speech model?

Most voice assistants work as a relay: speech is transcribed, a language model writes a reply, and a synthesizer reads it aloud. "It turns your voice into text and throws away an incredible amount of information text alone can't carry," the announcement says.

Dessa skips that relay. Block says it natively hears and produces emotion, sarcasm, laughter and sighs, and can interject "got it!" while the other person is still talking. The company says the model "learned language from hundreds of times fewer words" than large language models, but it did not give a figure.

How Dessa was trained and where it runs

Block says it "figured out how to train Dessa on only one NVIDIA GB200 NVL72," a single rack-scale system. It says Dessa runs faster than real time on a MacBook Pro with an M4 Max chip or better.

Those are vendor claims. Block has not disclosed the model's size, the hours of audio it trained on, its latency in milliseconds, or any benchmark scores, and no independent evaluation exists yet. Even its headline claim is hedged with "as far as we know."

Block posted demo recordings, including a conversation between Dessa and OpenAI's GPT-Live-1. The page notes that OpenAI is not affiliated with Block and has not endorsed Dessa. The Verge, reviewing the samples, noted that the model called one partner "Ronaldo" and drifted from Christopher Nolan's films to Bryan Singer's Superman movies.

How Dessa compares with other full-duplex voice models

Full-duplex voice AI is not new; the training recipe is what Block says is different.

Kyutai's Moshi paper described its system as "the first full-duplex spoken large language model," but Moshi sits on top of Helium, a text model pretrained on 2.1 trillion tokens. OpenAI opened its own full-duplex model to developers in September (OpenAI opens its full-duplex voice model to developers).

Context: from voice clones to a voice lab

The team carries history. Dessa was a Toronto machine learning startup whose staff joined Square, now Block, in February 2020, according to a Square press release that highlighted its deepfake-detection work. In May 2019 the startup drew attention for synthesizing comedian Joe Rogan's voice, Axios reported.

Block describes the group as a very small research lab led by Ragavan Thurairatnam that reports directly to Dorsey. It gives three reasons for the project: "curiosity (aka fun), better understanding between humans and AI, and safety."

What's next

Block says launch links are "coming soon" but has not named a date, price or license. For now it is giving a small group of AI safety researchers early access. "A model that learned to talk from sound alone is new territory, and we want outside eyes on it," the page says. Independent tests will show whether a speech-only recipe can compete with text-trained voice models.

FAQ

Can I use Block's Dessa today?

No. Dessa is a research preview. Block is giving early access to a small group of AI safety researchers and says broader launch links are coming soon.

What does "trained only on speech" mean?

Block says Dessa learned from recorded human conversation, from scratch, without any text data or pretrained language model. By contrast, Kyutai's Moshi is built on top of a text model.

What hardware does Dessa need?

Block says it runs locally, faster than real time, on a MacBook Pro M4 Max or better. Training used one NVIDIA GB200 NVL72.

References

• Dessa — Block
• Moshi: a speech-text foundation model for real-time dialogue — Kyutai, arXiv
• Welcoming the Dessa Team to Square — Square (Feb. 7, 2020)
• Joe Rogan finds his synthetic voice — Axios (May 2019)