Microsoft Open-Sources Cutting-Edge Voice AI Family VibeVoice: Processes 90-Minute Multi-Speaker Conversations in One Go, Rapidly Gains 27K Stars on GitHub

marsbitОпубліковано о 2026-03-30Востаннє оновлено о 2026-03-30

Анотація

Microsoft has open-sourced VibeVoice, a cutting-edge family of speech AI models for Automatic Speech Recognition (ASR) and Text-to-Speech (TTS). The project, gaining 27K stars on GitHub, offers powerful long-audio processing, multi-speaker dialogue generation, and real-time capabilities under an MIT license for local deployment. Key models include: - **VibeVoice-ASR-7B**: Processes up to 60 minutes of audio, outputs structured transcriptions with speaker identification, timestamps, and supports over 50 languages. - **VibeVoice-TTS-1.5B**: Generates expressive, 90-minute multi-speaker (up to 4 voices) conversations with natural flow and emotional nuance. - **VibeVoice-Realtime-0.5B**: Enables real-time TTS with ~300ms latency for interactive applications like voice assistants. The framework addresses limitations in long-sequence processing, speaker consistency, and naturalness. It includes safety features like audio watermarking and has sparked community-developed tools (e.g., a voice input method). Available on GitHub and Hugging Face, VibeVoice aims to advance innovation in content creation, accessibility, and voice interaction.

Microsoft recently open-sourced a cutting-edge voice AI model family named VibeVoice, which encompasses capabilities such as automatic speech recognition (ASR) and text-to-speech (TTS). The project has quickly garnered attention in the developer community due to its powerful long-audio processing, multi-speaker natural conversation generation, and real-time low-latency features. It has already gained approximately 27K Stars on GitHub.

As an open-source research framework, VibeVoice uses the MIT license, supports local deployment, requires no cloud subscription fees, and aims to promote collaboration and innovation in the field of speech synthesis. The model family mainly includes three core members, each with its own focus, collectively addressing the pain points of traditional voice AI in long-sequence processing, speaker consistency, and natural fluency.

VibeVoice-ASR-7B: A Structured Speech-to-Text Tool for Up to 60 Minutes

VibeVoice-ASR-7B is a unified speech-to-text model capable of processing audio files up to 60 minutes long in one go, directly outputting structured transcription results. The output includes not only "who is speaking" (speaker identification) and "when they speak" (precise timestamps), but also "what was said" (detailed content), and supports custom hotwords to effectively improve the recognition accuracy of proper nouns or technical terms. The model supports over 50 languages and is suitable for complex scenarios like long meeting recordings and podcast transcriptions.

Community developers have already built practical tools based on this model, such as a voice input method named Vibing, which supports macOS and Windows platforms. User feedback indicates that its recognition speed and accuracy perform well, significantly improving daily voice input efficiency.

VibeVoice-TTS-1.5B: Expressive Speech Generation for 90-Minute Multi-Speaker Content

VibeVoice-TTS-1.5B is a core model focused on text-to-speech, capable of producing continuous audio up to 90 minutes long in a single generation, supporting natural dialogue simulation with up to 4 different speakers. The generated speech is expressive, sounds natural and fluent, and can simulate realistic pauses, emphasis, and emotional transitions, making it very suitable for producing podcasts, long-form audio narratives, audiobooks, or multi-character dialogue content.

Compared to many traditional TTS models that only support 1-2 speakers, VibeVoice-TTS has achieved significant breakthroughs in long-form, multi-speaker consistency. Its underlying architecture uses continuous speech tokenizers (acoustic and semantic tokenizers) combined with a low frame rate design (7.5Hz), greatly improving computational efficiency for long-sequence handling.

VibeVoice-Realtime-0.5B: Real-Time TTS with ~300ms Latency

VibeVoice-Realtime-0.5B focuses on real-time scenarios, supporting streaming text input with an initial audio output latency of approximately 300 milliseconds, while also being able to generate long-form speech of about 10 minutes. This model is particularly suitable for interactive applications requiring immediate responses, such as real-time voice assistants or live broadcast dubbing scenarios.

Additionally, the project introduces experimental speaker support, including multilingual voices and various English style variants, providing developers with more customization options.

AIbase Review: Microsoft's open-sourcing of VibeVoice not only lowers the barrier to using high-performance voice AI but also provides a complete solution for local deployment. The project was briefly taken down due to potential misuse risks but was later re-released with safety mechanisms such as embedded watermarks and audible disclaimers, reflecting the principles of responsible AI development. Currently, developers can obtain model weights on the GitHub repository and Hugging Face, and quickly try them out on platforms like Colab.

With continued contributions from the open-source community (such as optimized forks for Apple Silicon), VibeVoice is expected to accelerate adoption in fields like content creation, accessibility tools, and voice interaction. Interested developers can visit the official Microsoft project page to explore further.

Project address: https://github.com/microsoft/VibeVoice

Трендові криптовалюти

CitreaCTR

wrapped stUSDTWSTUSDT

Пов'язані питання

QWhat is the name of the open-source voice AI model family recently released by Microsoft, and how many stars has it received on GitHub?

AThe open-source voice AI model family is called VibeVoice, and it has received approximately 27,000 stars on GitHub.

QWhat are the three core models in the VibeVoice family and their primary capabilities?

AThe three core models are: 1) VibeVoice-ASR-7B, which handles automatic speech recognition for up to 60 minutes of audio; 2) VibeVoice-TTS-1.5B, which generates expressive speech for up to 90 minutes with multiple speakers; and 3) VibeVoice-Realtime-0.5B, which provides real-time text-to-speech with about 300ms latency.

QWhat is a key feature of the VibeVoice-ASR-7B model regarding its output?

AA key feature is its ability to output structured transcriptions that include speaker identification (who is speaking), precise timestamps (when they speak), and the detailed content (what was said).

QHow does the VibeVoice-TTS-1.5B model achieve efficient long-sequence processing?

AIt uses continuous speech tokenizers (acoustic and semantic tokenizers) combined with a low frame rate design (7.5Hz), which significantly improves computational efficiency for long-sequence processing.

QWhat safety measures were implemented in the VibeVoice project to address potential misuse risks?

AThe project implemented embedded audio watermarks and audible disclaimer mechanisms as safety measures to address potential misuse risks.

Пов'язані матеріали

Welcome to the New Crypto World: Only This Time, The Place Where You Lose Money Is Called the Stock Market

Welcome to the New Crypto World: Only This Time, the Place to Lose Money is Called the Stock Market. The article depicts a future (2026) where global stock markets, particularly tech stocks, are exhibiting behaviors and risks reminiscent of the cryptocurrency world. The narrative centers on a sharp crash in South Korea's KOSPI index and leading stocks like SK Hynix, highlighting widespread retail losses, leverage-induced liquidations, and extreme volatility. It argues that a significant shift in market dynamics is underway: pricing is increasingly driven by narratives (like the AI boom) over fundamental valuation, amplified by high leverage and the rapid, emotion-fueled consensus formed on social media. Former crypto traders, seeking new opportunities, brought their high-risk strategies—chasing narratives, using leverage—into equity markets, focusing on sectors like AI and semiconductors. Ironically, data suggests some major tech stocks have recently shown greater volatility and faster crashes than Bitcoin, which is becoming relatively more stable and "stock-like" through institutional adoption. The piece details how leveraged ETFs in Korea, particularly those tied to single stocks, created a vicious cycle of forced selling that exacerbated the market plunge. It concludes that while stocks retain underlying fundamentals, the *trading* of them has undergone a "de-rationalization": markets now prioritize themes, social media sentiment, and leveraged bets over traditional analysis. In a paradoxical twist, Bitcoin is striving to become more like a traditional asset, while a segment of the stock market is becoming more like crypto—a cycle of grand stories, crowded trades, easy leverage, and the pervasive, often misplaced, belief that one can exit before everyone else. The final, poignant wish from a Korean retail investor—to go back to life before trading and get their money back—underscores that markets offer no refunds.

marsbit1 хв тому

Welcome to the New Crypto World: Only This Time, The Place Where You Lose Money Is Called the Stock Market

marsbit1 хв тому

The Father of Claude Code: Harness Has a Shelf Life of Only Six Months, Unshackle It

Claude Code's creator, Boris Cherny, argues that AI product harnesses (system prompts, skills, hooks) have a "shelf life" of only about six months. In a Y Combinator interview, he advises developers to regularly delete these components and rebuild them through ablation studies—testing each line's impact—because newer models like Opus 5 often don't need them. He describes models as "living organisms" with distinct "personalities" per generation, requiring empirical, non-prescriptive tuning. Cherny introduces two key concepts: "Product Overhang," where model capabilities outpace product integration, and "Unhobbling" or unleashing models by giving them harder, longer-running tasks with self-verification mechanisms. He shares an example of Claude independently rewriting an Electron app into Swift for over two weeks. For effective AI use, he dismisses "hacks" and urges an empirical approach: observe where the model struggles, then adjust. Treat the model like a colleague, not a tool to be minutely controlled. For learners, he advises combining computer science with applied skills like product design and data science. The core message is to embrace model capabilities through experimentation and reduce restrictive scaffolding.

marsbit7 хв тому

The Father of Claude Code: Harness Has a Shelf Life of Only Six Months, Unshackle It

marsbit7 хв тому

Feishu Has Become Doubao

This may be the most important day in the ten-year history of Feishu. On July 30, ByteDance announced a major restructuring via internal email. Feishu, an independent operation for a decade, will no longer function separately and will lose its independent sales and marketing teams. Its product team is being integrated into Doubao AI, and its Go-to-Market strategy merges into the Volcano Engine's To B system. This move signals a strategic pivot, not a failure of Feishu. The era where Feishu sought to be an all-encompassing super-app for productivity is ending. AI has become the dominant logic. Industry trends show a shift from building platform "entrances" to embedding AI directly into workflows, as seen with Microsoft Copilot and Google's Gemini. Data reveals the driver: over 90% of Feishu's new clients in Q2 2026 purchased its AI products. Enterprises are now buying integrated AI solutions, not standalone software. By integrating Feishu's deep enterprise workflow scenarios, Doubao evolves from a consumer-facing chatbot into a powerful enterprise productivity engine. This gives Doubao a critical asset: real-world business application. Concurrently, ByteDance established the "Creative Services Platform," unifying its previously fragmented sales fronts (Feishu, Doubao, Volcano Engine) into a single customer-facing entity. This represents ByteDance's first genuine shift to thinking like a true To B company, organizing around client needs rather than internal product silos. The restructuring underscores that AI is no longer just a business line but foundational infrastructure. Doubao is becoming the new organizational core, mirroring TikTok's past ascendancy. This demonstrates a profound shift: AI is beginning to reshape corporate structures, determining resource allocation and strategic priorities. ByteDance is likely just the first of many companies to undergo such an AI-driven reorganization.

marsbit11 хв тому

marsbit11 хв тому

The Second Half of the RWA Issuance Race: Amidst the Utilization Dilemma, Trillions Worth of On-Chain Assets Awaken

RWA (Real-World Asset) tokenization has surged to a record $32 billion in on-chain value. However, a structural challenge has emerged: approximately 87% of these tokenized assets are inactive, not participating in DeFi lending or trading activities. This "utilization gap" highlights a key question: what happens after assets are brought on-chain? Data shows a stark contrast among major platforms. While Securitize leads in scale ($4.9B), it has a DeFi utilization rate of only 0.7%. Similarly, Ondo Finance ($3.5B) sees about 2.7% utilization. In contrast, Maple Finance, with a smaller $2.3B in assets, achieves a 62% utilization rate by embedding its assets directly into lending activities from the outset. Three primary factors contribute to low utilization: 1) **Asset Nature**: Tokenized treasuries are often held for yield, not traded. 2) **Regulatory Barriers**: Securities laws and KYC requirements restrict free movement into permissionless DeFi pools. 3) **Infrastructure Gaps**: A lack of deep secondary markets, real-time pricing, and mature clearing mechanisms hinders liquidity. The RWA narrative is now shifting from a "race to issue" to a "battle for application." While native lending protocols like Maple have inherent advantages, distribution channels are becoming critical. The success of products like SyrupUSDG on platforms like Robinhood Crypto demonstrates that access to existing user bases can accelerate adoption far more effectively than protocol-led efforts alone. The $32 billion milestone validates the feasibility of tokenization, but low utilization underscores that bringing assets on-chain is just the first step. The true test for the sector is enabling these assets to circulate, be utilized, and create new economic value within the DeFi ecosystem.

marsbit12 хв тому

The Second Half of the RWA Issuance Race: Amidst the Utilization Dilemma, Trillions Worth of On-Chain Assets Awaken

marsbit12 хв тому

Fake Flare Network Website Stole $8.5 Million Worth of XRP

A fake staking website, impersonating Flare Network and FXRP, stole approximately 3.4 million XRP (worth around $8.5 million USD) from 71 investors over eight days in October last year. The fraudulent site lured victims by promising guaranteed principal and monthly returns of 1.5% to 1.8%. It tricked investors into withdrawing their XRP from local exchanges and sending it to wallets controlled by the scammers via offshore platforms. The site was shut down on October 23, and its operators vanished. Seoul police tracked assets worth 27.3 billion won ($18.8 million) linked to the scam, freezing 17.3 billion won. Two individuals involved have been handed over to prosecutors on aggravated fraud charges, and another is under investigation for document forgery. Police executed 54 search and seizure warrants and arrested a suspect who had returned from hiding abroad. South Korean authorities have declared a zero-tolerance policy for cryptocurrency fraud this year, warning investors to verify official sources before sending funds as part of heightened vigilance against such scams.

cryptonews.ru27 хв тому

Fake Flare Network Website Stole $8.5 Million Worth of XRP