In a groundbreaking development, MyShell, a Calgary-based startup founded in 2023, has released OpenVoice, an open-source AI that promises unparalleled speed and accuracy in voice cloning. Developed in collaboration with researchers from MIT and Tsinghua University, OpenVoice allows users to clone voices with just seconds of audio input, offering granular control over various parameters like tone, emotion, accent, and rhythm.
Two-Model System
OpenVoice operates through a two-model system, combining text-to-speech conversion with voice tone cloning. The first model, trained on 30,000 audio samples from English, Chinese, and Japanese speakers, handles language style, accents, emotions, and other speech patterns. The second “tone converter” model, drawing from over 300,000 samples and 20,000 voices, refines the cloned voice to achieve a high level of accuracy.
Efficiency and Speed
By leveraging a universal speech model and incorporating user-provided voice samples, OpenVoice stands out for its ability to clone voices with minimal data. This unique approach allows for significantly faster speech generation compared to alternatives like Meta’s Voicebox. Users can experience the capabilities of OpenVoice through demo sites on both MyShell and HuggingFace.
MyShell's Background
MyShell, with early funding of $5.6 million and over 400,000 users, positions itself as a decentralized platform for creating and discovering AI apps. Beyond voice cloning, the startup offers a diverse range of AI applications, including original text-based chatbot personalities, meme generators, and user-created text RPGs. Some content is behind a subscription fee, and the platform charges bot creators for promotional services.
Open-Source Initiative
MyShell’s decision to open-source OpenVoice through HuggingFace is a strategic move to advance an open model of AI development. By making the technology accessible to a broader community, MyShell aims to increase its user base while contributing to the democratization of AI capabilities.
