A new generation of talent is entering China’s entertainment industry and they may change how we think about “star talent” in ways that we’ve never imagined.
In a Christmas music video released by Chinese girl idol group SNH48, six of the group’s stars sing and dance with digital copies of themselves. The copies are AI 3D avatars that mimic the looks, voices and body language of the humans that they’re based on.
The new “digital idols” appear in four-minute music video that was co-produced by Tencent-backed artificial intelligence (AI) start-up ObEN. ObEN claims that this is the world’s first commercially released song co-starring human singers and their AI 3D avatars.
Xiong Wei, Vice-president of Shanghai-based SNH48 believes that the idol entertainment industry is about to change in a dramatic way where the AI 3D avatars perform in concerts and have personalized interaction with fans, and that the avatars will sell albums and appear in movies and television programs.
Xiong said, “This song is our first step to test the waters in the virtual idol market. We are planning to create more intelligent virtual idols, releasing albums and making movies for them.”
Xiong added: “Human idols are hardly perfect. They have their limits in terms of age and there may be gossip about their love affairs. None of these will be an issue for virtual idols.” He also noted that a female idol usually has a career lifespan of eight years before she stops “exuding a youthful glow”.
SNH48 is similar to Japanese idol girl group AKB48, and features more than 100 female performers. These girls are around 20-years old. They perform regularly at the group's own theatre and interact with fans predominantly via live promotional events. For many of the members, their participation is SNH48 is a step in their dream to become big stars in the Chinese entertainment world.
California-based ObEN said that by using computer vision, natural language processing and speech technology they are able to create 3D intelligent avatars that not only look, sound, and learn to behave like their individual human counterparts but can stay forever young.
To create the digital clones of the singers, ObEN used photographs and voice recordings. In order to create personalized AI algorithms, each singer was required to read 100 to 200 sentences that were used as sample data for the AI system. The result were digital avatars that could sing like each group member and even speak multiple languages in their own voices.
Adam Zheng, Chief Operating Officer and Co-founder of ObEN commented to media on the new digital idols and said: “Our personal AI technology has great potential in the entertainment industry. Let’s say you have an actor who stars in a movie. A producer may want him to sing the ending theme song, which he is not good at. We can make his AI avatar sing for him.”
SNH48’s partnership with ObEN is not the only one exploring the potential of AI in the entertainment and media industries and there have been a number of developments across the region:
China - Xinhua News Agency introduced “composite anchors” in November 2018. Using AI technology provided by Beijing-based search engine operator Sogou to combine the images and voices of human anchors. Xinhua said that it envisions using AI 3D anchors as a way to provide customized data to individuals while saving on human costs.
Japan - Hatsune Miku, the schoolgirl-like, turquoise-¬haired anime mascot is arguably the most famous virtual star. Designed by a Japanese software company, she was first used to sell a voice synthesizer. As consumers began using the product to compose original music, Miku became a sensation, drawing massive numbers of followers and even performing concerts where her hologram “sings” fan-written songs.
South Korea - AI Stars, a joint venture with S.M. Entertainment, South Korea’s largest entertainment agency, claims to be the first celebrity AI agency, combining the intellectual property of SM’s top k-pop stars with ObEN’s AI technology to create new interactive experiences and products.
Digital avatars like Miku and the Xinhua news anchors are not “intelligent” in a manner that lets them freely communicate with humans, but Xiong of SNH48 believes that in the future, digital group members will be able to appear in different places at the same time and that the technology will allow virtual idols to have personalized interaction with fans.
Xiong said, “Our stars are busy travelling across China to perform 1,000 shows a year in different cities so the virtual idols can sing and dance in different locations at the same time.”
Entertainment analysts say that it’s too early to know whether digital idols can generate enough excitement and revenue to create long-term interest or whether they’ll ultimately detract from human idols. Its also open to discussion how digital idols should be paid, compared to their human “originals” and whether “fans” will pay enough to cover AI 3D avatar development and maintenance costs.