Music at ByteDance

Projects from my contract with ByteDance as a Music Operations Campaign Lead, as well as from my summer internship for their SAMI team (Speech, Audio, and Music Intelligence). Contributed to projects such as Speech-to-Song, Text-to-Speech, and their AI-powered music creation app Ripple. This project is no longer being updated in the United States.

Speech to Song Feature

As of late 2024, the Speech-to-Song feature can be found in "audio effects" when editing a voiceover in CapCut. When selected, the vocal input will be time/pitch warped to a track to make it sound like singing or rapping.

Original Tracks - Reggae

In addition to testing to improve the feature, all tracks under the Reggae category of Speech-to-Song were created by me. Utilized Ableton Live, Native Instruments, and various instrument samples. Tracks had to be loop-able in order to accept a vocal input of any length.

REGGAE LOOP 90BPMSpeech to Song

00:00 / 00:43

REGGAE LOOP 85BPMSpeech to Song

00:00 / 00:45

REGGAE LOOP 80BPMSpeech to Song

00:00 / 00:48

REGGAE LOOP 75BPMSpeech to Song

00:00 / 00:51

Put simply, the Speech to Song algorithm analyzes the vocal input and determines which track, tempo, and key is most appropriate. From there, the key of the track is transposed and the vocals are warped to fit.

Ripple - AI Music Creation App

Ripple features templates for songs of many genres for users to implement. Users can change the tempo/key as they please, as well as "regenerate" the bass and drums with brand new sound design and pattern. As part of the contract with ByteDance, my production group with good friends Elliot Harper and Carlo Redl worked together to create 60 templates of various genres that could be found on the Ripple App. Ripple has since been deprioritized, and will no longer be available in the United States.

Highlighted Styles

Here are demos from some of my personal favorite templates:

LUNGStrap

00:00 / 00:16

RIDE HOMEpop

00:00 / 00:16

BEVERLY CHILLSg-funk

00:00 / 00:20

LOW SUNr&b

00:00 / 00:16

HARD LIFEtrap

00:00 / 00:13

SKY BLUEr&b

00:00 / 00:21

LIGHTYEARtrap

00:00 / 00:13

WAVESr&b

00:00 / 00:14

WESTERN SONSg-funk

00:00 / 00:21

EMERALD EYESold-school

00:00 / 00:21

DANGEROUSdrill

00:00 / 00:15

SAVE MEtrap

00:00 / 00:14

These templates along with its symbolic data have to be delivered in a very specific way, as the app has its own special sampler for bass and drums to support regeneration.

User Studies

I also led user studies in Miami for Ripple, which are directly responsible for new features and changes in the app. Here are a few of them:

More/easier customization for song structure
Loop playback function
Horizontal playback no longer affected when dragging vertically
Fixed export bug; did not sound the same as when played in the project
Multi-Take Vocals
"Punch-In" and prepare feature