Any plans to release a RVQ encoder or Flow-VAE encoder to enable audio-conditioned generation?

#11
by TheLatentSpacer - opened

Thanks for publishing these weights, the quality is great.

I'm wondering whether reference-audio conditioning (continuation, style priming, repaint, etc.) is possible with the released weights? As far as I can tell it isn't, because no audio-to-latent or audio-to-token path was published

Any plans to release any of these paths to enable audio-conditioned generation?

Thanks either way!

dav.pth already has the VAE encoder

the RVQ encoder is only required for covering songs.

also an open discussion about it is already there. did you check first?

Sign up or log in to comment