You can use various non-dairy milks, but I don’t think the grains will survive long-term, so you should use spare ones. It’s possible to use a mixture of milk + non-dairy milk as well, which might solve that problem. Dom has some info on his FAQ Page.
I’ve made a couple of batches of grape juice kefir (tastes like rosé) with my sugary grains, and I always use spare ones and eat them after it’s finished. My main batch is plain and I use it mainly to grow the grains. The sugary ones grow extremely fast though, so I’m not sure how practical it would be with milk grains.
Do you know if you’re lactose or casein intolerant? This article suggests it’s possible to resolve lactose intolerance simply by consuming probiotics with lactose-digesting genes, since bacteria readily exchange genes, even with completely unrelated species (like a duck and an elephant mating). Casein intolerance would be less straightforward, but still possible if the article is accurate. Your reaction could just be die-off, too.