You are certainly not alone in your suffering, I have also been completed debilitated by die off symptoms whenever I take any antifungals. I would concentrate on strengthening your other organs first. For me this was my gallbladder, thyroid and kidneys.
As far as fighting the candida is concerned my naturopath has recommended Saccharomyces boulardii which is a yeast that competes with the candida to out number it in the gut, but dies after 5-7 days so will not cause a different overgrowth. This is a far less agressive approach. I start taking the Saccharomyces boulardii next week (to give my organs time to strengthen first) so will let you know if I suffer bad die off from it, I’m hopeful I won’t! This approach may work well for you, as you also can’t handle the die off toxins.
The turning point of my treatment i feel has been sessions of kinesiology. It’s not for everyone and hard to believe until you do it but really has worked for me, definitely worth looking into as it may help you identify your other problems.