This is, indeed, the theory.
If the lab tests fail to find low hormone levels, then obviously something else is needed to reliably detect this so-called “adrenal fatigue”. However, no other test is known.
This means that the so-called “adrenal fatigue” remains an unproven theory.
Contrary to what you say, doctors really do care if someone walks in and says they’re feeling tired for the last five years. They just don’t run after one of the many unproven theories out there.