DId you get checked for varicoceles? Look into vasectomy induced varicoceles, and the reduction of testosterone that (potentially) comes with. Even if your serum/free test is good, your intratesticular testosterone could be crap with a varicocele.
TRT is also fairly difficult to get right, until it isn't. I have to run higher E2 or my libido is crap (higher SHBG, subclinical hypothyroidism). You could have high prolactin, crashed E2, high E2...it's borderline alchemy more than endocrinology.
But since it corresponded almost perfectly with the vasectomy, get into a
Urologist and get checked for varicoceles.