I'd stay as many days as you can to the extent it doesn't interfere significantly with work & personal life, or alibi(s) for that reason. I believe 3 to 4 days is sufficient and provides a peace of mind in addition to being able to rapidly address any post-op matters (which are typically minor in nature like nodules and such).
That said, if you can only spend a day or two because of life's constraints/circumstances, this isn't a deal-breaker; as long as you follow post-op instructions, it seems 99% of the time guys do just fine.
Not that I would EVER recommend this, I'm only sharing this hypothetical to emphasize the simplicity of how outpatient these procedure types are. In theory (again, not recommended), you could leave your day job for an hour long lunch break, go into the Clinic, have the procedure done, and be back in your office as if nothing had happened (other than your frequent trips to the restroom to massage your bandaged unit). Point being, it's a relatively smooth procedure with little downtime, especially when post-procedure directions are followed. And of course, this hypothetical doesn't take border crossing times and such, but I think you see where I'm going with the scenario.
Despite me going on about how relatively smooth the whole thing is, I would still recommend 3 to 4 days, and as much as 7 days if you have the time (traveling into San Diego can be quite fun in the Spring, Summer, and Fall). I'd recommend the Grand Hotel due to its proximity to the Clinic (they are adjacent to one another), but the Marriott is a popular alternative if you don't mind short cab rides to and from the Clinic.