A few things to consider on why this is almost always a non-issue:
(1) Penises come in a variety of shapes and sizes, so irregularities are more likely to be seen as "penile diversity" than it would be "enhancement."
(2) Skin also comes with variation, so bullet-point (1) still applies.
(3)
Girth enhancement is nowhere near mainstream, and there is too little information circulating to give a partner reason to even contemplate this possibility.
(4) You notice the changes in your penis because like many of us, it's easy to obsess over the nuances and changes from round-to-round. If your partner has no clue, they have no reason to suspect.
The above is true, regardless of filler choice.
I've always said - if you act like you have something to hide, you'll invite scrutiny. Carry it with confidence and you'll be fine. In the unlikely/super-rare event someone questions it, just dismiss it nonchalantly with a "I was born this way" or for those with significant alteration, "this was due to an injury years ago, and built up scar tissue."
In my time moderating these forums, partners questioning the penis were exceedingly rare (in fact, I can't think of an anecdote off the top of my head).