Speaking as former physicist, can only agree with you.
However, regarding the Dirac delta function an other generalized functions, even though they were introduced by physicists (I think Heaviside was an early proponent) in a hand wavy fashion, they were later put in rigorous mathematical footing in the theory of distributions. Distributions are nowadays used by mathematicians without any hesitation.
This also not the first instance of something like this, and it won't be the last. Physicists have come up with ad hoc methods that work, but they can't justify why with rigor. Some time later mathematicians formalize it and it becomes part of their tool set.
> they were later put in rigorous mathematical footing in the theory of distributions
Yeah some French dude called Laurent Schwarz that got the Fields medal for it.
IIRC he built the set of distributions as the dual vector space of the tiniest vector space he could think of: extremely well behaved functions (C-infinite functions with finite support - weird mathematical beasts that go to zero at the extremities of their support with all derivatives also going to zero there as well).
I never really managed to grok the intuition behind the formalization of Schwarz, whereas the hand-wavy physicist way is pretty straighforward to understand.
However, regarding the Dirac delta function an other generalized functions, even though they were introduced by physicists (I think Heaviside was an early proponent) in a hand wavy fashion, they were later put in rigorous mathematical footing in the theory of distributions. Distributions are nowadays used by mathematicians without any hesitation.
This also not the first instance of something like this, and it won't be the last. Physicists have come up with ad hoc methods that work, but they can't justify why with rigor. Some time later mathematicians formalize it and it becomes part of their tool set.