>"having a tough time finding why shouldn't use just custom elements with no framework"

This is exactly what I do on SPA front ends for many of my customers. Of course we use some libs but those have nothing in common with the frameworks, just narrowly scoped solutions. Saves gobbles of time and money.