Effective commercialization of station space is indeed a positive development, but you don't really need to privatize railways to get that (even without going into the whole "privatizing in Japan is not the same as here", since large private companies and local authorities coordinate strongly in ways that we wouldn't consider acceptable in the West). Very dense European cities, like London and Paris, are getting more and more of that type of development too; and even in Tokyo, not all stations have a commercial development on top. It's mostly a function of density levels, which are sky-high in Japan.
One clear element of the Japanese system is that stations are hugely overmanned, and staff are still paid pretty good money. That means facilities are spotless, and drifters or nuisance riders are removed promptly, making the system more appealing. This is very hard to implement in the West, where the sacred fear of unionization pushes for constant cuts, both in the number of humans involved in any task and in their remuneration levels.
I agree with your general points and I don't know what worked in Japan would work in the USA. I'm pretty confident what works in Europe will probably not work in the USA either though :(
> That means facilities are spotless
I can assure you no stations are remotely spotless. In fact I'm surprised some of them aren't considered fire hazards. Ueno, Shinjuku, Shibuya, Shinagawa, Akihabara, all have extremely messy areas. Newer stations appear clean but that's only because they're new AFAICT. Just commenting because I don't want people to ge the wrong impression. They might be cleaner than SF, NYC, Paris on average but they're not as clean as Stockholm or Singapore
Of the stations that you mention, I find Akihabara to be actually quite nice and reasonably well organized (for Japanese standards).
But I agree with you that the others are a mess. Shibuya and Ueno look like half the stations are falling apart, and Shinjuku is some kind of non-euclidean labyrinth, every time I go there, I get lost. If there is even a fire there, the death toll is going to be immense.
Akihabara, there are new and old parts of the station. The new part (lower level) are relatively clean. The old parts (upper level, Sobu-sen) are less clean (unless they've been renewed - I haven't been up in a 2-3 years)
I think that you and the OP have different notions of spotless.
In Japan, at major train stations I find the bathrooms reasonably clean and good to use. In New York, it smells of poop and pee because the odds are that someone probably publicly urinated, or some pipe somewhere in the subway tunnel is leaking sewage.
That's generally true of all bathrooms in the USA vs Japan. I think I've seen 1 disgusting bathroom in Japan a restaurant/mall/store in the more 15+ years I've spent there where as in the USA it's like 1 of 5 that's disgusting. In other words, it's not unique to train stations.
That said, the bathrooms between JR Shinjuku and Marunouchi-line used to smell pretty rancid. I haven't used them recently though.
Oh, I agree it's not unique of train stations. The Japanese overstaff pretty much anything - it's likely part of the reason for their massive public debt, but it helps maintaining infrastructure to good standards. It's the opposite attitude we see in most of the Western world, where public resources are cut to the bone until (and often even after) things rot and fall apart.
JR East covers not only metropolitan train services in Tokyo but also a wider real estate/retail portfolio and all of eastern Honshu, with about 46k total employees.
One clear element of the Japanese system is that stations are hugely overmanned, and staff are still paid pretty good money. That means facilities are spotless, and drifters or nuisance riders are removed promptly, making the system more appealing. This is very hard to implement in the West, where the sacred fear of unionization pushes for constant cuts, both in the number of humans involved in any task and in their remuneration levels.