Allow me to open with a wildly speculative question: What if the internet were public interest technology? I mean "internet" the way most people understand it, which is to say our whole digital sphere, and by "public interest" I don't mean tinkering at the margins to reduce harm from some bad actors or painting some glossy ethics principles atop a pile of exploitative rent-seeking — I mean through and through, warts and all, an internet that works in support of a credible, pragmatic definition of the common good.
The moment you try to define a singular "common good", you wind up with a variety of competing factions all putting forth their own wildly divergent and often contradictory notions of what that common good consists of.
Most people have an unfortunate tendency to project their own values and preferences onto the world at large, and fail to recognize when they cross the boundary out of their own spaces and into other people's.
Recognizing this means advancing solutions that primarily aim to minimize conflict among many parties, each pursuing their own particular concept of the good within their own boundaries, and avoiding trying to universalize any singular set of terminal values.
Attempting to pursue solutions that depend on everyone agreeing on the same set of terminal values will always fail, and will often generate intense conflict that escalates well beyond the bounds of the original question and causes a great deal of collateral damage.
> The moment you try to define a singular "common good", you wind up with a variety of competing factions all putting forth their own wildly divergent and often contradictory notions of what that common good consists of.
Yes, that's called compromise. It's basically one of the foundations of society and civilization. It's not a blocker for public-interest projects.
'Compromise' paints an idealistic picture, its often just a euphemism for 'the more powerful/entrenched party gets what they want by default, but they'll let the other side vocally express their frustration to maintain the veneer of an actual conversation.' What you're describing sounds like a top-down restructuring of the internet at large, which is a golden opportunity for any interested party to restructure it in their favor. Whoever has the means will do so, and it will not be motivated by public interest since the public are not the ones with the means. When I hear "an internet that works in support of a credible, pragmatic definition of the common good", to me that translates to "we're going to turn the surveillance up to 11 and ban encryption to combat CSAM", because that's how such projects pan out in reality
> which is a golden opportunity for any interested party to restructure it in their favor. Whoever has the means will do so, and it will not be motivated by public interest
What if the interested party is clever and defines "in my favor" to be equal to "the common good"? ;)
Honestly, I can't even finish reading through this comment. This is the opposite of idealism - is there a word for it? You seem to be complaining that something is impossible because it can't be implemented in a perfect utopian way. Well, guess what, we already have an implementation of the internet that imperfectly respects "the public good", and it sucks. Let's imagine something better.
> This is the opposite of idealism - is there a word for it?
Far from being the opposite of idealism, this approach is in fact the only one by which high ideals can be approximated in reality.
> You seem to be complaining that something is impossible because it can't be implemented in a perfect utopian way.
Quite to the contrary, the complaint is not merely that the pursuit of these goals would fall short of perfection, but rather that the consequences would largely be the inverse of the intentions.
In essence, idealism is its own opposite -- if you're looking for a single word to describe this critique, some good options might be "correctness", "efficacy", and "reasonableness".
>You seem to be complaining that something is impossible because it can't be implemented in a perfect utopian way.
That's not what I'm saying, in fact 'perfect utopian' projects are exactly what I'm expressing skepticism towards. The problems we have with the internet are mostly just symptoms of deeper societal issues, and they arent infrastructure problems that can be easily fixed like a road or bridge with some massive spending bill. If the US actually enforced anti-trust laws and broke up the tech cartels it would solve a whole lot of problems with the internet, but I doubt that idea would get much traction with whoever ends up on the 'new internet committee'. And I dont see it as a lack of idealism, its just plain pragmatism
Just take a look at the schism between Japanese Mastodon instances vs Western Mastodon instances and their vehement disagreement on what constitutes pornography and CSAM. These are userbases composed of mostly developed country users with liberal values, high incomes, and varying degrees of social infrastructure. If you can't even get these entities to agree, how do you think you'll get NATO countries onboard, let alone BRICS?
Nah, idealism is itself defeatism -- nothing sustains the status quo greater than people wallowing in speculation unmoored from reality instead of taking practical measures to address the problems before them.
No, compromise is when a variety of people with different interests and values try to find a middle-ground solution that's sufficiently acceptable to everyone involved, each according to their own particular criteria. Proposing solutions that can only be pursued if everyone adopts a single set of criteria does not encourage compromise, it encourages conflict.
Societies are not monolithic blobs with a singular "common good" -- they're complex networks of relations among different people with fundamentally varying worldviews and value systems. Making public-interests projects work entails respecting pluralism and individual autonomy. There's no alternative: projects that depend on conformity will inevitably fail.
Better to let ideas compete and build consensus than to surrender decision making abilities that effect everyone over to some arbitrary group of people that we are just told to trust (but don't worry they have good intentions).
We don't currently have a select group of people who share a vision of "common good" making decisions for us, and yet things keep improving. So I'd advocate we stick with what we know is working, rather than the above surrendering of autonomy in exchange for the promise of some utopia..
I have no doubt you could find some metrics to show a recent decline in, but (on the whole) is the average human not better off in the year 2024 than they were in 1954? 1924? 1824?
I do have a feeling we're not talking about the same thing here, so could you clarify how it's "obviously not true" that things have been/are improving?
I'm referring to the internet, not the whole of human existence. That would be too complicated a subject to make any such simplistic statements on. But actually I also disagree with your premise that we don't have small corrupt groups already deciding what's "best" for everyone, WRT the internet or life as a whole. Just because they're openly motivated by greed doesn't make them less harmful. It's the cabals atop Google and Facebook which ushered in an age of absolute surveillance on a mass scale, how cool and fun.
I never said anything about corruption one way or another, I said we don't have groups with an agreed upon definition of "common good" - look at basically any healthy decision-making group (government, party planning committee, whatever) and you'll find two sides with differing definitions.
I think we'd maybe both agree that Google and Facebook ushering in "absolute surveillance" probably came about from a single-minded view of "common good" within those companies - aka an example of a group of people aligned on a common good leading to things going poorly.
But it's silly to think that some other group with a different (but also single-minded) definition of common good is going to somehow fix all the problems and not cause new, potentially worse, problems. That's what I was attempting to get at with my initial comment.
Given that we weren't even really talking about the same thing from the start, and that I don't care enough to continue, I'm gonna opt out from this convo. Have a nice day though.
I mean to be honest something like the early social media platforms like myspace or even reddit up to a degree were public interest platforms.
And back then nobody really thought all that mich about financing, so these spaces weren't about extracting user data or shaping their opinions. The algorithms were simplistic as hell and the timelines still deserved that name.
The state runs libraries not just to give people access to books, but also because they are social community spaces. Why not provide something like that, just online. Something that doesn't need to make money, but provide a service that people can trust in a different way that a corporation.
It’s only the first public release, rest assured we have plans to develop it beyond that point :-)
Data tables are high on the list, but it’s going to be a lot of work and I can’t say when we’ll have something to share.
In the meantime almost any library available on npm should work out of the box—not just the ones that we added explicit support for (even though of course some might need more work than others).
See the example at https://huggingface.co/spaces/observablehq/fpdn where DuckDB is used both as a data loader (to download and digest 200GB worth of source data into a small 8MB parquet file) and on the client-side to allow the user to do live search queries on the minimized data. Server-side, we're using duckdb-the-binary, and client-side we're using duckdb-wasm.
So the 200Gb loading and digesting part is totally separate from the Observable Framework, right? You just do it with a standard ( non wasm duckdb as part of ETL) and later you just direct Observable Framework to read and plot the 8Gb file? Thanks
The code in a Framework can do whatever you want it to do—it can load data on demand, call an external API, etc. Precomputing data is only an option, not an obligation.
But even when you want things to be very interactive it is a good idea to minimize the data. Expose only the "rows and columns" that you need, and compress it as much as possible. This can be done in a data loader. For example, see the data app we deployed yesterday on hugging-face: its data loaders ingest a large source database (320 files totaling 200GB), and digests it into a single 8MB parquet file that we can then use on the page to "live query" 3 million newspaper titles and dates. https://huggingface.co/spaces/observablehq/fpdn
