> They did this by not breaking the universe in these major updates
I don't think the amount of breakage per se was the problem with Python 3. I think the problem was that for a long time (until u"" strings in 3.3? four years after 3.0?) there was ~no way for a library to support both 2 and 3 in a single codebase. That meant that each project had to either maintain a fork, or do a "flag day" migration, which in practice meant you couldn't migrate until all your dependencies were migrated. Big mistake in retrospect.
When I was doing undergrad CS in 2006, we had a choice between Perl and Python for our scripting assignments. As far as I know, no one chose Perl. It's comparatively a huge pain to get ramped up on, and the promised payoff is that you get all these implicitly stateful sigils that let you write contest-winning one-liners. But by the early 2000's, the culture was getting up to speed on obvious-in-retrospect principles like "avoid global variables" and "you should give your variables names that someone else can understand", and even as students we could all see that Perl wasn't going to be the future.
I chose Perl first. At that time around early 2000s it was the more popular. I liked how my scripts were short and compact but then I I had noticed that I couldn't understand what I did week or so later. The "write-only language" joke was kind of true for me. Then remember finding Python and rewriting all my script in Python and have been using it ever since.
The boring answer is that standard atomics didn't exist until C++11, so any compiler older than that didn't support them. I think most platforms (certainly the popular desktop/server platforms) had ways to accomplish the same thing, but that was up to the vendor, and it might not've been well documented or stable. Infamously, `volatile` used to be (ab)used for this a lot before we had proper standards. (I think it still has some atomic-ish properties in MSVC?)
I think it's pretty rare to do a straight up atomic load of a refcount. (That would be the `use_count` method in C++ or the `strong_count` method in Rust.) More of the time you're doing either a fetch-add to copy the pointer or a fetch-sub to destroy your copy, both of which involve stores. Last I heard the fetch-add can use the "relaxed" atomic ordering, which should make it very cheap, but the fetch-sub needs to use the "release" ordering, which is where the cost comes in.
I imagine it's kind of like "What's stopping someone from forging your signature on almost any document?" The point is less that it's hard to fake, and more that it's a line you're crossing where everyone agrees you can't say "oops I didn't know I wasn't supposed to do that."
I still wouldn't call it GC in that case. It's pretty much exactly the same as std::shared_ptr in C++, and we don't usually call that GC. I don't know about the academic definition, but I draw the line at a cycle collector. (So e.g. Python is GC'd, but Rust/C++/Swift are not.)
I consider reference to be garbage collection, and so do most CS textbooks. However Rc/Arc/shared_ptr are GC facilities used (often sparingly) inside predominantly non-GC'd languages, so, yeah, I wouldn't say Rust "is" or "has" GC. It has facilities for coping with cleanup, both RAII and GC.
Large C++ codebases have the same problems that large codebases have in any language: too many abstractions, inconsistent ways of doing things, layers of legacy. It comes with the job. The difference is that in C/C++, hard-to-read code also means hard-to-guess pointer lifetimes.
If the C++ code I worked on looked like that[1] and was actually C with classes, then I’d be switching to Rust too.
For Google and Microsoft it probably makes sense to rewrite Windows and Android in Rust. They have huge amounts of legacy code and everybody’s attacking them.
It doesn’t follow that anyone else, or the majority has to follow then. But that’s predictably exactly what veteran rustafarians are arguing in many comments in this thread.
[1] Pointers getting passed all over the place, direct indexing into arrays or pointers, C-style casts, static casts.
That (PVOID)(UINT_PTR) with offsetting and then copying is ridiculous.
I like to focus on the ways that C is actually quite complicated, especially the complications that directly provoke UB when you don't know about them. Integer promotion and strict aliasing are at the top of my list.
Rust futures are "just" structs with a poll() method. The poll() method is a function like any other, so it can have local variables on the stack as usual, but anything it wants to save between calls needs to be a field of the struct instead of a stack local. The magic of async/await is that the compiler figures out which of your async function's variables need to be fields on that struct, and it generated the struct and the poll method for you.
I see. The Rust implementation effectively splats out the transitive closure of all your callee stack frames upfront which would enable continuing previously executing async functions.
> This seems like a contradiction to me. How can future1 acquire the Mutex in the first place, if it cannot run? The word "given" is really odd to me.
`future1` did run for a bit, and it got far enough to acquire the mutex. (As the article mentioned, technically it took a position in a queue that means it will get the mutex, but that's morally the same thing here.) Then it was "paused". I put "paused" in scare quotes because it kind of makes futures sound like processes or threads, which have a "life of their own" until/unless something "interrupts" them, but an important part of this story is that Rust futures aren't really like that. When you get down to the details, they're more like a struct or a class that just sits there being data unless you call certain methods on it (repeatedly). That's what the `.await` keyword does for you, but when you use more interesting constructs like `select!`, you start to get more of the details in your face.
It's hard to be more concrete than that without getting into an overwhelming amount of detail. I wrote a set of blog posts that try to cover it without hand-waving the details away, but they're not short, and they do require some Rust background: https://jacko.io/async_intro.html
So my understanding was correct, it requires the programmer to deal with scheduling explicitly in userspace.
If I'm writing bare metal code for e.g. a little cortex M0, I can very much see the utility of this abstraction.
But it seems like an absolutely absurd exercise for code running in userspace on a "real" OS like Linux. There should be some simpler intermediate abstraction... this seems like a case of forcing a too-complex interface on users who don't really require it.
There is one: tasks. But having the lower level (futures) available makes it very tempting to use it, both for performance and because the code is simpler (at least, it looks simpler). Some things that are easy with select! would be clunky with tasks.
On the other hand, some direct uses of futures are reminiscent of the tendency to obsess over ownership and borrowing to maximize sharing, when you could just use .clone() and it wouldn’t make any practical difference. Because Rust is so explicit, you can see the overhead so you want to minimize it.
To be clear, if you restrict yourself to `async`/`.await` syntax, you never see any of this. To await something means to poll it to completion, which is usually what you want. "Joining" two futures lets you poll both of them concurrently until they're both done, which is kind of the point of async as a concept, and this also doesn't really require you to think about scheduling. One place where things get hairy (like in this article) is "selecting" on futures, which polls them all until one of them is done, and then stops polling the rest. (Normally I'd loosely say it "drops the rest on the floor", but the deadlock in this article actually hinges on exactly what gets "dropped" when, in the Rust sense of the `Drop` trait.) This is where scheduling as you put it, or "cancellation" as Rust folks often put it, starts to become important. And that's why the article concludes "In the end, you should always be extremely careful with tokio::select!" However, `select!` is not the only construct that raises these issues. Speaking of which...
> But it seems like an absolutely absurd exercise for code running in userspace on a "real" OS like Linux
Clearly you have a point here, which is why these blog posts are making an impact. That said, one counterpoint is, have you ever wished you could kill a thread? The reason there are so many old Raymond Chen "How many times does it have to be said: Never call TerminateThread" blog posts, is that lots of real world applications really desperately want to call TerminateThread, and it's hard to persuade them to stop! The ability to e.g. put a timeout on any async function call is basically this same superpower, without corrupting your whole process (yay), but still with the unavoidable(?) difficulty of thinking about what happens when random functions give up halfway through.
I don't think the amount of breakage per se was the problem with Python 3. I think the problem was that for a long time (until u"" strings in 3.3? four years after 3.0?) there was ~no way for a library to support both 2 and 3 in a single codebase. That meant that each project had to either maintain a fork, or do a "flag day" migration, which in practice meant you couldn't migrate until all your dependencies were migrated. Big mistake in retrospect.
reply