This is a neat article, but it does have some errors.
One subtle point that the post gets wrong:
> So where does that come from? The answer is that Python stores everything inside dictionaries associated with each local scope. Which means that every piece of code has its own defined “local scope” which is accessed using locals() inside that code, that contains the values corresponding to each variable name.
The dictionary returned by `locals()` is not literally a function's local namespace, it's a copy of that namespace. The actual local namespace is an array that is part of the frame object; in this way, references to local variables may happen much more quickly than would be the case if it had to look each variable up in a dictionary every time.
One consequence of this is that you can't mutate the dict returned by `locals()` in order to change the value of a function-local variable.
Another, less-subtle error in the post is this:
> int is another widely-used, fundamental primitive data type. It’s also the lowest common denominator of 2 other data types: , float and complex. complex is a supertype of float, which, in turn, is a supertype of int.
> What this means is that all ints are valid as a float as well as a complex, but not the other way around. Similarly, all floats are also valid as a complex.
Oh, no no no. Python integers are arbitrary-precision integers. Floats are IEEE 754 double-precision binary floating-point values, and as such only support full integer precision up to 2^53. The int type can represent values beyond that range which the float type cannot.
And while it is true that the complex type is just two floats stuck together, I would very much not call it a supertype. It performs distinct operations.
> Accessing an attribute with obj.x calls the __getattr__ method underneath. Similarly setting a new attribute and deleting an attribute calls __setattr__ and __detattr__ respectively.
Attribute lookup in Python is way more complex than this. It's an enormous tar pit, too much so to detail in this comment, but __getattr__ is most often not involved, and the `object` type doesn't even have a __getattr__ method.
> Attribute lookup in Python is [...] an enormous tar pit
Spot on. Python is widely described as a simple language, but the complexity of attribute lookup is one thing that shows that's not true at all.
Many things in Python are easy, such as adding `@property` above a method definition to turn it into a getter. But `@property` is far from simple - the way it actually works is very complex (for example, properties have to be data descriptors, because non-data descriptors cannot override object attributes of the same name).
> Python is widely described as a simple language, but the complexity
> of attribute lookup is one thing that shows that's not true at all.
Python is a simple language to _learn_. My children learned the basics of Python before their seventh birthdays. But Python is not a simple language to _implement_.
It’s an easy language to learn the basics of, sure. But the complexity of things like attribute lookup doesn’t only affect language implementors.
The complexity is all exposed to Python programmers, which makes the language hard to master and, unlike truly simple languages, too large for anyone to understand completely.
There was a time (during Python 2 days) where one could master the language - completely. Today's 3.10 is indeed a very large language; fortunately, one doesn't have to exercise every new feature. And, it's pretty easy to find examples of all features that one might stumble across while reading other people's code.
The really cool thing about this, how descriptors have their __get__ called, is that methods are implemented this way. So when you access instance.method(), it’s a normal lookup for the attribute named “method”, which is (normally) itself a descriptor, so the __get__ magic is called and this binds the method to the instance at the moment it’s needed! Then you can just call it like a normal function. It’s incredibly elegant but extremely obscure. And vital to understand if you want to dive into monkey patching, which is an incredible skill to have!
Good article, thanks. It's worth pointing out that the summary (class hierarchy data descriptor > instance `__dict__` > class hierarchy other) only applies when looking up a normal attribute on a normal object:
* Special-method lookup (e.g. of `__add__` when you do `a + b`) works differently because it doesn't look at the instance `__dict__`, only the class hierarchy.
* Lookup on a class works differently because as well as looking at `__dict__`, it has to consider superclasses.
Much of the complexity relates to the different ways of handling the instance `__dict__`. By contrast, Ruby is able to have much simpler lookup rules because it never considers the instance, only ever the class hierarchy.
> Ruby is able to have much simpler lookup rules because it never considers the instance, only ever the class hierarchy.
“But!” I can hear people say, “I heard you could attach methods to instances in Ruby! How is this possible if Ruby never considers the instance in resolving a method call?”
Well, that’s tricky. The thing is, Ruby instances have no public members, instance variables are private (for direct access, but not in any strict way, because there are public methods called instance_variable_get and instance_variable_get on Object which…do exactly what the names say.)
But every Ruby instance conceptually has (though it doesn’t concretely have one unless you add something to it) a unique class for the instance that is the first thing in its class heirarchy. And you can add methods to that class (the instance “metaclass”, which is different than a Python metaclass) for the effect of attaching them uniquely to the instance itself.
It's a fun dream, but without the ability for a variable to carry along with it a notion of, say, how it implements slice notation, or the ability to avoid verbose names like foo_to_repr(foo) and bar_to_repr(bar) so that names don't collide... arguably we never would have seen the rise of scientific Python! Object oriented programming is an incredibly good abstraction for a lot of real-world scenarios.
Yeah, calling float and complex "supertypes" probably wasn't the best idea, but I couldn't think of a better explanation that wouldn't take too long to explain. I'll ponder about that one.
the getattr thing seems like a huge rabbit hole, I'm totally going to look into this. Thank you :)
Yeah, but we're not talking about pure mathematics. We're talking about floats, and I find that it's very important to be clear about the limitations. It's easy to get some nasty bugs if you start assuming that you can cram just any int into a float.
And I have no objections to the article's description of the bool type.
One subtle point that the post gets wrong:
> So where does that come from? The answer is that Python stores everything inside dictionaries associated with each local scope. Which means that every piece of code has its own defined “local scope” which is accessed using locals() inside that code, that contains the values corresponding to each variable name.
The dictionary returned by `locals()` is not literally a function's local namespace, it's a copy of that namespace. The actual local namespace is an array that is part of the frame object; in this way, references to local variables may happen much more quickly than would be the case if it had to look each variable up in a dictionary every time.
One consequence of this is that you can't mutate the dict returned by `locals()` in order to change the value of a function-local variable.
Another, less-subtle error in the post is this:
> int is another widely-used, fundamental primitive data type. It’s also the lowest common denominator of 2 other data types: , float and complex. complex is a supertype of float, which, in turn, is a supertype of int.
> What this means is that all ints are valid as a float as well as a complex, but not the other way around. Similarly, all floats are also valid as a complex.
Oh, no no no. Python integers are arbitrary-precision integers. Floats are IEEE 754 double-precision binary floating-point values, and as such only support full integer precision up to 2^53. The int type can represent values beyond that range which the float type cannot.
And while it is true that the complex type is just two floats stuck together, I would very much not call it a supertype. It performs distinct operations.
> Accessing an attribute with obj.x calls the __getattr__ method underneath. Similarly setting a new attribute and deleting an attribute calls __setattr__ and __detattr__ respectively.
Attribute lookup in Python is way more complex than this. It's an enormous tar pit, too much so to detail in this comment, but __getattr__ is most often not involved, and the `object` type doesn't even have a __getattr__ method.