Everything you said is true for both of our programs, the only difference is whether or not it's hidden behind function calls you can't see and don't have access to.
You don't really think that functional languages aren't appending things, using temp vars, and using conditional logic behind the scenes, do you? What do you think ".filter(node => !node.isHidden)" does? It's nothing but a for loop and a conditional by another name and wrapped in an awkward, unwieldy package.
>which introduces a performance regression from having to periodically grow the array
This is simply ridiculous, do you just believe that the magic of Lisp/FP allows it to pluck the target variables out of the sky in perfectly-sized packages with zero allocation or overhead?
> the only difference is whether or not it's hidden behind function calls you can't see and don't have access to.
You "can't see and don't have access to" `if`, `range`, or `append` but somehow you don't find this a problem at all. I wonder why not?
> You don't really think that functional languages aren't appending things, using temp vars, and using conditional logic behind the scenes, do you?
By this metric all languages that compile down to machine instructions are equivalent. After all, it winds up in the same registers and a bunch of CMP, MOV, JMP, and so on.
`.distinct()` could sort the result and look for consecutive entries, it could build up a set internally, it could use a hashmap, or any one of a million other approaches. It can even probe the size of the array to pick the performance-optimal approach. I don't have to care.
> [".filter(node => !node.isHidden)" is] nothing but a for loop and a conditional by another name and wrapped in an awkward, unwieldy package.
This is honestly an absurd take. I truly have no other words for it. map, filter, and friends are quite literally some of the clearest and most ergonomic abstractions ever devised.
Speaking of filters and clear ergonomic abstractions, if you like programming languages with keyword pairs like if/fi, for/rof, while/elihw, goto/otog, you will LOVE the cabkwards covabulary of cepstral quefrency alanysis, invented in 1963 by B. P. Bogert, M. J. Healy, and J. W. Tukey:
> it could build up a set internally, it could use a hashmap, or any one of a million other approaches. It can even probe the size of the array to pick the performance-optimal approach. I don't have to care.
Well, this is probably why functional programming doesn't see a lot of real use in production environments. Usually, you actually do have to care. Talk about noticing a performance regression because I was simply appending to an array. You have no idea what performance regressions are happening in ANY line of FP code, and on top of that, most FP languages are dead-set on "immutability" which simply means creating copies of objects wherever you possibly can... (instead of thinking about when it makes sense and how to be performant about it)
Your assumption that filter/map/reduce is necessarily slower than a carefully handcrafted loop is wrong, though. Rust supports these features as well and the performance is equivalent.
Also countless real-world production environments run on Python, Ruby, JS etc, all of which are significantly slower than a compiled FP program using filter & map.
> FP languages are dead-set on "immutability" which simply means creating copies of objects
Incorrect. The compiler can make it mutable for better performance, and that gives you the best of both worlds: immutability where a fallible human is involved, and mutability where it matters.
> The compiler can make it mutable for better performance
Well, we already know that no pure FP language can match the performance of a dirty normal imperative language, except for Common Lisp (which I am happy to hear an explanation for how it manages to be much faster than the rest, maybe it's due to the for loops?). And another comment here already mentioned how those "significantly slower" scripting languages have a healthy dose of FP constructs -- which are normally considered anti-patterns, for good reason. The only language that competes in speed in Rust, which just so happens to let you have fast FP abstractions so long as you manually manage every piece of memory and its lifetime, constantly negotiating with the compiler in the process, thereby giving up any of the convenience benefits you actually get from FP.
You complain about not having control of how the `filter` works under the hood but are happy to give the language control over memory management. How much abstraction is too much abstraction? Where do you draw the line?
I consider an abstraction to be a good abstraction if I don't need to care for its internal workings all the time - whether that's some build step in the CI, or the usage of data structures from simple strings to advanced Cuckoo filters and beyond. Even Python uses a Bloom filter with operations on strings internally AFAIK. Correctness and maintainability trumps performance most of the time, and map, filter and immutability are building blocks to achieve just that. Those constructs won't prevent you from looking deeper and doing something different if you really have to (and know what you're doing), they just acknowledge that you'll probably don't need to do that.
> Correctness and maintainability trumps performance
Great, please make all the software even slower than it already is. I am overjoyed to have to purchase several new laptops a decade because they become e-waste purely due to the degradation of software performance. It is beyond ridiculous that to FP programmers daring to mutate variables or fine-tune a for loop is an exceptional scenario that you don't do "unless you have to" and which requires "knowing what you're doing". Do you know what you're doing? How can you be a software engineer and think that for loops are too difficult of a construct, and that you need something higher-level and more abstract to feel safe? It's insane. Utterly insane. Perhaps even the root of all evil, Code Golf manifested as religion.
False equivalence. You're saying that the statement "both for.. append and .map() executing the _same steps_ in the _same order_ are the same" is equivalent to saying that "two statements being composed of cmp,jmp, etc (in totally different ways) are the same" That is a dishonest argument.
> Distinct could sort and look for consecutive, it could use a hashmap
People love happy theories like this, but find me one major implementation that does this. For example, here is the distinct() implementation in the most abstraction-happy language that I know - C#
This does the usual hashset based approach only if the array has strings. Otherwise, it gets the comparator and does a sort | uniq. So, you get a needless O(nlogn), without having a way to distinct() by say, an `id` field in your objects. Very ergonomic...
On the other hand...
seen := map[string]bool{}
uniq := []string{}
for _, s := range my_strings {
if !seen[s] {
uniq = append(uniq, s)
seen[s] = true;
}
}
Let us say you want to refactor and store a struct instead of just a string. The code would change to...
seen_ids := map[string]bool{}
uniq := []MyObject{}
for _, o := range my_objects {
if !seen_ids[o.id] {
uniq = append(uniq, o)
seen_ids[o.id] = true;
}
}
Visually, it is basically the same, with clear visual messaging of what has changed. And as a bonus, it isn't incurring a random performance degradation.
I'm not sure reaching for PHP to dunk on functional languages is the win you think it is?
> Visually, it is basically the same, with clear visual messaging of what has changed.
In order to do this you had to make edits to all but a single line of logic. Literally only one line in your implementation didn't change.
Compare, with Ruby:
# strings
uniq = strings.uniq()
# structs, if they are considered equal based on the
# field in question (strings are just structs and they
# implement equality, so of course this is the same)
uniq = structs.uniq()
# structs, if you want to unique by some mechanism
# other than natural equality
let uniq = structs.uniq(&:id)
With Rust:
# strings
let uniq = strings.unique();
# structs, if they are considered equal based on the
# field in question (strings are just structs and they
# implement equality, so of course this is the same)
let uniq = structs.unique();
# structs, if you want to unique by some mechanism
# other than natural equality
let uniq = structs.unique_by(|s| s.id);
You cannot tell me with a straight face that your version is clearer. You'll note that both languages have essentially the exact same code, which is to say: nearly none at all.
> Distinct could sort and look for consecutive, it could use a hashmap. It can even probe the size of the array to pick the performance-optimal approach. I don't have to care.
which says that you just use an abstraction and it transparently "does the right thing". The C# example showed how abstractions actually don't do that, and instead simply provide a lowest common denominator implementation. The rust examples you used also uses a hashmap unconditionally. The PHP example was showing how when abstractions attempt to do that it ends up even worse - when you move from strings to structs, you get a slowdown.
In practice, different strategies are always implemented as different functions (see python `moreitertools` `unique_justseen` and `unique_everseen`), at which point its no longer an abstraction (which by definition serves multiple purposes) and it just becomes a matter of whether or not the set of disparate helper functions is written by you, the standard library, or a third party. In rust, you would do `vec.sort()`, `vec.dedup()` for one strategy, and call `v.into_iter().unique().collect()` for the other strategy. That is not an "abstraction" [which achieves what you claimed they do].
> It unconditionally uses a hashset regardless of input.
This follows what I like to call macro optimization. I don't care if a hash set is slightly slower for small inputs. Small inputs are negligibly fast anyway. The hash set solution just works for almost any dataset. Either it's too small to give a crap, midsized and hash set is best, or huge and hash set is still pretty much best.
That's why we default to the best time/space complexity. Once in a blue moon someone might have an obscure usecase where micro optimizing for small datasets makes sense somehow, for the vast majority of use cases this solution is best.
> Everything you said is true for both of our programs, the only difference is whether or not it's hidden behind function calls you can't see and don't have access to.
This is a key difference between imperative programming and other paradigms.
> You don't really think that functional languages aren't appending things, using temp vars, and using conditional logic behind the scenes, do you?
A key concept in a FP approach is Referential Transparency[0]. Here, this concept is relevant in that however FP constructs do what they do "under the hood" is immaterial to collaborators. All that matters is if, for some function/method `f(x)`, it is given the same value for `x`, `f(x)` will produce the same result without observable side effects.
> What do you think ".filter(node => !node.isHidden)" does?
Depending on the language, apply a predicate to a value in a container, which could be a "traditional" collection (List, Set, etc.), an optional type (cardinality of 0 or 1), a future, an I/O operation, a ...
> It's nothing but a for loop and a conditional by another name and wrapped in an awkward, unwieldy package.
There is something to be said for the value of using appropriate abstractions. If not, then we would still be writing COBOL.
> You don't really think that functional languages aren't appending things, using temp vars, and using conditional logic behind the scenes, do you? What do you think ".filter(node => !node.isHidden)" does? It's nothing but a for loop and a conditional by another name and wrapped in an awkward, unwieldy package.
But the whole point of higher-level languages is that you don't have to think about what's going on behind the scenes, and can focus on expressing intent while worrying less about implementation. Just because a HLL is eventually compiled into assembler, and so the assembler expresses everything the HLL did, doesn't mean the HLL and assembler are equally readable.
(And I think that your parent's point is that "awkward, unwieldy package" is a judgment call, rather than an objective evaluation, based, probably, on familiarity and experience—it certainly doesn't look awkward or unwiely to me, though I disagree with some of the other aesthetic judgments made by your parent.)
You don't really think that functional languages aren't appending things, using temp vars, and using conditional logic behind the scenes, do you? What do you think ".filter(node => !node.isHidden)" does? It's nothing but a for loop and a conditional by another name and wrapped in an awkward, unwieldy package.
>which introduces a performance regression from having to periodically grow the array
This is simply ridiculous, do you just believe that the magic of Lisp/FP allows it to pluck the target variables out of the sky in perfectly-sized packages with zero allocation or overhead?