What I've noticed, and personally experienced, about these porting projects is that they're generally a premature optimization but are a great way to learn a new language. The gains from a port are largely intrinsic in nature, residing with the programmers. However, systems and operations largely carry on just fine with the original language chosen.
Only one story comes to mind where a programmer had a legitimate problem using a high-level language and solved it with a lower-level language, and that was Armin Roacher, who solved a significant performance issue using Rust: https://blog.sentry.io/2016/10/19/fixing-python-performance-...
Can you think of other stories? Please, share.
I encourage all to be more skeptical about potential gains and realistic about who is actually gaining.
Developers have the gain, company suffers, developers move on for higher salary with more skills, CTO is in it, doesn't fullfil his duty towards the company or investors, CEO has no clue and doesn't manage the CTO.
The story I see over and over again as a startup CEO consultant.
Developers learn a new language that is faster to develop, safer and faster to run than existing solutions (which work but are of increased technical debt) and successfully & quickly port the existing solution to the new tool.
The company doesn't really benefit from it apparently because they never scale enough or because they lacked a target so they had their developers working on sth small instead on working on the company's products.
This skill is highly valued and the developers leave for an other company while the original one struggles to continue.
Do you think that it may be the case that the problem in all these is not within the developers or tool? Furthermore, shouldn't the CEOs and the CTOs know better and to a better job?
> Developers learn a new language that is faster to develop, safer and faster to run than existing solutions
Would that it was that simple. Most of the time I've seen this it's been more like “writing new code is fun, fixing a bad design in our current code is too much like work” without factoring in the cost of rewriting, testing, optimizing all of the code which wasn't a problem relative to simply replacing the hotspot which was.
Yes, sometimes a codebase is so bad that there's nothing of value to be preserved but that's pretty rare and unless it's some turd from an acquisition or consultants the odds are extremely high that the broken business culture responsible for the first bad codebase will cause just as many problems the next time around.
That does happen but in my experience it's far less common than fanboyism or poor architectural/analytic skills — things like spending months rewriting a Python program in Go for a single-digit percentage gain because someone on HN said the magic pixie dust would make it faster and nobody checked whether it was I/O bound, spending most of its time in OpenSSL, etc. I've seen people propose rewriting code in C rather than optimize a database query because Real Programmers use hard things like C whereas SQL is beneath them. (I wish I was making that up)
To be clear: I'm not saying that Go isn't a perfectly fine choice, only that our field is more prone to fads than we like to admit and most of the things which cause bugs or make people wait are decisions higher in the stack.
I understand the situations you are describing and I still have the impression that the main problem behind this is inadequate leadership (and probably at several layers...).
There are more than enough Real Programmers or lacking soft eng who are not experienced or knowledgeable enough to know when to use C/python/go/sql/etc but again for me the problem is either whover hired them and whoever is managing them.
I agree that the main problem is leadership — and arguably mentorship as well. As a field we're too predisposed to view all problems as technical and ignore the social factors which lead to the visible technical symptoms.
Well, the "in the eye of ... with no proper evaluation" argument can be had for a lot of other aspects of software development and product management, doesn't it?
Hell, I've seen this as a lead/senior engineer. I'm seeing it right now, in fact, as a person on our team wants to write some stuff in language X, which nobody else on the team has experience with, and which isn't used anywhere else in the company, and to solve a problem that is perfectly well solved (in both development time and performance) with the language and tooling we already use.
As far as I can tell this engineer just wants to stick "built stuff in X" on his resume. He doesn't have a good reason to not use our existing stack. If I were the manager I'd demand more technical justification.
But this also might be a problem of a company not providing any improvement / career prospects for your programmers.
If working as a programmer means sticking to C# or Visual Basic or Python codebase and just working on adding new features or new business logic to your application for years and years, many engineers will not find that fulfilling. The best engineers will always want to learn about new tech and approaches (dev ops, containers, PaaS, automation, new languages and tools, AI / machine learning etc).
You cannot blame them for leaving. What you should instead do is provide interesting challenges for them, if they want to use some new technology, work on finding a justifiable use case that management will approve for it.
I understand that for a company that treats tech as a cost centre this approach makes sense but then they can't wonder when best engineers keep leaving to proper tech companies that do actual R&D and where they can become best engineers they can be.
You need to allow your best programmers to learn new tech and progress in their career or else they'll eventually leave for greener pastures and you'll be left with mediocre people who are content and don't want to learn new things.
It's like that analogy I have heard over and over again.
CFO asking CTO: What happens if we invest in developing / training our people and they leave?
CEO replies: What happens if we don't and they stay?
This has nothing to do with whether engineering is a "cost center" or not. It's about fundamentally what is important about the practice of engineering. Whether to use Python or Go or Java or Rust or whatever is rarely anything other than bike shedding. In those cases where it's not, the key problems are rarely which language is technically better: more often the problems are associated with labor supply and how well the proposed language' ecosystem plays with existing implementations'.
To be fair I've been on the underling side of that where I wanted to build some new server we needed in a new language (coincidentally, Golang), but it was honestly just for curiosity/shits&giggles. It became somewhat of a running gag for ~5 years where every new service we needed I'd suggest Golang.
I'm a senior engineer and I'm also the person that wants to use the shiny new thing.
I'm just learning Go and I'm itching to find a project to use it on. It's super hard to justify it though when C# or Python would work just as well and those are languages that we already use.
The best time to do it is on a new project or component of an existing system, in my opinion. It doesn't make it a good idea, but it can be a nice way to introduce something new and possibly more productive.
How about this: once you write code in Go and test it well, you can deploy it and forget about it unless there is a hardware issue? I recently ported some C++ code to Go which processes 8 Billion events per day flawlessly.
Once you write a well designed, well tested, and feature complete piece of software, you can deploy it and forget about it unless the server it's running on breaks.
Go isn't special in that regard, unless there's something that makes Go easier to write, test, or deploy, which might be the case, but you haven't supported that.
Our team was able to develop/deploy about 20 Microservices in Go in the past year or so which is really awesome. I can say this after having worked with several other languages.
I ported a Groovy/Grails app to Java/SpringMVC once.
My technical motivations were more about moving away from Hibernate. Arguably there were plenty of reasons for leaving it as it was, though -- if I had known enough to be able to solve certain data access complexities back then, Grails was in many ways a better fit for the problem. The experience in a (more-or-less) functional language wouldn't have hurt any either.
The business motivation was simple: Java/SpringMVC programmers are (or were then) far more plentiful, so adding/replacing tech resources was made much simpler.
This example actually seems to be the opposite of an optimization. There's 50% more code to maintain and there appears to be almost no appreciable benefit to compensate for that.
LOC is a bad measure for the complexity of code. Quite the opposite, expressive code can be faster to understand and easier to maintain than the same functions with half the code size.
In any case, switching to a compiled language and static types should bring some benefit, in addition to the gain in speed.
IME 95% of the time SLOC approximates code complexity just fine. Just because you can write horrendous 'clever' one liners in perl does not mean that most code out in the wild is like that. I certainly don't think the same author writing the same app in two different languages will be writing like that.
Zero mention was made of bugs caught or customer-noticeable speed improvements so I'm inclined to think that there was actually no benefit to having static typing.
No, there was no appreciable user benefit, and that was intended. To respond to others: it wasn't a "premature" optimization, because it wasn't any kind of optimization. The sole benefit in this case was for me to learn Go programming -- and I noted that in the article title and intro. :-)
No-one goes out of their way to choose a slower language. People who choose to write something in Python are getting value out of it, or at least believe they are; if writing the program in Go will be more costly (in development time, defect rate, library availability or something else) than writing it in Python then writing it in Go is premature optimization, and if writing it in Go wouldn't be more costly than writing it in Python then why were you thinking of writing it in Python at all?
IME thinking about language performance at all is premature optimization; language performance almost never makes the difference, other between-language differences will swamp any gains from language performance.
> Are you serious? Python code is about 10 times slower than equivalent code in Go.
And 98% of the time it doesn't matter. For most business problems that you'd want to solve with a computer, for modern hardware, Python's performance is more than adequate. It's really not worth worrying about.
No. This is only true if you have a script you run once an hour or so.
If you have a server side application that you hope to be successful then the code will be running all the time and you will pay the price literally because cloud hosting services charge for CPU and RAM usage, which will be off the charts if your server is in Python.
Not to mention all the time you will spend firefighting the performance issues that will inevitably arise. Which also literally costs money in terms of developer salaries and opportunity cost (you can't spend that time to develop new features).
> If you have a server side application that you hope to be successful then the code will be running all the time and you will pay the price literally because cloud hosting services charge for CPU and RAM usage, which will be off the charts if your server is in Python.
Your hosting fees are a tiny fraction of your expenses. Development time is worth a lot more.
> Not to mention all the time you will spend firefighting the performance issues that will inevitably arise. Which also literally costs money in terms of developer salaries and opportunity cost (you can't spend that time to develop new features).
Scaling is a nice problem to have. If you get to the point where you actually need to improve performance, you'll be able to afford to spend time on it.
> Your hosting fees are a tiny fraction of your expenses.
Only when your requirements are low.
Things get very expensive when you start requiring real performance.
> Scaling is a nice problem to have. If you get to the point where you actually need to improve performance, you'll be able to afford to spend time on it.
You don't know that. If you're a startup, it's very likely that you're still not profitable at that point.
But the code doesn't have to become more complicated for it to be premature optimization. It literally just means improving something (generally performance) before there's a clear need.
> Writing the same code in a fast compiled language is just the default thing that you should be doing when you are not doing premature optimization.
I assume you mean "rewriting" here, since we're talking about porting existing code. Sometimes it's the best use of time, sometimes it's premature.
Remember the context here. The idea is "premature optimization is evil". A lot of people often abuse the quote and then conclude "Caring about performance is bad" which is a complete non-sequiter and not the originally intended meaning of the quote.
The reason premature optimization is evil is that it consumes a lot of time and makes the coder more difficult to read and reason about.
If there's no clear benefit to this, then all the work done to optimize it resulted in _negative_ value. There was no gain, only time wasting and more complicated code.
An example of where performance doesn't matter: a piece of code is only executed once every 10 seconds and it finishes executing in 5ms. Spending time to reduce its execution time to 1ms does not produce any tangible benefits.
An example of where performance does matter: a piece of code is running all the time and it takes 500ms. If you can reduce it to 20ms, it's totally worth it.
In Go it's impossible to make for...range support user-defined types, and you have to write out every loop the hard way (and rewrite any loop that was using a builtin type). In Python you just define __iter__. That's what's meant by expressive, you can write what you mean instead of having to explain yet again how it should be implemented.
Of course they're different on that level of detail. In go, if you want programmable iteration behavior, you would create a channel that is fed values by a coroutine. Then you can range over it. It's not a new type, but that distinction seems a bit picky.
What I am talking about is the static type system. You can't express that in Python, despite how much more convenient some of its features are.
I agree that writing in a fast, compiled, statically typed language is not necessarily premature optimization. It is my first choice for a new project in the absence of any other mitigating factors.
There are often mitigating factors, though.
And switching from an existing language to use a new one that is possibly more optimized carries costs that have to be considered. If the new implementation shaves a few percent off some time or other resource scale, and the sum of that savings over the life of the program is more than the cost to do development and maintenance it's worth it. But that often isn't the case.
Let's say that C produces 100% of the performance available. How much does Go produce? Maybe 90%? How much does Python, say, produce? 25%? So going from Python to a compiled language produces a huge gain, and going from Go to C produces a little bit more.
But why not get all the gain? Well, sure, if performance is the only thing you care about. But it usually isn't. You might also care about networking, or multithreading. Would I rather write that code in C, or in Go? Is it enough better in Go to be worth that last 10% of performance? Arguably, yes.
When someone says "X matters", it doesn't mean that they are saying "X is the only thing that matters". It is not correct to conclude that they should, for consistency, be saying "turn the X knob as far as you can". Instead, interpret it as saying "given your other constraints, look for a sweet spot where X is somewhat toward this end of its range."
Yes, that's my point. It is premature optimization, because you do not have a use case for such performance. Instead you might want Python's extensive numerical libraries, or you enjoy its quick prototyping capacities. The same way that you do not always go for C, and sometimes prefer Go, because of non performance related requirements.
That's why saying a compiled performant language should be default is not logic I understand. Default should be whatever you need for your requirements. If you don't need anything special, just go with what you already know well.
Fine, but if that's your point, don't say "Why not use C all the time by default then?" Actually make that point, rather than leaving us all guessing as to what your point is, and more important, why it's your point.
And, as I said, "Why not use C all the time by default then?" does not actually follow as the logical endpoint of the argument that your trying to answer, so making that your response really doesn't contribute much to the advancement of the conversation.
Using a compiled language is not a premature optimization if you're not sacrificing expressiveness.
With that said, Go is not the best choice. Like, for anything.
That was just a harmless (I hope) hyperbole.
I've actually used Go a couple of times for image processing workers and such. And that's when I developed my distaste for this language.
I mean, I get it, it's easy to pick up and use. Coroutines and channels are useful, and make easy to write async stuff.
But god damn, it's so dull and boring, there's no joy in it. We're programmers, we're supposed to extract abstractions and express them in the code we write. And IMO your language should definitely try to help with that.
Instead, Go wants you to do err, value = ..., if err == nil {
... } on almost every single line.
You can't adequately compose stuff. Well, you can try, but you quickly run into its ugly, verbose syntax for anonymous functions or lack of generics.
Maybe I'm not getting something, maybe it's just not for me, I dunno.
But next time I need high performance compiled language, I'll use Swift or maybe even Rust. Also I think Scala Native is somewhat stable already, gotta look into that.
OP was saying porting a project to Go sounds like premature optimization. The alternative is obviously to continue to use the original language.
The question to be answered is why are you porting it? Fun and giggles? Hoping to achieve better performance? Scale? Portability? Correctness? Simplicity? Support?
I'd say most of the time its for fun and giggles. Which I think is as fine a reason as all the others, but when its costing someone's elses money, maybe you should be more considerate.
Ya, maybe for some. I think for businesses part of it is familiarity, as its now a lot of people's first language. I also think its a choice made often when you start, because it allows you to go faster in the beginning. Which is where a lot of new projects need to be the fastest. Its still a great choice for projects that are short lived, that's why ML works so well on it, most of ML is just quick iterations on prototypes until you find a solid model.
Depends. If we're talking about high performance, yet expressive language without a steep learning curve, I say Swift.
The only major drawback right now is lack of pure Swift AWS SDK. And unfortunately, AWS team has no plans to write one, yet.
The most general answer is OCaml, though in practice I'd probably use something else depending on the details. For most cases there's a better pick than OCaml, but I don't think OCaml is ever a worse choice than Go.
Only one story comes to mind where a programmer had a legitimate problem using a high-level language and solved it with a lower-level language, and that was Armin Roacher, who solved a significant performance issue using Rust: https://blog.sentry.io/2016/10/19/fixing-python-performance-...
Can you think of other stories? Please, share.
I encourage all to be more skeptical about potential gains and realistic about who is actually gaining.