We mostly use replicated MergeTree, SummingMergeTree, AggregatedMergeTree etc. And yes, we do use Kafka engine to consume directly from Kafka, in additions to standard Clickhouse inserters.
any reason why you guys choose to pipe the compressed content into `tar xzf` process inside the container instead of extracting it outside and overlay the extracted content onto the container via overlayfs or something similar ?
this uses codeMirror as a core editing component
ace is faster and more code friendly than codeMirror. (multiSelect, indentGuides, folding)
but codemirror is smaller and supports variable width and bidirectional fonts
Just tried this out on my setup, and it made a huge difference. A large project that used to take ~1s to load the first time is now pretty much instantaneous.
I tried to write an interface to ImageMagick's MagickWand in Golang as well to solve the same problem. You can find it here: https://github.com/dqminh/mage
Branching can often be a good strategy if you have a large team and you don't want large uncompleted chunks of work to block quick fixes & deploys, etc. The situation you want to avoid is:
1. Engineers A & B are working on a medium-sized story that's not really done yet, but is in master anyway
2. Somebody discovers a small but critical bug in production
3. Engineer C can write a bugfix very soon, but can't deploy it to production because the other story isn't done yet.
But if your team never ends up saying "I'd deploy that but master isn't clean right now due to different changes", you probably just don't need to branch that much. Maybe your team is smallish or maybe you're capable of always breaking up work into really small increments. It really depends on everybody's situation.
Branching is more overhead in some circumstances but I've also found they can make things much easier. But I also wish that more people were comparing strategies and trade-offs because this stuff gets subtle and there are really lots of ways to do it.
We faced this situation before. What we do is:
- When there is a feature that is not done yet, but has some user-facing code, we use feature toggler to disable it on production
- we deploy quick fix by doing a cherry-pick
We discussed about branching a while ago to solve this problem more effectively but still not satisfied with the pros vs cons. For us, branching only makes sense if we switch to use branching completely, and thats quite a big change.
Yeah, personally I know a number of solid devs who like feature toggling but the notion has always rubbed me the wrong way. Especially because half the time I'm adding a non-trivial feature, I'm always semi-refactoring the code underneath to deal with the common patterns and concepts that emerge as a result of that addition. But I'm a big refactoring-as-you-go kind of person--I don't know how common this is.
Branching is definitely a significant investment for a team. It's best done when you've got somebody in-house who can guide everybody through the etiquette and how to use git to best support it. And if you want to do it all the way then you want to support in multiple places in your development stack -- QA servers, staging servers, CI, etc.
But for large units of work that genuinely should stay out of master -- large user-flow rewrites, big code refactorings or database migrations -- I find it a great way to have significant experimentation off to the side and then bring it back into master when it's ready for prime-time.
this was a question I had - how do you have your QA,testing,staging setup to test branches ? Or is fast and frequent commits (by design) eliminate the need for the commit-deploy-test cycle and replace it with a review step instead ?
Generally speaking I like to have at least one CI environment available for every branch that makes you nervous, plus master. So what number you're dealing with depends on the team. Sometimes you have eight engineers but only four of them are doing something major at any given time.
Of course, whatever you call what you deploy to production -- master, production, timestamp tagging, etc -- it should ideally pass CI before getting deployed as well. And there are merge conflicts to deal with, too -- that ideally belongs to whoever's doing branching.
So, maybe my ideal workflow looks like this:
* Branch away from master
* As you work in your branch, grab a CI server if you feel like it (telling
everyone else)
* Commit code in your branch, keeping an eye on your CI as you go
* Periodically merge in from master, dealing with merge conflicts in your branch,
*not* in master
* When you feel like it's ready to go, get whatever review you feel like you
need -- code review, product manager review, etc
* If you feel 99% certain it's good, merge it into master and push that back to
origin
* Wait for master CI to pass
* Deploy to production
* Beer
From an etiquette point of view, there are things that can go wrong, not least the individual commitment to deal with merge conflicts before they get involved in master. And conflict resolution cannot be rushed. If you have engineers who are just gonna edit files randomly to get git to accept their merge, you're going to have problems. (But if you have those sorts of engineers, you already have problems.)
Out of curiosity, do you work for a relatively small company? I ask because I can't imagine this scaling particularly well, with production code containing a large number of "dead" code paths that you have to cross reference with your feature toggler to check on their status.
Do you feel like you gain anything significant from your method (apart from the simpler git interaction from having just one developer branch)?
A local git branch is super-cheap, so there's hardly any downside to using them. The upside, even for a single developer, is the ability to work on multiple features/bugfixes simultaneously, and merge them into master only when they're ready.
Where I work, there are several different git styles, but they all work nicely together if everyone adheres to one simple rule. I'm going to say it loudly, because it's important:
origin/master must be deployable.
If that means developing in a branch, fine. If that means accumulating several patches locally in master that you haven't pushed yet because you're not done, fine. Just make sure that origin/master can always be deployed to the servers.
Actually, come to think of it, both of those things are pretty much equivalent to using branches for anything non-trivial.
JS/Ruby's low-fi version of using hashes actually helps there as you have to provide hash arguments via the hash.
Python 3 lets you fix this issue (a great reason to switch, by the way) by using a ⭑[0] argument (not a ⭑-arg, which you can use in Python 2 but which will simply make your stuff fall into a black hole unless the creator of the function asserts no positional argument was passed via e.g. `assert not arg`; furthermore Python 2 would require that a default value — to check for — be given for each keyword argument):
>>> def foo(*, a, b):
... print(a, b)
...
>>> foo()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: foo() needs keyword-only argument a
>>> foo(1, 2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: foo() takes exactly 0 positional arguments (2 given)
>>> foo(a=1, b=2)
1 2
2. the arguments are separate from the method name in Python, in your example the method is `do_send_mail`, in Smalltalk and Objective-C it's `doSendMailToRecipient:cc:subject:body:` which has advantages and inconvenients. Python is easier for default values (you need a new method in Smalltalk or Obj-C), but the Smalltalk/Obj-C method is simpler for behavioral differences (as the dispatch is done at callsite, Python will need internal soup)
3. There is no order in Python's argument (whereas changing the order in Smalltalk/Obj-C changes the method called), which means callers can lower the call's readability by swapping arguments around in... less than sensible ways.
(nb: I love Python, don't interpret my comment as a put down of it, just pointing out fundamental and important difference between Python's keyword arguments and Smalltalk's keyword messages)
thanks for the point. I have no background in Python so I didn't take Python into consideration. In my opinion, there are more people like me: coming from pure HTML/CSS/PHP background straight to Objective C
For what it's worth, the OpenBSD team is working on improved buffering for high-RAM machines:
"Computers today have immense quantity of unused memory. This new software almost works, we still have some minor bugs to fix, but the results so far are incredible. On computers with 16GB of RAM we can dedicate as much as 13 GB to buffering and almost no data is read from the hard drive, everything is in the RAM. It performs even faster than SSDs. We only have to write to the disk anymore and that’s mainly for reliability reasons. I estimate we’re a year away from the whole thing working flawlessly. From this aspect the 6 month release cycle is a bit limiting."
Pretty sure that's wrong except in contrived disk-heavy workloads. RAM needs power in proportion to its storage size.
My 2009 MBP, which I immediately upgraded to 8GB RAM on purchase, has never come close to the stock advertised battery life. And, supposedly a reason Apple optimizes iOS devices for such small RAM sizes (sizes which are never emphasized in published descriptions) is that minimizing RAM is an important factor for their long battery life.
