I think what many people are finding out is they don’t really need distributed processing. DuckDB on a single node can get you really far, and it’s much simpler.
On a computer, just uBlock Origin and it works wonders.
Tangential question - what is the best solution for iPhone? On Androids you can use Firefox with uBlock, but it seems none of the Safari extensions on iPhone actually work, I tried some paid ones too. Brave seems to work decently well, but I have no idea why - if other browsers have some OS limitation, how does Brave go around it?
A huge downside of writing this down on paper is team collaboration. Imagine that instead this is kept in a tool like Notion which the entire team can access. At first this feels incredibly uncomfortable - your notes are there for everyone to see. However it’s a massive force multiplier because often the work is picked up by someone else later on, and instead of asking you they can self-serve. It’s pretty common that by the time they need it you would no longer be at the company.
I don’t have experience writing in a lab journal format, but for documents like growth experiments and how they worked, or RFCs, this is a godsend. It takes a lot of work to keep it tidy, but it’s worth it.
Absolutely not. My working / lab notes capture raw thoughts, often abbreviated and with shortcuts taken. Others would have trouble gleaning much meaning from them, with so many scratched out words, wildly drawn arrows, and peculiar diagrams.
Take my raw notes, rewrite them for another audience, and post those on Notion? Maybe. But it's likely a different document (an essay, tutorial, FAQ) could be rewritten out of my notes, one which would be more useful for my teammates.
I agree with having shared team documentation but this has always been gray for me. My notebook ( and my softcopy log for the week) are incredibly raw (one-lines through mistakes, my personal commentary/snark, "napkin math",etc.). I am willing to share them with teammates/our lead/ or whoever may benefit but this has backfired on me. We perform analytical verification work and I've seen middling performers on our team look to avoid performing thorough work by copying from the notes of someone else ( I get this is work and not school so typical academic rules don't apply but our work is very investigative/exploratory in it's nature, performing derivations to ensure our approach is correct is encouraged and not considered a waste of time. Also contexts change quickly and a deep understanding is necessary) Thus when someone is trying to take a shortcut it leaves a bad taste in my mouth. Also "micro-managers" see pages of notes open and treat it as fair game to pull up to my desk and expect immediate updates instead of requesting reasonably or waiting for the appropriate forum. I get this is a culture/staff issue more so than other things.
I maintain a daily email to myself in a shared mailbox. I have 3 sections:
SIGNIFICANT
\* topic1
\* this happened
\* topic2
DONE
\* This project
\* Made this change
\* Made another change
\* Some training
\* Completed this section
TODO
...
I'm in a setting where I'm incredibly temporary. I could be tasked elsewhere tomorrow. Every day I reply to my previous email and work on the draft throughout the day as my notebook. At the end of the day I send it, received in the mailbox I'm attending. I title the email "Captain's Log" and my supervisor and peers can read it, as well as the draft, whenever. This keeps them clued in on where my head is at, what I'm working on, etc. Great for performance reviews mostly. Not as convenient as something like my Remarkable tablet.
> instead of asking you they can self-serve. It’s pretty common that by the time they need it you would no longer be at the company.
All lab notebooks are company property. You don't keep them locked away in your desk or take them with you when you leave. Any current or future employee with the right clearances should be able to serve themselves to the entire archive. No need to ask the original author.
Searchability is by far the most significant advantage of electronic documents, but you can get pretty far with paper notebooks if you keep a decent table of contents. Even better if you regularly scan, OCR, and upload your pages to the archive. IMO the inconvenience of scanning is greatly outweighed by the paper notebook's guarantee of immutability. Tools like Notion make it too easy to erase information, either accidentally or not.
This is a render-unto-Caesar problem. It all depends on the audience. Here's what works for me.
1. Email is good for sharable things like interview or meeting notes. It's searchable and you can dump it to Google Docs/Notion from the draft if that's appropriate. Using Gmail keeps you from trying to format it too much while writing.
2. My lab book is for me. Writing with pen/paper forces one to sort out ideas up front--not a lot but just enough for them to make sense and be readable later.
It seems as if everyone will have a different take on this.
I agree that it’s useful for the rest of the team to have access to your notes, but I find that it’s far more useful to write my notes with a pen and paper rather than typing them out. It forces me to slow down and think through what I’m doing, in addition to aiding in memorizing the parts of my notes that I find important.
The important point is that the raw notes are for the note-taker. They not only document what you thought, they are themselves a part of and tool for the thinking process. Writing is thinking[1], and the initial results aren't going to be very useful for someone else, because they'll be rambling and messy, with lots of false starts, errors, and just wrong ideas. They're for the author to remember the path to the outcome[2], they are not the outcome itself.
For the rest of the team, those raw notes are source material. That can be the input into lightweight documents like technical memos and decision records.
There's an important thing that is easy to miss if you just use a Google Takeout for the export – your Shared albums will only contain pictures that you yourself added!
I went through this journey a few months ago, and it's pretty hard to catch because those albums will appear in the export, and they are not empty, so you don't expect that they are partial exports. The way to export them was going one by one in the regular Google Photos app, and downloading each album as a ZIP – that way you get pictures also from other contributors to that album.
Funnily enough, I also experienced some rate limiting and had to wait a few minutes after every 6-7 albums.
My end goal was importing the Takeout into Apple Photos library locally stored on my Mac. Some other steps I had to take were:
- Fixing the metadata [0] so they showed up with a correct date
- Importing albums first, and the "Stream" (a folder per year) second, because otherwise the deduplication would mean the pictures already in the stream wouldn't get added to an album.
In a Google Photos shared photo album, there is the button to "save photos [to your library]". I would expect that you need to click that in order to have those pictures being contained in your Google Takeout and your library. Did you do that?
> going one by one in the regular Google Photos app, and downloading each album as a ZIP
But be careful here. Some files will only be available at reduced quality and have all metadata stripped (even if the owner has elected to share it and it is visible in the web UI). The only way to get the original seems to be to contact the original uploader.
Let's consider an example that would be a case of Simpson's Paradox. Suppose you are A/B testing two different landing pages, and you want to know which will make more people become habitual users. You partition on whether the user adds at least one friend in their first 5 minutes on the platform. It might be that landing page A makes people who add a friend in the first 5 minutes more likely to become habitual users, and it also makes people who don't add a friend in the first 5 minutes more likely to become habitual users. But page A makes people less likely to add a friend in the first 5 minutes, and people who add a friend in the first 5 minutes are overwhelmingly more likely to become habitual users than people who don't. So, in this case at least, it seems like the aggregate statistics are most relevant, but the fact that page A is bad mainly because it makes people less likely to add a friend in the first 5 minutes is also very interesting; maybe there is some way of combining A and B to get the good qualities of each and avoid the bad qualities of both
With random bucketing happening at the global level for any test, the proper thing to do is to take any segments that show interesting (and hopefully statistically significant) results that differ from the global results and test those segments individually so the random bucketing happens at that segment level.
There are two issues at play here -- one is that the sample sizes for the segments may not be high enough, the other is that the more segments you look at , the greater the probability for finding a false positive.
It can only happen with unequal populations. If you decide to include people in the control or test group randomly you're fine (you can use statistical tests to rule out sample biad).
So I can get a better sense of how to use this tool, if I wanted to see examples of say a 'Users' collection API from multiple Django projects, how would I do so? Looks like I can do keyword matches and language specific filtering but would love tips beyond that!
Awesome! I sometimes search across very niche projects on Github, where their codesearch doesn't show the results I want. I KNOW that the result exists, and can see it in the repository, but their codesearch doesn't show the results I want.
