Finetuning is pretty much necessary for regression tasks. Also useful for classification since you can get the direct probabilities in case you want to do some thresholding.
From what i read, this post doesnt announce we’ve found some crazy extremophile unicellular microbe. Just that there is evidence to suggest they are there (due to the chemical makeup of soil/boreholes).
Imagine it’s raining, or they come really fast. Even if not so, it is always expected to tip the person doing delivery. That’s just the custom, like tipping in restaurant or tipping the bartender is the custom.
This is the problem. You basically said "we do it like this because that's the way we've always done it," which is the weakest form of justification for anything.
Rain, snow, etc...do you tip the person who delivers your mail? They do it in an LLV (a rather treacherous vehicle with little to no climate control) or on foot, but nobody tips them. When the pizza delivery person applied for the job, they did so knowing they would have to deliver in bad weather, but somehow we reach the conclusion that the responsibility of making sure that driver is being paid adequately for their risk and efforts is shifted to the customer, rather than than their employer.
Now, I should clarify that despite my years of restaurant service where my $2.65/hour paycheck existed nominally for the sole purpose of covering taxes (hence, my "take home" pay coming directly from the customers to my pocket), that I am in the camp of abolishing tipping altogether. Raise the wages of all service workers to a livable wage, which all these companies can certainly afford, and we'd be done with it. But I know that's a huge leap, so we need to take baby steps to get there.
Having a well-defined notion of which positions should be tip-based and which should not is the first baby step.
Great film, but bad scene, honestly. The arguments it makes are intended to make Mr Pink look like the pseudo-intellectual a-hole of the group, rather than be the social commentary on capitalism, labor relations and whose responsibility employee compensation actually is or should be, which is at the crux of any good discussion about the appropriateness of tipping.
I guess what I am getting at with my other comments is that we do not have a clear understanding of said appropriateness, and thus, we, the consumer, along side the food service worker, are generally taken advantage of by the companies that perpetuate the idea while said companies are off the hook for labor costs.
Now, before someone (if anyone is still following this thread) chimes in with "but if the restaurants pay the bartenders/servers a full wage, the food and drinks would be way more expensive!" I am here to tell you "travel more." I have been to many other countries where tipping is not at all a thing, and the food costs about the same as it does in the US.
When you walk into a restaurant in the US, you're getting ripped off. The dish you just paid $16US for cost them about $3 to make, including wage. It's not like the cooks are prepping one dish at a time, or the servers are only taking one table at a time...not to mention most restaurants in the US are using frozen, prepared ingredients that they are really just heating up or re-hydrating. Overhead costs like electricity and rent? A drop in the bucket compared to what small businesses have to deal with. The staff is making bare-minimum wages as it is while the parent companies and investors are making bank. That money from your $16US meal goes up, but very little of it actually comes back down.
Tipping exists because greed at the top exists and its unfair to both food service workers and the customers, but we've been at it so long that it's been normalized. And now it's spreading to other industries, like retail and online sales.
A very wholesome read. Thank you for sharing. I’ve never been so into outdoors/camping/fishing, but it made me reflect on some of my adventure trips I’m doing right now while I’m still young. And maybe these will be talked about in my future family.
I agree. And i think other comments dont understand how utterly difficult this is. I think that there is a translation tool underneath that translates into English. I wonder if it can also figure out binary ascii or rot13 text. Hex to letter would be a very funky translation tool to have
Concat could work too although less efficient because you need to make a new tensor.
Actually summing might learn a concat on its own. Imagine the embedding learned for a token takes up the first N-20 dimensions and leaves the last 20 dimensions as 0. And the positional encoding causes the first N-20 dims to be 0 and the last 20 to encode the information. Then when you sum you are actually concatenating. So I think of them as equivalent except add is more efficient/preserves the dim space, while concat would grow the dim space. And for something like position, which certainly does not need to occupy 1000+ dimensions, it would not make sense to concat all of that since it would be wasteful
Suppose you had a 4096 (llama-2) sized activations. Maybe, you make do with 3084 activations and concatenate 1024 positional activations onto that.
Then you pass that to Mk Mq Mv and generate K, Q, V.
The only thing that would change would be the Mff-out, which would now be a (big)x3084 matrix instead of (big)x4096
In any case you would be retraining, so changing the dims of the tensors I think is not a big deal... In fact in this case they would be smaller (at the cost of fewer interlayer activations), but you would have the same number of tensors.
> Actually summing might learn a concat on its own.
But you see the point? You're forcing the model to learn something that maybe it didn't need to. That's like saying "well a fully connected network might learn convolution on its own". Historically breakthroughs in capability have accompanied one of: [more data | more layers | smarter constraints on activations]
Unless you have some sort of argument that forcing it to learn position has carryover value in generating activations, it seems, naively, a bad idea.
