I'm fine with forks but hard to see @jeffail's marketing/persona of Benthos forked too. It was quirky, to say the least. Difficult line to walk, forking code brings repo docs but it feels insincere reading not on the original.
Personally, I'm happy to see them evolving it with my juices left intact. When we transitioned benthos.dev to https://docs.redpanda.com/redpanda-connect we gained a much larger (and more competent) docs team, but as a consequence had to trim a lot of the old personality. I don't regret the transition but I do take a bit of comfort seeing the WarpStream peeps continue to have fun with it.
It's like shedding an almost intact full body skin and finding comfort in watching colleagues take it bowling, having a great time.
Was hoping you'd appreciate our efforts to retain your original quirky vision. We named the rice ball Geoff as a homage to you (intentionally spelled the silly way). (https://warpstreamlabs.github.io/bento/docs/about)/
FWIW in the UK, Geoff is the usual spelling (from Geoffrey) of the name. Jeff (from Jeffrey) also exists in the UK, but is much rarer, even if it's the most common form in the US.
Congrats on the launch! I can definitely understand the pain point; frequent plan/pricing iteration in the early days always leaves a pile of "grandfathered entitlements" that get carried around.
Two questions.
1) Entitlements seem to permeate systems in few ways (pricing pages & billing systems as you've called out) but also into feature flags systems like LaunchDarkly (my plan offers access to beta feature channel) and authorization systems (RBAC, FGA, etc). Do you see replacing those systems with your SDK or Planship is more of an integrator that helps keep them synchronized?
2) I couldn't tell from glancing through your SDK docs (might have missed it) but do you provide any audit/temporal history? If I store user events in a data warehouse (timestamp, customer_id, action_performed), can I determine a customers plan from that historical timestamp or only their current plan?
1. Today, we coexist with entitlement solutions (feature flags, auth systems, etc) by either working alongside them or feeding into entitlement aggregators. Basically, we handle the pricing-related data, logic, and aggregation that eventually reduces down to flags (Or numeric values, lists of items, or other value types). We offer SDKs to make these pricing entitlements easily accessible within a product marketing page, app, etc., but our API can just as easily be used to integrate with other systems.
2. We store complete subscription renewal history but the API for accessing it isn’t public yet. Audit trails, both for customer behavior like you mentioned, as well as admin tasks (E.g. Entitlement value X was added to plan Y) will be available via our API and in our console.
I started looking into this but DeleteObject doesn't support these conditional headers on general purpose buckets; only directory buckets (Express Zone One).
> Also note that we have never mentioned anything about cardinality. Because it doesn’t matter - any field can be of any cardinality. Scuba works with raw events and doesn’t pre-aggregate anything, and so cardinality is not an issue.
This is how we end up with very large, very expensive data swamps.
that depends on the sampling rate no? I would much rather have a rich log record sampled at 1% than more records that dont contain enough info to debug..
It is a tragedy of the current generation of observability systems that they have inculcated the notion that telemetry data should be sampled. Absolute nonsense.
The people feeling the pain of (and paying for) the expensive data swamp are often not the same people who are yolo'ing the sample rate to 100% in their apps, because why wouldn't you want to store every event?
Put another way, you're in charge of a large telemetry event sink. How do you incentivise the correct sampling behaviour by your users?
I have used that approach before with sentry. It was a non-issue. It depends on nature of the project of course, we had a system that was running every second so if it failed it generated a lot of data..
I agree. Sampling logs.. sounds dangerous. Obviously every system is different.
At least in GCP you can apply a filter to prevent ingestion and set different expiries on log budgets. This can help control costs without missing important entries.
Curious if cuelang just ended up being too much of a hurdle for onboarding. I like it and have used it quite a bit but there's something about the syntax that makes it impenetrable for many.
There's some of that. CUE is incredibly powerful, but it can be polarizing. But the fundamental problem is that developers don't want to learn a new language to write CI/CD pipelines: they want to use the language they already know and love.
So, no matter what language we had chosen for our first SDK, we would have eventually hit the same problem. The only way to truly solve the "CI/CD as code" problem for everyone, is to have a common engine and API that can be programmed with (almost) any language.
