Security people will argue forever “defense in depth” this, “real world doesn’t match the study” that. Yawn.

Here’s the hard truth: cybersecurity today is basically fashion. It’s not science, it’s herd behavior. The industry is still running on the “nobody gets fired for buying IBM” mentality. Careers aren’t built on being right, they’re built on chasing whatever tool is trending on LinkedIn this quarter.

If studies like this mattered, then the security community would have paid attention to this study long ago:

https://er.educause.edu/articles/2005/1/fostering-email-secu...

> Could a rogue agent theoretically run a destructive command? Sure. Have I seen it happen in weeks of usage? Never.

I've been in cybersecurity over a decade, and this still blows my mind. It’s classic cognitive dissonance or just normalized deviance. People get used to doing unsafe things until the system breaks.

Best analogy I use: seatbelts. In the U.S., wearing a seatbelt is automatic. Zero thought. In other parts of the world, not so much. Ask why, and you’ll hear: “I’m a good driver. Never had an accident. Don’t need it.” That’s not logic. That’s luck confused with safety.

The difference? Conditioning. Risk comprehension. Cultural defaults.

Same thing happens in software. No amount of UI warnings will stop people from doing dumb things. Running as root, disabling SELinux, exposing prod databases to the open internet. Happens constantly.

Anthropic gave a user the ability to do something they know is risky. Anthropic understands "LLM Trifecta" vulns. This person has no idea.

Strap in, we're in for a wild ride.

At the last job we deployed to thousands of nodes across AWS, Azure, and Aliyun. There was no unique needs across those environments, from a deploy perspective. There where some minor pain points from a config and monitor perspective. There where massive pain points from a infra management perspective. Our SREs knew AWS inside and out but struggled with Azure and Aliyun documentation and support. We had to build out API middleware, so existing could could seamlessly operate in all 3 environments. That's not set it and forget it. Keeping everything consistent across 3 cloud APIs requires constant care and feeding.

Most people and by extension, most businesses don’t think from first principles. They copy what others do because it’s easier. It reduces cognitive load. But that kind of thinking leads to cargo cults. People doing things that look right but make no sense when you break them down. Business schools teach unit economics and first-principles logic (like in The Goal), but most companies still optimize for quarterly performance. It’s backwards. You end up with systems that reward short-term hacks instead of long-term efficiency.

In the real world, business moves fast. Too fast for most people to stop and think. If you don’t build in ways to slow down and reason from fundamentals, you’ll just react your way into mediocrity. The companies that win long term (Amazon, Toyota, SpaceX) they go back to physics-level thinking. They understand the real constraints and design around them. That’s the cheat code. First-principles thinking isn’t optional—it’s the only way to build something that actually works and lasts.

how do you tell the world what's important in a case like this?

If you want alignment, set clear rules that everyone understands. At SpaceX, their spending policy was simple: If it helps us get to Mars faster, spend it. If not, don’t. That simple policy helped keep the whole company focused.

Simpler rules, better decisions, faster progress.

I haven't seen anything useful in the agent space. I definitely haven't seen anything I would trust in a business process.

At the same time, I'm constantly amazed at how accessible LLMs have made automating things with Python. I'm seeing more non-SDEs describe what they want to do and then iterate on a solution with the LLM.

So I see more happening in this space, but it's a little more deterministic and a little less abstracted than the current products are headed.

I also see Gitlab or GitHub dabbling in this area. At some point you need to deploy the code. GitHub actions, workspaces, and pages are not that far off from a product that could cater to this need.

Security is about real risk reduction, not chasing whatever’s trendy - but that's what most security teams do and then complain about the results.

Most business functions are metric-driven. Security should be no different. The right approach: convert qualitative insights into hard data, then systematically drive that metric down.

It's not easy. It's hard work, but I've done it at 3 companies. It's doable.

I look at this through the following perspectives:

As a security engineer, this is pretty obvious - blindly handing over login credentials to AI agents? What could possibly go wrong? Feels like a ticking time bomb until courts start hashing out liability.

As a technologist, it really makes you rethink UI design. Do we even need traditional interfaces anymore? If LLMs handle backward compatibility so seamlessly, we might just skip straight to conversational AI as the default interface.

As an investor, there’s going to be a gold rush. Early movers won’t need perfect accuracy—just "good enough" to capture market share. Classic first-mover advantage.

And as a manager, this screams inefficiency. Expect something like Six Sigma 2.0 to emerge. AI driven quality control to fix AI driven errors. Irony is strong with this one.

I see two possibilities here. Either Musk is a foreign agent attempting to cripple the US government or he's a student of Blitzscaling and Jim Collins' The Map.

Incentives can help determine his motives. What could a foreign government offer Musk that his billions can't already purchase?

The spending policy at SpaceX could offer insight into motive. As I understand it, it's simply: If spending the money gets us to Mars sooner, then it's authorized.

So is this all about getting to Mars sooner? If so, then if your bottleck is government bureaucracy, then wouldn't it make sense to spend hundreds of millions on a campaign so you can seize control of the government and overhaul it in the service of getting to Mars sooner?

I doubt they're no performers, they're likely selective performers. Issues like this can often be explained by Public Goods Game theory. If there's no economic incentive, then some participants will choose to not contribute or under-contribute.

https://en.m.wikipedia.org/wiki/Public_goods_game

Well, there is an economic incentive - "salary". If you're not interested in it, may be it's time to move on as time is your own most valuable asset and choosing to waste it on "free ride" - it's something beyond my comprehension: you're literally waiting to be exposed and fired.

note: the org is not large, just 250 ppl or so.

The key is modularity with good interface design, then you have AI generate each component and play more of a QA role to validate each component is functional.

That makes sense