Is there a market for a distributed audit infra with attestations? If I can have ChatGPT audit a file (content hash) with a known-good prompt, and then share the link as proof of the full conversation, would this be useful evidence to de-risk?
If each developer can audit some portion of their dep tree and reuse prior cached audits, maybe it’s tractable to actually get “eyeballs” on every bit of code?
Not as good as human audit of course, but could improve the Pareto-frontier for cost/effectiveness (ie make the average web dev no-friction usecase safer).
I think there is, definitely, and that will be a solid route out of this supply chain debacle we find ourselves in.
It will have to involve identity (public key), reputation (white list?), and signing their commits and releases (private key). All the various package managers will need to be validating this stuff before installing anything.
Then your attestation can be a manifest "here is everything that went into my product, and all of those components are also okay.
> Secondly, if you re-analyzed the same malicious script over and over again it would eventually pass inspection, and it only needs to pass once
You’d need some probabilistic signal rather than a binary one. Eg if some user with zero reputation submits a single session saying “all good”, this would be a very weak signal.
If one of the Python contributors submits a batch of 100 reasoning traces all showing green, you’d be more inclined to trust that. And of course you would prefer to see multiple scans from different package managers, infra providers, and OS distributions.
If each developer can audit some portion of their dep tree and reuse prior cached audits, maybe it’s tractable to actually get “eyeballs” on every bit of code?
Not as good as human audit of course, but could improve the Pareto-frontier for cost/effectiveness (ie make the average web dev no-friction usecase safer).