The model only sees a stream of tokens, right? So how do you signal a change in authority (i.e. mark the transition between system and user prompt)? Because a stream of tokens inherently has no out-of-band signaling mechanism, you have to encode changes of authority in-band. And since the user can enter whatever they like in that band...
But maybe someone with a deeper understanding can describe how I'm wrong.
When LLMs process tokens, each token is first converted to an embedding vector. (This token to vectors mapping is learned during training.)
Since a token itself carries no information about whether it has "authority" or not, I'm proposing to inject this information in a reserved number in that embedding vector. This needs to be done both during post-training and inference. Think of it as adding color or flavor to a token, so that it is always very clear to the LLM what comes from the system prompt, what comes from the user, and what is random data.
This is really insightful, thanks. I hadn't understood that there was room in the vector space that you could reserve for such purposes.
The response from tempaccsoz5 seems apt then, since this injection is performed/learned during post-training; in order to be watertight, it needs to overfit.
You'd need to run one model per authority ring with some kind of harness. That rapidly becomes incredibly expensive from a hardware standpoint (particularly since realistically these guys would make the harness itself an agent on a model).
I assume "harness" here just means the glue that feeds one model's output into that of another?
Definitely sounds expensive. Would it even be effective though? The more-privileged rings have to guard against [output from unprivileged rings] rather than [input to unprivileged rings]. Since the former is a function of the latter (in deeply unpredictable ways), it's hard for me to see how this fundamentally plugs the whole.
I'm very open to correction though, because this is not my area.
My instinct was that you would have an outer non-agentic ring that would simply identify passages in the token stream that would initiate tool use, and pass that back to the harness logic and/or user. Basically a dry run. But you might have to run it an arbitrary number of times as tools might be used to modify/append the context.
ROM bootloader loads a second stage bootloader (e.g. [1]). The ROM bootloader verifies that the second stage loader is signed with keys fused into the MCU. The second stage bootloader in turn verifies application images shipped by the vendor, using a different set of keys.
When the vendor discontinues support for the device, they make available to their customers an optional update to the second stage bootloader that allows any application image to run, not just images signed by the vendor. This updated second stage loader is signed with the keypair fused into the MCU, and so it will run as per normal. They ideally make it so this update can only be installed with some sort of local interaction with the device, not automatically over the air.
Devices in the field running ordinary OEM firmware continue to be protected from malicious OTA updates. Customers who wish to unlock their devices also have the means to do so.
This is very technically straightforward to implement, but it needs to be considered when the device is designed. Regulations would be required to make sure that happens.
That does sound better, but haven't you just made the unlocked seconds stage bootloader functionally equivalent to secure boot keys?
Instead of [get released SB keys] -> [boot arbitrary payloads]
It becomes [get unlocked second stage bootloader] -> [boot arbitrary payloads]
Although, I guess that the details matter in terms of the process used to supply OTAs and second stage bootloaders. If changing to the unlocked bootloader requires physical access (or some such thing), then I could see that working.
Secure boot is desirable for a lot of reasons including design protection (stopping your product being cloned), supply chain security, preventing malicious updates etc.
The question is one of how you can hand control to the user without endangering your legitimate commercial interests as well as the security of the rest of the fleet, exactly how you tackle that will depend on the product.
There are two parts of "sending encrypted files": the encryption and the sending. An offline tool (e.g. PGP or age) seems only necessary when you want to decouple the two. After all, you can't do the sending with an offline tool (except insofar as you can queue up a message while offline, such as with traditional mail clients).
The question thereby becomes "Why decouple the sending from encryption?"
As far as I can see, the main (only?) reason is if the communication channel used for sending doesn't align with your threat model. For instance, maybe there are multiple parties at the other end of the channel, but you only trust one of them. Then you'd need to do something like encrypt the message with that person's key.
But in the use-case you mentioned (not wanting to publicly post a log file), I don't see why that reason would hold; surely the people who would send you logs can trust trust Signal every bit as easily as PGP. Share your Signal username over your existing channel (the mailing list), thereby allowing these people to effectively "upgrade" their channel with you.
Sticking to the use case of serving that 0.1% of users, why can’t a service or other encrypted transport be a solution? Why doesn’t Signal fit the bill for instance?
Why not Just(TM) enforce a reproducible build process? That brings some of its own challenges, but would represent a real upgrade over building out some Swiss cheese like this.
802.1x instead of switch ACLs
SSSD (Linux) or Active Directory (Windows) or other more custom solutions for dynamic DNS
Firewalls rules that use those dynamic DNS names
Jool’s site also has really great articles and diagrams on different translation techniques. Highly educational. I know it’s really helped me.
reply