making the roaming future

Came a bit late to the whole AI agents fun, so I decided to write my own harness thing in Golang. The idea of it running with scissors around my filesystem was pretty scary, so I limited its file write capabilities to the workspace directory.

Only a handful of shell commands are whitelisted. The harness attempts to limit command output to the workspace directory too, and, as a bonus, pipes commands to avoid cluttering up the context. Still need to encourage tail/head/grep use for text files to further optimize I/O and context use.

Additional shell commands can be enabled by creating a skill that has to be verified and approved by the user using slash commands. Still figuring out if and how those commands could be prompt-injected, so still lots to learn there as well.

But seeing the thing figure out commands by searching them through the Brave API, finding required parameters, and start performing background or scheduled tasks that consist of natural language commands is really cool. And all that using a self-hosted model.

Started getting the hype.

Also can see how agentic workflows like that quickly expose weaknesses of small, sub-10B models. Those get confused really fast, forget commands, tools, and skills, or ignore skill details, go for default parameters, and focus on the wrong stuff.

But on a sufficient model, it all just works, and the ability to tell my model-wrapper through WhatsApp to download a YT video for me, and check the news for a specific topic is pretty rewarding.

Need to make it figure out more about the world, tap into audio/video feeds, and can start preventing crimes too.