For non technical users they would have to find an existing plugin or maybe it doesn't exist so they are stuck.
But I can't help but think why wouldn't for example an application just have a prompt box that builds an extension dynamically on the fly (e.g. "make me a UI panel with some buttons that does X")
Obviously it would be "use at your own risk" and YMMV
Have any companies or applications actually done something like this?
I think modifying the real source code would get out of hand but perhaps just generating extensions that have access to a sand boxed set of safer APIs.
I do not literally use a prompt box inside that lab app, but it can sit there too, and then it looks like your case: the agent lives inside the app and builds on request. My agent works from "outside" from economic considerations: I want the coding agent on subscription instead of paying per api call for a chat-"resident" inside the box. So the agent-devops operates from my CLI and uses the lab app to prototype apps inside of it. And the lab hosts and runs them.
An app is a python backend with one declared entrypoint (python is the native backend but recently I added support for node sidecars), plus UI components in react/redux/whatever (each UI piece brings its own npm build, the host builds it into static assets and serves them, so for example angular fits similarly). The app exposes "surfaces" - it can declare a handler for data the box routes to it over channeled websocket and SSE and it can declare its own REST and MCP. That is how the clients reach the app, and its own UI components are such clients too. Auth is configured once at the host level.
App authoring/changing process looks like: I issue a token for my agent in the host app, and then the agent can add a whole new app to the host or remove one, or add/remove/change any part of an app, python or react, or its config, and run one reload command against the host. The host reloads the backend, rebuilds the widgets and serves them, the other apps at that time stay unaffected and are up. So if the ask was "make me a UI panel with some buttons that does X", after the reload that panel is simply there, in the running app's ui. Then the agent tests what it built through the browser or the API. I just look at the running result in the lab's frontend app where I can work with ui parts of my apps directly.
The "sandboxed APIs" concept is here too, and the sandbox is the app itself: generated code belongs to its own app and is available only via the APIs of this app through the host. Since the host scales multi-worker and multi-machine, the generated app has to be written in this understanding (like if it was a fast api app). For the agent the host is a simulator box, but everything in it is real.
You can connect a model via an open router or Anthropic app key, and file tickets for an llm to “go to town” on editing the IDE itself
The entire IDE code is editable (everything) in the IDE itself
[0] - https://con-dog.github.io/slices-demo/
But basically it’s an attempt at an ide for someone with ADHD, instead of holding a whole feature in your brain you can look at 1 “slice” at a time (fully self contained feature)
It has the ability to connect an openrouter or Anthropic api-key to get an agent to modify the editor itself by using the editor itself. A ticketing system like a kanban board, instead of a chat log
[0] - https://github.com/con-dog/slices-demo/blob/main/README.md
I asked chatgpt and it said hot reloading in a browser, but thats not what i'm talking about. I'm meaning realtime high performance desktop applications that the user can prompt to change the app in realtime while it is running.
The closest thing i saw to this was something called VibeOS, but thats a whole operating system.