Acting like an Agent, using Rust.
The Question
As an engineer and a manager of engineering teams, I asked myself the question. What would I do if I wanted to build a product engineering team from scratch? What would the roles be, which positions would be singular, and which parts would be played by teams of workers? This is how I thought to think about building out functions inside companies. And then, I would hire the best talent for all these roles.
I would create the following roles:
- A manager responsible for questions like "Can we take on more work," "Is the work well defined," or "Should we do some minor restructuring."
- An architect responsible for determining the overarching design
- Some engineers responsible for building and maintaining the individual modules
The exciting part of building both products and teams comes from constraints. What would you do if you need access to the best talent? What would you do if the people you could hire were eager but mediocre at their jobs? What mechanisms would you put in place then?
Why assume mediocrity? Because I am planning to do all of this using LLMs. And what is an LLM other than an incompetent human?
The Answer
Let's define an Actor in Rust:
#[async_trait]
pub trait Agent {
type Msg;
fn init() -> Self;
async fn handle(&mut self, msg: Self::Msg) -> Result<(), Box<dyn Error>>;
}
To be an actor, you need an input message type and handle function that accepts the message type.
We implement also need a way to run an agent:
#[derive(Debug)]
pub struct AgentRunnerHandle<A>
where
A: Agent,
{
tx: Arc<mpsc::Sender<ServiceMsg<A::Msg>>>,
}
#[derive(Debug)]
pub enum ServiceMsg<T> {
Msg(T),
Shutdown,
}
impl<A> AgentRunnerHandle<A>
where
A: Agent + Send,
A::Msg: Debug + Send + Clone + 'static + PartialEq<A::Msg>,
{
pub fn start() -> AgentRunnerHandle<A> {
let (tx, rx) = mpsc::channel(32);
let tx = Arc::new(tx);
{
tokio::spawn(async move {
let mut a = AgentRunnerInternal::<A> {
inbox: rx,
state: A::init(),
};
a.inner_loop().await;
});
}
AgentRunnerHandle { tx }
}
pub async fn send(&self, msg: A::Msg) -> Result<(), SendError<ServiceMsg<A::Msg>>> {
self.tx.send(ServiceMsg::Msg(msg)).await
}
pub async fn stop(&self) -> Result<(), SendError<ServiceMsg<A::Msg>>> {
self.tx.send(ServiceMsg::Shutdown).await?;
Ok(())
}
pub async fn closed(&self) {
self.tx.closed().await
}
}
struct AgentRunnerInternal<A>
where
A: Agent,
{
inbox: mpsc::Receiver<ServiceMsg<A::Msg>>,
state: A,
}
impl<A> AgentRunnerInternal<A>
where
A: Agent + Send,
A::Msg: Debug + Send + Clone + 'static + PartialEq<A::Msg>,
{
async fn inner_loop(&mut self) {
let mut running = true;
while running {
if let Some(msg) = self.inbox.recv().await {
match msg {
ServiceMsg::Msg(m) => {
if let Ok(()) = self.state.handle(m).await {
} else {
panic!("failed to run handle")
}
}
ServiceMsg::Shutdown => running = true,
}
}
}
}
}
Now we can implement our own agent.
#[derive(Debug, Clone, PartialEq)]
enum Task {
Set { value: i32 },
Print
}
#[derive(Debug)]
struct Service {
value: Option<i32>,
}
#[async_trait]
impl Agent for Service {
type Msg = Task;
fn init() -> Self {
Service {
task: None,
}
}
async fn handle(&mut self, msg: Self::Msg) -> Result<(), Box<dyn Error>> {
match msg{
Set {value} => self.value = Some(value),
Print => {println!("{:?}", self.value)},
};
Ok(())
}
}
This allows us to set up independent agents for our program's workers.
That is it for now. Next time we will connect the openAI's API and use this to update and run agents.