I Just Wanted to Reuse Async Functions in Rust. I Ended Up Building a Tiny Task Manager
I am currently studying Rust, and one of the things that kept bothering me was surprisingly simple:
How do I reuse an async function more than once in a clean way?
At first, this did not look like a big problem. I already had async functions. I already had Tokio. I could already spawn tasks.
So what was the issue?
The issue was that I did not just want to run a task once.
I wanted to:
- register different async functions
- run them inside the same async system
- treat them as generic tasks
- restart some of them depending on their role
- keep the system logic separate from the task logic
That is the point where “I just need an async function” slowly becomes “I think I am building a task manager”.
This article is the story of that shift.
If you want to follow along with the code, the project is here:
https://github.com/icsboyx/async_task_manager
And this is the central idea that finally made the whole thing click for me:
A Future is not a task definition.
A Future is one execution.
If I want restartable async work, I should store the recipe: Fn() -> Future.
The Real Problem Was Not Async
The real problem was not writing async functions.
Rust makes it easy to write something like this:
pub async fn task1() -> anyhow::Result<()> {
loop {
println!("task 1 is running");
tokio::time::sleep(std::time::Duration::from_secs(1)).await;
}
}
This function intentionally never returns, so there is no trailing Ok(()).
If I later add shutdown logic, I can exit the loop cleanly with something like break Ok(()).
The problem starts when you want to say:
“I want to store multiple async functions, possibly with different behaviors, and start them when I want.”
That sounds natural, but in Rust it immediately forces you to think about:
- concrete types
- futures
- trait objects
- closures
- pinning
- ownership
- reusability
And that is where I hit my first wall.
My First Wrong Mental Model
My first instinct was something like:
let tasks = vec![task1, task2, task3];
Conceptually, that is exactly what I wanted:
- here are my tasks
- keep them in a collection
- run them when needed
- restart them if needed
But that nice mental model hides the real issue.
Each async fn returns a future, and each future has its own concrete type.
So the moment I want one task manager to store several different async starters behind one uniform interface, I need to be much more explicit about the abstraction I want.
That was the first real lesson:
If I want to restart async work, I should not store a future that is already created.
I should store something that can create a fresh future every time.
That single realization changed the whole design.
task1 vs task1(): The Difference That Actually Matters
This was the point where Rust forced me to think more clearly.
The first mistake would be trying to store this:
Why is that wrong for my use case?
Because task1() is not the task definition.
It is a future instance created right now.
That means:
- it represents one execution
- once that execution finishes, that value is done
- if I want to restart the task, I do not want the old future back
- I need a brand new future for the next run
So the real thing I needed was not:
but:
or, more generally:
That is the key mental shift.
Function Item, Function Pointer, Closure, dyn Fn
This also helped me understand something I had been hand-waving before.
task1 is not automatically “a dynamic function”.
At first, it is a function item: a concrete value tied to one exact function.
Rust knows exactly which function it is and gives it a concrete type.
From there, I can move through more general callable shapes:
-
task1: a concrete function item
-
fn() -> _: a function pointer
-
move || ...: a closure
-
dyn Fn() -> _: a dynamically dispatched callable trait object
That distinction mattered a lot for me, because the real transition was not just:
but:
- from one concrete callable
- toward one uniform callable abstraction I could store in the task manager
Once I saw it that way, the problem stopped being “why is Rust being difficult?” and became “what callable shape do I actually need?”
The Important Type Transition: From Concrete to Dynamic
This was probably the most important Rust lesson in the whole project.
When I write:
pub async fn task1() -> anyhow::Result<()>
Rust gives me a concrete future type behind the scenes.
The same is true for task2, task3, and every other async fn.
So even if all my functions look conceptually similar, Rust still sees different concrete future types.
That means this line is doing two jobs in the article:
let tasks = vec![task1, task2, task3];
Repetita iuvant.
Yes, I am showing the same line again on purpose.
The first time, it was my naive mental model.
This time, it marks the exact point where the type system pushes back.
If I want a HashMap, a queue, or a task manager to store all tasks uniformly, I need one shared stored type.
I cannot keep each future in its own concrete shape and still expect one collection to treat them all the same.
So this is where I had to make the move that, for me, changed the whole design:
- from one concrete future type
- to one dynamic future abstraction I could actually store
Because this is the point where Rust stops feeling like:
I wrote some async functions
and starts feeling like:
I need to model a reusable async execution boundary
In other words, I stopped thinking:
Store this exact future type.
and started thinking:
Store anything that behaves like a future returning Result<()>, even if the concrete future behind it changes from task to task.
In cleaned-up form, the idea that finally made sense to me was this:
use std::{
future::Future,
pin::Pin,
sync::Arc,
};
type TaskFuture =
Pin<Box<dyn Future<Output = Result<(), anyhow::Error>> + Send + 'static>>;
type TaskFn =
Arc<dyn Fn() -> TaskFuture + Send + Sync + 'static>;
And this was exactly the moment where my brain went:
wow wow wow wow wow… what the duck is this?
Because the first time you see something like this while learning Rust, it does not look like “a reusable task”.
It looks like somebody dropped an entire type system on your keyboard.
And yet, once I slowed down and read it piece by piece, it stopped looking absurd.
This pair of types says exactly what I need:
- a callable thing I can keep
- that I can invoke more than once
- and that gives me an owned future I can spawn each time
That is the real abstraction boundary in the design.
Why dyn Future Needs Box
Trait objects like dyn Future do not have a known size at compile time.
Rust therefore cannot store them directly by value.
I need indirection.
That is why the owned dynamic future shape is typically:
Box<dyn Future<Output = Result<(), anyhow::Error>> + Send>
The Box gives me:
- one stable heap allocation for the concrete future
- one uniform, sized handle I can store
- a way to erase the concrete future type behind a common interface
The same idea applies to the callable itself.
I do not want one Task<F, Fut> per task starter.
I want one storable task function type shared by the whole manager.
Why Pin<Box<dyn Future<...>>>
This was another thing that looked scary until it clicked.
Many futures produced by async fn are not Unpin.
After polling starts, Rust may require that they are not moved around in memory.
That is why the common owned dynamic future shape is not just:
Box<dyn Future<Output = Result<(), anyhow::Error>> + Send>
but:
Pin<Box<dyn Future<Output = Result<(), anyhow::Error>> + Send>>
So:
-
dyn Future gives me dynamic dispatch
-
Box gives me indirection and a sized owned container
-
Pin gives me the movement guarantees a future may require
Once that clicked, the ugly type stopped looking random.
It started looking like a precise description of the thing I actually needed to own.
The Closure That Makes the Whole Thing Reusable
Even after understanding the dynamic future type, I still needed a bridge between:
- the original concrete callable
- and the uniform task function type I wanted to store
That bridge is the closure.
In cleaned-up form, the constructor looks like this:
impl Task {
pub fn new<F, Fut>(
name: impl Into<String>,
function: F,
task_type: TaskType,
) -> Self
where
F: Fn() -> Fut + Send + Sync + 'static,
Fut: Future<Output = Result<(), anyhow::Error>> + Send + 'static,
{
let start_fn: TaskFn = Arc::new(move || {
Box::pin(function()) as TaskFuture
});
Self {
uuid: uuid::Uuid::new_v4().to_string(),
name: name.into(),
start_fn,
task_type,
}
}
}
At first glance, this looks like glue code.
It is not.
This closure is doing the real adaptation work.
It does three jobs at once:
- it owns the original concrete callable
- it calls it each time a fresh execution is needed
- it erases the concrete future type into
TaskFuture
That is what makes the design restartable.
Without this adapter, I would not really have a reusable task definition.
I would just have a bunch of specific functions with no common storage shape.
The move also matters here.
It makes the closure take ownership of function, so the task can keep that callable around and use it later, long after Task::new has returned.
If I had to summarize this entire section in one sentence, it would be:
The reusable thing is not the future.
The reusable thing is the factory that creates a fresh future on demand.
A Diagram of What Actually Happens
Here is the transformation that finally helped me think about it clearly:
flowchart LR
A["Concrete async function<br/>task1"] --> B["Concrete callable<br/>F: Fn() -> Fut"]
B --> C["Adapter closure<br/>move || Box::pin(function())"]
C --> D["Dynamic task factory<br/>TaskFn"]
D --> E["Stored inside Task<br/>start_fn"]
E --> F["Called by TaskManager"]
F --> G["Fresh owned future<br/>TaskFuture"]
G --> H["Spawned into Tokio JoinSet"]
This is also the point where the whole thing started to feel like function inception:
a function that stores a function that creates a future that runs a task
Ridiculous? A little.
Correct? Also yes.
Building a Task Type
Once I understood that I needed a restartable task factory, the next step was creating a task representation with both execution behavior and system meaning.
Conceptually, the task shape became:
pub struct Task {
uuid: String,
name: String,
start_fn: TaskFn,
task_type: TaskType,
}
I liked this design because each task now had:
- an identity
- a name for logs
- a reusable startup function
- a system-level policy
And that last point turned out to be very important.
Running a Task Is Not Enough. You Need to Know What It Means
While building the manager, I realized that “a task ended” is not enough information.
A task manager should not only know how to run a task.
It should know what that task means to the system.
Some tasks are optional.
Some tasks should stop the whole system when they end.
Some tasks are essential enough that if they disappear, the rest of the system no longer makes sense.
That is why I introduced this enum:
pub enum TaskType {
ShutDownOnExit,
Mandatory(i32),
Detachable(i32),
}
This was a big architectural improvement, because now the manager was not just executing tasks.
It was interpreting them.
ShutDownOnExit
If this task exits, the manager stops.
This is perfect for something like a ctrl_c_handler.
Mandatory(restarts)
This task is essential for the system.
If it exits, restart it while budget remains.
If the restart budget is exhausted, stop the whole manager.
For me, this means:
if this task is gone, keeping the rest of the system alive no longer makes sense
Detachable(restarts)
This task is useful, but not essential.
If it exits successfully, I can discard it.
If it fails, I can restart it while budget remains.
If the restart budget is exhausted, I discard it and let the rest of the system continue.
That made the whole project feel much more like a small supervised async system, and much less like a pile of random spawned tasks.
The Restart Counter Was More Subtle Than I Expected
Another thing that looked easy at first was restart logic.
I thought:
“Fine, I just decrement a counter and restart the task.”
But once I started implementing it, I had to answer a more precise question:
What does 3 actually mean?
Does it mean:
- the task can run 3 times total?
- the task can fail 3 times?
- the task can be restarted 3 times after the first start?
I chose the last meaning.
So in my design:
Mandatory(3) or Detachable(3) means:
- 1 initial start
- up to 3 restarts
That sounds simple, but it forced me to be careful with off-by-one logic.
The behavior I wanted was:
- if remaining restarts are greater than zero, restart and decrement
- if remaining restarts are zero, do not restart anymore
That was one of those small implementation details that taught me a lot about translating a verbal idea into precise behavior.
Why JoinSet Helped a Lot
Once I had reusable task factories, I still needed a way to:
- spawn multiple tasks
- wait for whichever one completes next
- know which task completed
- separate task completion from task policy decisions
tokio::task::JoinSet was a very good fit for this.
The manager can spawn tasks into the set and then wait for completions:
result = self.task_set.join_next_with_id() => {
// handle task completion here
}
That with_id part matters a lot.
If a task finishes, I do not just want to know that something finished.
I want to know which task finished, so I can:
- remove it from the active set
- look up its metadata
- apply the correct restart or shutdown policy
Another subtle but important point is that if the spawned task returns Result<(), anyhow::Error>, then the join result is nested.
In practice, that means these cases are different:
Some(Ok((id, Ok(())))) // Tokio join succeeded and the task returned Ok
Some(Ok((id, Err(e)))) // Tokio join succeeded but the task returned an application error
Some(Err(e)) // Tokio JoinError: panic, cancellation, abort, etc.
None // No more tasks in the JoinSet
That distinction matters because “the join worked” is not the same thing as “the task succeeded”.
JoinSet gave me a clean place to separate:
- execution mechanics
- task outcome
- system policy
And that made the task manager much easier to reason about.
The Core Idea of the Manager
At a high level, the flow became:
- Create tasks and push them into a queue.
- Spawn them into a
JoinSet.
- Wait for whichever task finishes next.
- Identify the task.
- Inspect how it ended.
- Apply the policy defined by
TaskType.
- Restart, discard, or stop the system.
That is the moment where the project stopped being “a few async functions” and became a tiny async supervisor.
The Part That Taught Me the Most
If I had to summarize what this project really taught me, it would be this:
1. Reusability in async Rust is about factories
If I want to execute the same async logic multiple times, I should usually store something that creates a future, not the future itself.
2. System behavior should not live inside each task
The task should focus on doing work.
The manager should decide what to do when that work ends.
That separation made the code much easier to reason about.
3. Semantics matter more than syntax
A lot of the difficulty was not “how do I write this in Rust?”
It was:
- what exactly does restart mean?
- when should the whole system stop?
- which tasks are optional and which are essential?
Those are architecture questions, not syntax questions.
Rust just forces you to answer them more explicitly.
Current Shape of the Project
Right now, the project is small on purpose.
It starts a few demo tasks, including a Ctrl+C handler, and routes their behavior through a small task manager with different policies.
It is not a production-grade supervisor, and I do not think it needs to be.
For me, it is a learning project with a very specific goal:
to understand how to reuse async functions cleanly, and how to build a small but structured execution model around them.
And honestly, I think that is why this project was useful.
It did not try to solve everything.
It solved one problem that kept bothering me, and in doing so it forced me to understand several Rust concepts more deeply.
Final Thought
I started with a very small question:
How can I reuse async functions in Rust without turning everything into a mess?
I ended up learning about:
- boxed futures
- trait objects
- closures as adapters
- pinning
- task supervision
- restart policy design
- async orchestration with Tokio
That is one of the things I like most about learning Rust:
you often start by trying to solve a local problem, and end up improving the way you think about the whole system.
If you are learning Rust too, and you have hit the same wall around reusable async functions, I hope this helps.
Sometimes the real progress is not just making the code compile.
Sometimes it is finally understanding what kind of abstraction you actually needed.
If you want to explore the code itself, you can find the project here:
https://github.com/icsboyx/async_task_manager
