What 180 Generations of Genetic Algorithm Trading Taught Me About Overfitting

At first glance, Unix’s fork() + exec() model feels… wrong.

Why would an operating system copy an entire process, only to immediately replace it with something else?

It seems wasteful, indirect, and unnecessarily complicated.

And yet, this design has survived for over 50 years, and still powers modern systems today.

In this post, we’ll explore what fork() and exec() actually do, why this design exists, and why it remains so hard to replace.

fork()

fork() is a system call in POSIX operating systems. It creates an almost exact replica of the process that calls it. The calling process is known as the parent, and the new one is the child.

fork() takes no arguments and returns the PID of the child to the parent and 0 to the child. This return value is used to determine whether a process is the parent or the child.

When fork() was first introduced, the parent and child ran in separate memory spaces with identical contents. This was expensive, especially for large processes with large memory footprints.

To optimize this, modern implementations of fork() use copy-on-write (COW). The parent and child have separate virtual address spaces, but initially they point to the same physical memory. Instead of copying all memory eagerly, the kernel duplicates pages only when one of the processes modifies them. This significantly reduces memory overhead.

exec()

The exec() family of functions replaces the current process image with a new one. It does not create a new process; it transforms the existing one.

In essence, it wipes the current address space and loads a new program into it. On success, it never returns.

fork() + exec() pattern

So what is the fork() + exec() pattern?

1. The parent calls fork() to create a child process
2. The child calls exec() to replace itself with a new program

This is where the design initially feels counterintuitive. We just duplicated the address space (or at least created new page tables in modern implementations) only to discard it immediately. Why not just create a new process directly?

This is where the genius of the design becomes clear.

The key idea behind the fork() + exec() pattern is the separation of process creation from program execution. By separating these two concerns, we gain a powerful capability: a configurable “gap” between creation and execution.

Why this matters

This “gap” allows you to modify the execution environment before calling exec(). For example:

• file descriptors (via dup2)
• working directory
• user/group IDs
• signal handlers

Here is what that looks like in C:

pid_t pid = fork();

if (pid == 0) {
    // We are inside the child process.
    // This is the "gap" where we can redirect I/O, change directories, etc.
    char *args[] = {"ls", NULL};
    execvp("ls", args); 

    // If exec succeeds, the code below here NEVER runs!
} else if (pid > 0) {
    // We are the parent. We wait for the child to finish executing.
    wait(NULL); 
}

If you’ve ever typed ls > output.txt in your terminal, you’ve used this gap. The shell calls fork(), the child process uses dup2() to overwrite its standard output with output.txt, and then it calls exec('ls'). The ls program has no idea its output is being redirected to a file, it just writes to standard output as usual. The shell sets the stage before the actor even arrives.

Another important benefit is inheritance as a composition mechanism. Instead of passing large configuration objects (looking at you, Windows 👀), processes inherit resources, making composition natural.

A key example is pipes. A pipe is a unidirectional data channel used for inter-process communication (IPC). It is extensively used in shells (via the | operator) to pass the output of one process as input to another.

Calling pipe() returns two file descriptors representing the ends of the pipe. Because file descriptors are inherited, each process simply closes the ends it doesn’t use and voilà IPC just works.

This design reflects the Unix philosophy:

small, composable primitives instead of one large, complex API

Instead of one complex interface, Unix provides two simple APIs:

• one to copy a process
• one to replace a process

This simplicity creates a flexible and easy-to-reason-about system.

Trade-offs

While the fork() + exec() pattern is an inspired design that still forms the backbone of modern systems, it does have trade-offs.

The most obvious one is the overhead of fork(). Even with COW, the kernel must create new page tables for the child process. This can be expensive, especially for large processes like servers or language runtimes. It can also introduce page faults when memory is modified.

A more subtle but significant drawback is its interaction with multi-threaded programs. When fork() is called, only the calling thread survives in the child process, while the state of other threads (such as locks) may remain. This can easily lead to deadlocks.

For this reason, in multi-threaded programs, fork() is almost always followed immediately by exec(). Anything more complex can lead to inconsistent state.

Additionally, this model can be inefficient for high-frequency process creation workloads. Modern systems often prefer threads, async/event loops, or worker pools for such cases. Because of this, alternatives like posix_spawn() exist.

Final thought

Even after 50+ years, the fork + exec model remains dominant. Not because it’s perfect, but because it offers a level of flexibility that is still hard to match.

If you enjoyed this, I’m currently diving deep into systems programming—building a Unix shell from scratch, implementing a custom malloc allocator, and more.

I’ll be sharing what I learn along the way, so feel free to follow along.

You can also check out the code for the shell project is here:

👉 https://github.com/IsbatBInHossain/ishell

Most game NPCs aren’t lifeless because the animations are bad.
They feel lifeless because nothing you do ever sticks.

You help them, threaten them, embarrass them, save them, steal from them, and five seconds later they’re back to factory settings. No tension. No grudges. No slow trust. No one tells anyone else what happened.

They just flip to the next state and keep going.
Players notice this instantly, even if they can’t explain why.

While building my own system, I kept running into the same realization:

NPCs don’t need a bigger brain.
They need continuity.

The problem with scripted behavior

A lot of NPC logic still feels like stage acting.

A behavior tree says:

• patrol here
• chase there
• say this line
• go idle

That works fine for guards or enemies.

It breaks the moment you want social behavior.

Because social behavior is messy.

It depends on mood, memory, reputation, hierarchy, recent events, and sometimes just bad timing.

Two NPCs can say the exact same line, but if one trusts you and the other fears you, the moment feels completely different.

That’s where most systems fall apart.

So instead of treating behavior as “pick an action,” I started treating it as a loop between emotion, memory, needs, and learning.

The behavior tree didn’t go away.
It just stopped being the brain.

What I actually built

Each NPC in my system carries state, not just a task.

They have:

• A live emotional state PAD (Pleasure, Arousal, Dominance)
• Five needs constantly drifting:
• hunger
• safety
• belonging
• status
• curiosity
• A Social Ledger tracking relationships with other agents
• A second layer tracking trust, fear, respect, and hostility

They also remember things.

Short-term memory stores recent interactions.
Long-term memory keeps only high-impact events so it doesn’t turn into noise.

On top of that, there’s a reinforcement learning loop.

Instead of hardcoding behavior like “friendly NPCs socialize,” the system evaluates what actually happened:

• Did this increase pleasure?
• Did it improve status?
• Did it reduce stress?

That feedback shapes future decisions.

And then there’s one small piece that ended up being critical.

A simple behavior tree interrupts everything when survival needs spike.

That one decision fixed a lot of “smart but stupid” behavior.

How it works (without the fluff)

Each update cycle looks roughly like this:

• emotions shift
• needs increase
• relationships decay slightly
• last action gets evaluated
• next action is chosen

Emotion is not just a label. It drives everything.

Each NPC has a PAD state that:

• starts from a personality baseline
• drifts over time
• reacts to interactions
• spreads between nearby agents

At one point my NPCs basically turned into an emotional hivemind. Everyone felt the same thing all the time.

Interesting, but useless.

So I had to shrink the influence radius just to keep individuality intact.

Memory is where things get interesting

The Social Ledger is doing most of the real work.

Every interaction updates trust and familiarity.

Subscribe to the Medium newsletter
Over time:

• friends form
• rivals emerge
• some NPCs become isolated

And then there’s gossip.

NPCs don’t just react to you. They talk about you.

When one NPC shares information:

• it is filtered by trust
• influenced by familiarity
• slightly distorted by noise

So reputation spreads, but not perfectly.

That one system changed everything.

Now the world feels less mechanical and more social.

Learning with survival constraints

Decision-making runs on a short cycle.

Actions are scored based on:

• emotional impact
• dominance shifts
• need satisfaction
• environmental pressure

But learning does not have full control.

If survival thresholds are hit:

• hunger forces gathering
• danger forces withdrawal
• social deprivation forces interaction

Without that override, agents became classic reinforcement learning failures. They kept optimizing the wrong thing while clearly ignoring what mattered.

What actually happened

Memory mattered more than complexity.

Even before adding more actions, NPCs already felt more alive because interactions had consequences.

I once watched two NPCs start as neutral strangers. A few in-game days later, they ended up in a full dominance fight. Nothing was scripted. One gathered resources near the other’s territory, a bit of gossip spread, trust dropped, and things escalated.

The system just carried it forward.

Then everything started feeding into everything else.

A simple resource event could:

• increase dominance
• create territory
• trigger intrusion
• escalate into conflict later

A single fight could:

• shift hierarchy
• change fear and respect
• spread tension through the group

And gossip about the player turned out to be one of the most powerful parts.

You stop being new to every NPC.

Your reputation arrives before you do.

What did not work at first

A few things broke early.

Trust changed too slowly, so relationships were hard to notice.

Learning updates were too weak, so behavior barely shifted.

Starting everyone emotionally neutral caused the whole system to stall. No hierarchy formed.

Fixing these was not about adding more systems.

It was about tuning the ones that already existed so changes were visible.

Because if nothing noticeable happens, it does not matter how correct your system is. It still feels dead.

What this means for game AI

The solution is not making NPCs smarter in the usual sense.

It is making them carry state.

Let them:

• feel something
• remember something
• react differently next time

The biggest improvement in my system did not come from expanding the behavior tree.

It came from combining emotion, memory, hierarchy, gossip, and learning.

That mix did not give me perfectly scripted characters.

It gave me something better.

NPCs that drift.

And once they drift, they start to surprise you.

If you’re building something similar or experimenting with game AI, I’d genuinely like to hear what you’re working on.