Cogs and Levers A blog full of technical stuff

Hooking open() with LD_PRELOAD

Introduction

Modern Linux systems provide a fascinating feature for overriding shared library behavior at runtime: LD_PRELOAD. This environment variable lets you inject a custom shared library before anything else is loaded — meaning you can intercept and modify calls to common functions like open, read, connect, and more.

In this post, we’ll walk through hooking the open() function using LD_PRELOAD and a simple shared object. No extra tooling required — just a few lines of C, and the ability to compile a .so file.

Intercepting open()

Let’s write a tiny library that intercepts calls to open() and prints the file path being accessed. We’ll also forward the call to the real open() so the program behaves normally.

Create a file named hook_open.c with the following:

#define _GNU_SOURCE
#include <stdio.h>
#include <stdarg.h>
#include <dlfcn.h>
#include <fcntl.h>

int open(const char *pathname, int flags, ...) {
    static int (*real_open)(const char *, int, ...) = NULL;
    if (!real_open)
        real_open = dlsym(RTLD_NEXT, "open");

    va_list args;
    va_start(args, flags);
    mode_t mode = va_arg(args, int);
    va_end(args);

    fprintf(stderr, "[HOOK] open() called with path: %s\n", pathname);
    return real_open(pathname, flags, mode);
}

This function matches the signature of open, grabs the “real” function using dlsym(RTLD_NEXT, ...), and then forwards the call after logging it.

Note We use va_list to handle the optional mode argument safely.

Compiling the Hook

Compile your code into a shared object:

gcc -fPIC -shared -o hook_open.so hook_open.c -ldl

Now you can use this library with any dynamically linked program that calls open.

Testing with a Simple Program

Try running a standard tool like cat to confirm that it’s using open():

LD_PRELOAD=./hook_open.so cat hook_open.c

You should see:

[HOOK] open() called with path: hook_open.c
#define _GNU_SOURCE
...

Each time the program calls open(), your hook intercepts it, logs the call, and passes control along.

Notes and Gotchas

  • This only works with dynamically linked binaries — statically linked programs don’t go through the dynamic linker.
  • Some programs (like ls) may use openat() instead of open(). You can hook that too, using the same method.
  • If your hook causes a crash or hangs, it’s often due to incorrect use of va_arg or missing dlsym resolution.

Where to Go From Here

You can expand this basic example to:

  • Block access to specific files
  • Redirect file paths
  • Inject fake contents
  • Hook other syscalls like connect(), write(), execve()

LD_PRELOAD is a powerful mechanism for debugging, sandboxing, and learning how programs interact with the system. Just don’t forget — you’re rewriting the behavior of fundamental APIs at runtime.

With great power comes great segfaults!

Hexagonal Architecture in Rust

Introduction

Hexagonal Architecture, also known as Ports and Adapters, is a compelling design pattern that encourages the decoupling of domain logic from infrastructure concerns.

In this post, I’ll walk through a Rust project called banker that adopts this architecture, showing how it helps keep domain logic clean, composable, and well-tested.

You can follow along with the full code up in my GitHub Repository to get this running locally.

Project Structure

The banker project is organized as a set of crates:

crates/
├── banker-core       # The domain and business logic
├── banker-adapters   # Infrastructure adapters (e.g. in-memory repo)
├── banker-fixtures   # Helpers and test data
└── banker-http       # Web interface via Axum

Each crate plays a role in isolating logic boundaries:

  • banker-core defines the domain entities, business rules, and traits (ports).
  • banker-adapters implements the ports with concrete infrastructure (like an in-memory repository).
  • banker-fixtures provides test helpers and mock repositories.
  • banker-http exposes an HTTP API with axum, calling into the domain via ports.

Structurally, the project flows as follows:

graph TD subgraph Core BankService AccountRepo[AccountRepo trait] end subgraph Adapters HTTP[HTTP Handler] InMemory[InMemoryAccountRepo] Fixtures[Fixture Test Repo] end HTTP -->|calls| BankService BankService -->|trait| AccountRepo InMemory -->|implements| AccountRepo Fixtures -->|implements| AccountRepo

Defining the Domain (banker-core)

In Hexagonal Architecture, the domain represents the core of your application—the rules, behaviors, and models that define what your system actually does. It’s intentionally isolated from infrastructure concerns like databases or HTTP. This separation ensures the business logic remains testable, reusable, and resilient to changes in external technology choices.

The banker-core crate contains the central business model:

pub struct AccountId(pub String);

pub struct Account {
    pub id: AccountId,
    pub balance_cents: i64,
}

pub trait AccountRepo {
    fn get(&self, id: &AccountId) -> Result<Option<Account>>;
    fn upsert(&self, account: &Account) -> Result<()>;
}

The Bank service orchestrates operations:

pub struct Bank<R: AccountRepo> {
    repo: R,
}

impl<R: AccountRepo> Bank<R> {
    pub fn deposit(&self, cmd: Deposit) -> Result<Account, BankError> {
        let mut acct = self.repo.get(&cmd.id)?.ok_or(BankError::NotFound)?;
        acct.balance_cents += cmd.amount_cents;
        self.repo.upsert(&acct)?;
        Ok(acct)
    }
    // ... open and withdraw omitted for brevity
}

The Bank struct acts as the use-case layer, coordinating logic between domain entities and ports.

Implementing Adapters

In Hexagonal Architecture, adapters are the glue between your domain and the outside world. They translate external inputs (like HTTP requests or database queries) into something your domain understands—and vice versa. Adapters implement the domain’s ports (traits), allowing your application core to remain oblivious to how and where the data comes from.

The in-memory repository implements the AccountRepo trait and lives in banker-adapters:

pub struct InMemoryAccountRepo {
    inner: Arc<Mutex<HashMap<AccountId, Account>>>,
}

impl AccountRepo for InMemoryAccountRepo {
    fn get(&self, id: &AccountId) -> Result<Option<Account>> {
        Ok(self.inner.lock().unwrap().get(id).cloned())
    }
    fn upsert(&self, account: &Account) -> Result<()> {
        self.inner.lock().unwrap().insert(account.id.clone(), account.clone());
        Ok(())
    }
}

This adapter is used both in the HTTP interface and in tests.

Testing via Fixtures

banker-fixtures provides helpers to test the domain independently of any infrastructure:

pub fn deposit(bank: &Bank<impl AccountRepo>, id: &AccountId, amt: i64) -> Account {
    bank.deposit(Deposit { id: id.clone(), amount_cents: amt }).unwrap()
}

#[test]
fn withdrawing_too_much_fails() {
    let bank = Bank::new(InMemRepo::new());
    let id = rand_id("acc");
    open(&bank, &id);
    deposit(&bank, &id, 100);

    let err = bank.withdraw(Withdraw { id, amount_cents: 200 }).unwrap_err();
    assert!(matches!(err, BankError::InsufficientFunds));
}

Connecting via Transport

The outermost layer of a hexagonal architecture typically handles transport—the mechanism through which external actors interact with the system. In our case, that’s HTTP, implemented using the axum framework. This layer invokes domain services via the ports defined in banker-core, ensuring the business logic remains insulated from the specifics of web handling.

In banker-http, we wire up the application for HTTP access using axum:

#[tokio::main]
async fn main() -> Result<()> {
    let state = AppState {
        bank: Arc::new(Bank::new(InMemoryAccountRepo::new())),
    };
    let app = Router::new()
        .route("/open", post(open))
        .route("/deposit", post(deposit))
        .route("/withdraw", post(withdraw))
        .with_state(state);
    axum::serve(tokio::net::TcpListener::bind("127.0.0.1:8080").await?, app).await?;
    Ok(())
}

Each handler invokes domain logic through the Bank service, returning simple JSON responses.

This is one example of a primary adapter—other adapters (e.g., CLI, gRPC) could be swapped in without changing the core.

Takeaways

  • Traits in Rust are a perfect match for defining ports.
  • Structs implementing those traits become adapters—testable and swappable.
  • The core domain crate (banker-core) has no dependencies on infrastructure or axum.
  • Tests can exercise the domain logic via fixtures and in-memory mocks.

Hexagonal Architecture in Rust isn’t just theoretical—it’s ergonomic. With traits, lifetimes, and ownership semantics, you can cleanly separate concerns while still writing expressive, high-performance code.

Backpropagation from Scratch

Introduction

One of the most powerful ideas behind deep learning is backpropagation—the algorithm that lets a neural network learn from its mistakes. But while modern tools like PyTorch and TensorFlow make it easy to use backprop, they also hide the magic.

In this post, we’ll strip things down to the fundamentals and implement a neural network from scratch in NumPy to solve the XOR problem.

Along the way, we’ll dig into what backprop really is, how it works, and why it matters.

What Is Backpropagation?

Backpropagation is a method for computing how to adjust the weights—the tunable parameters of a neural network—so that it improves its predictions. It does this by minimizing a loss function, which measures how far off the network’s outputs are from the correct answers. To do that, it calculates gradients, which tell us how much each weight contributes to the overall error and how to adjust it to reduce that error.

Think of it like this:

  • In calculus, we use derivatives to understand how one variable changes with respect to another.
  • In neural networks, we want to know: How much does this weight affect the final error?
  • Enter the chain rule—a calculus technique that lets us break down complex derivatives into manageable parts.

The Chain Rule

Mathematically, if

\[z = f(g(x))\]

then:

\[\frac{dz}{dx} = \frac{df}{dg} \cdot \frac{dg}{dx}\]

Backpropagation applies the chain rule across all the layers in a network, allowing us to efficiently compute the gradient of the loss function for every weight.

Neural Network Flow

graph TD A[Input Layer] --> B[Hidden Layer] B --> C[Output Layer] C --> D[Loss Function] D -->|Backpropagate| C C -->|Backpropagate| B B -->|Backpropagate| A

We push inputs forward through the network to get predictions (forward pass), then pull error gradients backward to adjust the weights (backward pass).

Solving XOR with a Neural Network

The XOR problem is a classic test for neural networks. It looks like this:

Input Output
[0, 0] 0
[0, 1] 1
[1, 0] 1
[1, 1] 0

A simple linear model can’t solve XOR because it’s not linearly separable. But with a small neural network—just one hidden layer—we can crack it.

We’ll walk through our implementation step by step.

Activation Functions

import numpy as np

def sigmoid(x):
    return 1 / (1 + np.exp(-x))

def sigmoid_derivative(x):
    return x * (1 - x)

def mse_loss(y_true, y_pred):
    return np.mean((y_true - y_pred) ** 2)

We’re using the sigmoid function for both hidden and output layers.

The sigmoid activation function is defined as:

\[\sigma(x) = \frac{1}{1 + e^{-x}}\]

Its smooth curve is perfect for computing gradients.

Its derivative, used during backpropagation, is:

\[\sigma'(x) = \sigma(x) \cdot (1 - \sigma(x))\]

The mse_loss function computes the mean squared error between the network’s predictions and the known correct values (y).

Mathematically, the mean squared error is given by:

\[\text{MSE}(y, \hat{y}) = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2\]

Where:

  • \(y_i\) is the actual target value (y_true),
  • \(\hat{y}_i\) is the network’s predicted output (y_pred),
  • \(n\) is the number of training samples.

Data and Network Setup

X = np.array([
    [0, 0],
    [0, 1],
    [1, 0],
    [1, 1]
])

y = np.array([
    [0],
    [1],
    [1],
    [0]
])

The x matrix defines all of our inputs. You can see these as the bit pairs that you’d normally pass through an xor operation. The y matrix then defines the “well known” outputs.

np.random.seed(42)
input_size = 2
hidden_size = 2
output_size = 1
learning_rate = 0.1

The input_size is the number of input features. We have two values going in as an input here.

The hidden_size is the number of “neurons” in the hidden layer. Hidden layers are where the network transforms input into internal features. XOR requires non-linear transformation, so at least one hidden layer is essential. Setting this to 2 keeps the network small, but expressive enough to learn XOR.

output_size is the number of output neurons. XOR is a binary classification problem so we only need a single output.

Finally, learning_rate controls how fast the network learns. This value scales the size of the weight updates during training. By increasing this value, we get the network to learn faster but we risk overshooting optimal values. Lower values are safer, but slower.

W1 = np.random.randn(input_size, hidden_size)
b1 = np.zeros((1, hidden_size))
W2 = np.random.randn(hidden_size, output_size)
b2 = np.zeros((1, output_size))

We initialize weights randomly and biases to zero. The small network has two hidden units.

Training Loop

We run a “forward pass” and a “backward pass” many times (we refer to these as epochs).

Forward pass

The forward pass takes the input X, feeds it through the network layer by layer, and computes the output a2. Then it calculates how far off the prediction is using a loss function.

# Forward pass
z1 = np.dot(X, W1) + b1
a1 = sigmoid(z1)

z2 = np.dot(a1, W2) + b2
a2 = sigmoid(z2)

loss = mse_loss(y, a2)

In this step, we are calculating the loss for the current set of weights.

This loss is a measure of how “wrong” the network is, and it’s what drives the learning process in the backward pass.

Backward pass

The backward pass is how the network learns—by adjusting the weights based on how much they contributed to the final error. This is done by applying the chain rule in reverse across the network.

# Step 1: Derivative of loss with respect to output (a2)
d_loss_a2 = 2 * (a2 - y) / y.size

This computes the gradient of the mean squared error loss with respect to the output. It answers: How much does a small change in the output affect the loss?

\[\frac{\partial \text{Loss}}{\partial \hat{y}} = \frac{2}{n} (\hat{y} - y)\]
# Step 2: Derivative of sigmoid at output layer
d_a2_z2 = sigmoid_derivative(a2)
d_z2 = d_loss_a2 * d_a2_z2

Now we apply the chain rule. Since the output passed through a sigmoid function, we compute the derivative of the sigmoid to see how a change in the pre-activation \(z_2\) affects the output.

# Step 3: Gradients for W2 and b2
d_W2 = np.dot(a1.T, d_z2)
d_b2 = np.sum(d_z2, axis=0, keepdims=True)
  • a1.T is the transposed output from the hidden layer.
  • d_z2 is the error signal coming back from the output.
  • The dot product calculates how much each weight in W2 contributed to the error.
  • The bias gradient is simply the sum across all samples.
# Step 4: Propagate error back to hidden layer
d_a1 = np.dot(d_z2, W2.T)
d_z1 = d_a1 * sigmoid_derivative(a1)

Now we move the error back to the hidden layer:

  • d_a1 is the effect of the output error on the hidden layer output.
  • We multiply by the derivative of the hidden layer activation to get the true gradient of the hidden pre-activations.
# Step 5: Gradients for W1 and b1
d_W1 = np.dot(X.T, d_z1)
d_b1 = np.sum(d_z1, axis=0, keepdims=True)
  • X.T is the input data, transposed.
  • We compute how each input feature contributed to the hidden layer error.

This entire sequence completes one application of backpropagation—moving from output to hidden to input layer, using the chain rule and computing gradients at each step.

The final gradients (d_W1, d_W2, d_b1, d_b2) are then used in the weight update step:

# Apply the gradients to update the weights
W2 -= learning_rate * d_W2
b2 -= learning_rate * d_b2
W1 -= learning_rate * d_W1
b1 -= learning_rate * d_b1

This updates the model just a little bit—nudging the weights toward values that reduce the overall loss.

Final Predictions

print("\nFinal predictions:")
print(a2)

When we ran this code, we saw:

Epoch 0, Loss: 0.2558
...
Epoch 9000, Loss: 0.1438

Final predictions:
[[0.1241]
[0.4808]
[0.8914]
[0.5080]]

Interpreting the Results

The network is getting better, but not perfect. Let’s look at what these predictions mean:

Input Expected Predicted Interpreted
[0, 0] 0 0.1241 0
[0, 1] 1 0.4808 ~0.5
[1, 0] 1 0.8914 1
[1, 1] 0 0.5080 ~0.5

It’s nailed [1, 0] and is close on [0, 0], but it’s uncertain about [0, 1] and [1, 1]. That’s okay—XOR is a tough problem when learning from scratch with minimal capacity.

This ambiguity is actually a great teaching point: neural networks don’t just “flip a switch” to get things right. They learn gradually, and sometimes unevenly, especially when training conditions (like architecture or learning rate) are modest.

You can tweak the hidden layer size, activation functions, or even the optimizer to get better results—but the core algorithm stays the same: forward pass, loss computation, backpropagation, weight update.

Conclusion

As it stands, this tiny XOR network is a full demonstration of what makes neural networks learn.

You’ve now seen backpropagation from the inside.

A full version of this program can be found as a gist.

Building Lazy Composition in Rust

Introduction

Rust programmers encounter combinators all the time: map(), and_then(), filter(). They’re everywhere in Option, Result, Iterator, and of course, Future. But if you’re coming from a functional programming background — or just curious how these things work — you might ask:

What actually is a combinator?

Let’s strip it down to the bare minimum: a value, a function, and some deferred execution.

A Lazy Computation

We’ll start with a structure called Thunk. It wraps a closure that does some work, and it defers that work until we explicitly ask for it via .run().

pub struct Thunk<F> {
    f: Option<F>,
}

impl<F, R> Thunk<F>
where
    F: FnOnce() -> R,
{
    pub fn new(f: F) -> Self {
        Self { f: Some(f) }
    }

    pub fn run(mut self) -> R {
        let f = self.f.take().expect("already run");
        f()
    }
}

It’s essentially a one-shot deferred computation. We stash a closure inside, and we invoke it only when we’re ready.

Here, F is the type of the closure (the function) we’re wrapping, and R is the result it will produce once called. This lets Thunk be generic over any one-shot computation.

The work here is really wrapped up by self.f.take() which will force the value.

Simple.

Example

Here’s what this looks like in practice:

fn main() {
    let add_one = Thunk::new(|| 3 + 1);
    let result = add_one.run();
    println!("Result: {}", result); // prints 4
}

No magic. No threading. No async. Just a delayed function call.

Composing Thunks

The real value in combinators is that they compose. We can make more complex computations out of simpler ones — without immediately evaluating them.

Here’s how we can build on top of multiple Thunks:

fn main() {
    let m = Thunk::new(|| 3 + 1); // 4
    let n = Thunk::new(|| 9 + 1); // 10

    let o = Thunk::new(|| m.run() + n.run()); // 14
    let result = o.run();

    println!("Result: {}", result);
}

We’ve built a new computation (o) that depends on two others (m and n). They won’t run until o.run() is called — and then they run in the correct order, and just once.

Look Familiar?

If you’ve spent time in Haskell, this structure might look suspiciously familiar:

fmap :: Functor f => (a -> b) -> f a -> f b

This is a form of fmap. We’re not building a full trait implementation here, but the shape is the same. We can even imagine extending Thunk with a map() method:

impl<F, R> Thunk<F>
where
    F: FnOnce() -> R,
{
    pub fn map<G, S>(self, g: G) -> Thunk<impl FnOnce() -> S>
    where
        G: FnOnce(R) -> S,
    {
        Thunk::new(|| g(self.run()))
    }
}

And now:

let t = Thunk::new(|| 42);
let u = t.map(|x| x * 2);
assert_eq!(u.run(), 84);

No typeclasses, no lifetimes — just combinator building blocks.

From Lazy to Async

Now here’s the twist. What if our .run() method couldn’t give us a value right away? What if it needed to register a waker, yield, and be polled later?

That’s exactly what happens in Rust’s async system. The structure is the same — a value and a function bundled together — but the execution context changes. Instead of calling .run(), we implement Future and respond to .poll().

Here’s what that looks like for a simple async Map combinator:

use std::future::Future;
use std::pin::Pin;
use std::task::{Context, Poll};
use pin_project::pin_project;

// Our Map combinator
#[pin_project]
pub struct Map<Fut, F> {
    #[pin]
    future: Fut,

    f: Option<F>, // Option to allow taking ownership in poll
}

impl<Fut, F> Map<Fut, F> {
    pub fn new(future: Fut, f: F) -> Self {
        Self { future, f: Some(f) }
    }
}

impl<Fut, F, T, U> Future for Map<Fut, F>
where
    Fut: Future<Output = T>,
    F: FnOnce(T) -> U,
{
    type Output = U;

    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
        let mut this = self.project();

        match this.future.poll(cx) {
            Poll::Pending => Poll::Pending,
            Poll::Ready(val) => {
                let f = this.f.take().expect("polled Map after completion");
                Poll::Ready(f(val))
            }
        }
    }
}

// Helper function to use it ergonomically
pub fn map<Fut, F, T, U>(future: Fut, f: F) -> Map<Fut, F>
where
    Fut: Future<Output = T>,
    F: FnOnce(T) -> U,
{
    Map::new(future, f)
}

Let’s take a step back and notice something: this structure is almost identical to Thunk. We’re still storing a value (future) and a function (f), and the combinator (Map) still controls when that function is applied. The difference is that we now interact with the asynchronous task system via poll(), instead of calling .run() ourselves.

This is how Future combinators in futures and tokio work under the hood — by carefully pinning, polling, and composing smaller futures into larger ones.

This is essentially a hand-rolled version of what futures::FutureExt::map() gives you for free.

As a simple example, we can use this as follows:

#[tokio::main]
async fn main() {
    let fut = async { 21 };
    let mapped = map(fut, |x| x * 2);
    let result = mapped.await;
    println!("Result: {}", result); // Should print 42
}

Conclusion

We often think of combinators as “just utility functions.” But they’re really more than that: they’re a way of thinking. Package a value and a transformation together. Delay the work. Compose more when you’re ready.

So the next time you write .map(), remember — it’s just a Thunk waiting to happen.

Create your own Filesystem with FUSE

Introduction

FUSE is a powerful Linux kernel module that lets you implement your own filesystems entirely in user space. No kernel hacking required. With it, building your own virtual filesystem becomes surprisingly achievable and even… fun.

In today’s article, we’ll build a filesystem that’s powered entirely by HTTP. Every file operation — reading a file, listing a directory, even getting file metadata — will be handled by a REST API. On the client side, we’ll use libcurl to perform HTTP calls from C, and on the server side, a simple Python Flask app will serve as our in-memory file store.

Along the way, you’ll learn how to:

  • Use FUSE to handle filesystem operations in user space
  • Make REST calls from C using libcurl
  • Create a minimal RESTful backend for serving file content
  • Mount and interact with your filesystem like any other directory

Up in my github repository I have added this project if you’d like to pull it down and try it. It’s called restfs.

Let’s get into it.

Defining a FUSE Filesystem

Every FUSE-based filesystem starts with a fuse_operations struct. This is essentially a table of function pointers — you provide implementations for the operations you want your filesystem to support.

Here’s the one used in restfs:

static struct fuse_operations restfs_ops = {
    .getattr = restfs_getattr,
    .readdir = restfs_readdir,
    .open    = restfs_open,
    .read    = restfs_read
};

This tells FUSE: “When someone calls stat() on a file, use restfs_getattr. When they list a directory, use restfs_readdir, and so on.”

Let’s break these down:

  • getattr: Fills in a struct stat with metadata about a file or directory — size, mode, timestamps, etc. It’s the equivalent of stat(2).
  • readdir: Lists the contents of a directory. It’s how ls knows what to show.
  • open: Verifies that a file can be opened. You don’t need to return a file descriptor — just confirm the file exists and is readable.
  • read: Reads data from a file into a buffer. This is where the real I/O happens.

Each function corresponds to a familiar POSIX operation. For this demo, we’re implementing just the basics — enough to mount the FS, ls it, and cat a file.

If you leave an operation out, FUSE assumes it’s unsupported — for example, we haven’t implemented write, mkdir, or unlink, so the filesystem will be effectively read-only.

Making REST Calls from C with libcurl

To interact with our HTTP-based server, we use libcurl, a powerful and flexible HTTP client library for C. In restfs, we wrap libcurl in a helper function called http_io() that performs an HTTP request and returns a parsed response object.

Here’s the core of the function:

struct _rest_response* http_io(const char *url, const char *body, const char *type) {
   CURL *curl = NULL;
   CURLcode res;
   long status = 0L;

   struct _http_write_buffer buf;
   buf.data = malloc(1);
   buf.size = 0;

   curl = curl_easy_init();

   if (curl) {
      curl_easy_setopt(curl, CURLOPT_URL, url);
      curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, type);

      if (body) {
         curl_easy_setopt(curl, CURLOPT_POSTFIELDS, body);
         curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, strlen(body));
      }

      curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, http_write_callback);
      curl_easy_setopt(curl, CURLOPT_WRITEDATA, (void *)&buf);

      curl_easy_setopt(curl, CURLOPT_USERAGENT, _http_user_agent);

      struct curl_slist *headers = NULL;
      headers = curl_slist_append(headers, "Content-Type: application/json");
      curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);

      res = curl_easy_perform(curl);
      curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &status);
      curl_easy_cleanup(curl);
      curl_slist_free_all(headers);

      if (res != CURLE_OK) {
         fprintf(stderr, "error: %s\n", curl_easy_strerror(res));
         if (buf.data) free(buf.data);
         return NULL;
      }
   }

   return rest_make_response(buf.data, buf.size, status);
}

Let’s break it down:

  • curl_easy_init() creates a new easy handle.
  • CURLOPT_URL sets the request URL.
  • CURLOPT_CUSTOMREQUEST lets us specify GET, POST, PUT, DELETE, etc.
  • If a body is provided (e.g. for POST/PUT), we pass it in using CURLOPT_POSTFIELDS.
  • CURLOPT_WRITEFUNCTION and CURLOPT_WRITEDATA capture the server’s response into a buffer.
  • Headers are added manually to indicate we’re sending/expecting JSON.
  • After the call, we extract the HTTP status code and clean up.

The result is returned as a _rest_response struct:

struct _rest_response {
   int status;
   json_object *json;
   char *data;     // raw response body
   size_t length;  // response size in bytes
};

This makes it easy to access either the full raw data or a parsed JSON object depending on the use case.

To parse the JSON responses from the server, we use the json-c library — a lightweight and widely used C library for working with JSON data. This allows us to easily extract fields like st_mode, st_size, or timestamps directly from the server’s responses.

To simplify calling common HTTP methods, we define a few handy macros:

#define rest_get(uri)         http_io(uri, NULL, "GET")
#define rest_delete(uri)      http_io(uri, NULL, "DELETE")
#define rest_post(uri, body)  http_io(uri, body, "POST")
#define rest_put(uri, body)   http_io(uri, body, "PUT")

With these in place, calling a REST endpoint is as simple as:

struct _rest_response *res = rest_get("/getattr?path=/hello.txt");

This layer abstracts away the curl boilerplate so each FUSE handler can focus on interpreting the result.

The Backend

So far we’ve focused on the FUSE client — how file operations are translated into HTTP requests. But for the system to work, we need something on the other side of the wire to respond.

Enter: a minimal Python server built with Flask.

This server acts as a fake in-memory filesystem. It knows nothing about actual disk files — it just stores a few predefined paths and returns metadata and file contents in response to requests.

Let’s look at the key parts:

  • A Python dictionary (fs) holds a small set of files and their byte contents.
  • The /getattr endpoint returns a JSON version of struct stat for a given file path.
  • The /readdir endpoint lists all available files (we only support the root directory).
  • The /read endpoint returns a slice of the file contents, based on offset and size.

Here’s a simplified version of the server:

from flask import Flask, request, jsonify
from urllib.parse import unquote
import os, stat, time

app = Flask(__name__)
fs = { '/hello.txt': b"Hello, RESTFS!\\n" }

def now(): return { "tv_sec": int(time.time()), "tv_nsec": 0 }

@app.route('/getattr')
def getattr():
    path = unquote(request.args.get('path', ''))
    if path == "/":
        return jsonify({ "st_mode": stat.S_IFDIR | 0o755, ... })
    if path in fs:
        return jsonify({ "st_mode": stat.S_IFREG | 0o644, "st_size": len(fs[path]), ... })
    return ('Not Found', 404)

@app.route('/readdir')
def readdir():
    return jsonify([name[1:] for name in fs.keys()])  # ['hello.txt']

@app.route('/read')
def read():
    path = request.args.get('path')
    offset = int(request.args.get('offset', 0))
    size = int(request.args.get('size', 4096))
    return fs[path][offset:offset+size]

This is enough to make ls and cat work on the mounted filesystem. The client calls getattr and readdir to explore the directory, and uses read to pull down bytes from the file.

End to End

With the server running and the client compiled, we can now bring it all together.

Start the Flask server in one terminal:

python server.py

Then, in another terminal, create a mountpoint and run the restfs client:

mkdir /tmp/restmnt
./restfs --base http://localhost:5000/ /tmp/restmnt -f

Now try interacting with your mounted filesystem just like any other directory:

➜  restmnt ls -l
total 1
-rw-r--r-- 1 michael michael  6 Jan  1  1970 data.bin
-rw-r--r-- 1 michael michael 15 Jan  1  1970 hello.txt

➜  restmnt cat hello.txt
Hello, RESTFS!

You should see logs from the server indicating incoming requests:

[GETATTR] path=/
127.0.0.1 - - [18/Aug/2025 21:29:46] "GET /getattr?path=/ HTTP/1.1" 200 -
[READDIR] path=/
127.0.0.1 - - [18/Aug/2025 21:29:46] "GET /readdir?path=/ HTTP/1.1" 200 -
[GETATTR] path=/hello.txt
127.0.0.1 - - [18/Aug/2025 21:29:46] "GET /getattr?path=/hello.txt HTTP/1.1" 200 -
127.0.0.1 - - [18/Aug/2025 21:29:47] "GET /open?path=/hello.txt HTTP/1.1" 200 -
127.0.0.1 - - [18/Aug/2025 21:29:47] "GET /read?path=/hello.txt&offset=0&size=4096 HTTP/1.1" 200 -
[GETATTR] path=/
127.0.0.1 - - [18/Aug/2025 21:29:47] "GET /getattr?path=/ HTTP/1.1" 200 -

Under the hood, every file operation is being translated into a REST call, logged by the Flask server, and fulfilled by your in-memory dictionary.

This is where the whole thing becomes delightfully real — you’ve mounted an HTTP API as if it were a native part of your filesystem.

Conclusion

restfs is a fun and minimal example of what FUSE can unlock — filesystems that aren’t really filesystems at all. Instead of reading from disk, we’re routing every file operation over HTTP, backed by a tiny REST server.

While this project is intentionally lightweight and a bit absurd, the underlying ideas are surprisingly practical. FUSE is widely used for things like encrypted filesystems, network mounts, and user-space views over application state. And libcurl remains a workhorse for robust HTTP communication in C programs.

What you’ve seen here is just the start. You could extend restfs to support writing files, persisting data to disk, mounting a remote object store, or even representing entirely virtual data (like logs, metrics, or debug views).

Sometimes the best way to understand a system is to reinvent it — badly, on purpose.