Understanding the Transformer Architecture

Introduction

Natural language processing (NLP) has gone through several paradigm shifts:

• Bag-of-Words — treated text as unordered word counts; no sequence information. We’ve spoken about this previously.
• Word Embeddings (word2vec, GloVe) — learned fixed-vector representations that captured meaning. We’ve looked at these previously.
• RNNs, LSTMs, GRUs — processed sequences token-by-token, retaining a hidden state; struggled with long-range dependencies due to vanishing gradients.
• Seq2Seq with Attention — attention helped the model “focus” on relevant input tokens; a leap in translation and summarization.
• Transformers (Vaswani et al., 2017 — “Attention Is All You Need”) — replaced recurrence entirely with self-attention, allowing parallelization and longer context handling.

Transformers didn’t just improve accuracy; they unlocked the ability to scale models massively.

In this post, we’ll walk though an understanding of the transformer architecture by implementing a GPT-style Transformer from scratch in PyTorch, from tokenization to text generation.

The goal: make the architecture concrete and understandable, not magical.

Overview

At a high level, our model will:

1. Tokenize text into integers.
2. Map tokens to dense embeddings + positional encodings.
3. Apply self-attention to mix contextual information.
4. Use feed-forward networks for per-token transformations.
5. Wrap attention + FFN in Transformer Blocks with residual connections and layer normalization.
6. Project back to vocabulary logits.
7. Generate text autoregressively.

Tokenization

Before our model can process text, we need to turn characters into numbers it can work with — a process called tokenization. In this example, we use a simple byte-level tokenizer, which treats every UTF-8 byte as its own token. This keeps the implementation minimal while still being able to represent any possible text without building a custom vocabulary.

class ByteTokenizer:
    """
    UTF-8 bytes <-> ints in [0..255].
    NOTE: For production models you'd use a subword tokenizer (BPE, SentencePiece).
    """
    def __init__(self) -> None:
        self.vocab_size = 256

    def encode(self, text: str) -> list[int]:
        return list(text.encode("utf-8"))

    def decode(self, ids: list[int]) -> str:
        return bytes(ids).decode("utf-8", errors="ignore")

Example:

tok = ByteTokenizer()
ids = tok.encode("Hello")
print(ids)        # [72, 101, 108, 108, 111]
print(tok.decode(ids))  # "Hello"

Embeddings & Positional Encoding

Once we have token IDs, we map them into embedding vectors — learned dense representations that capture meaning in a continuous space. Each token ID indexes a row in an embedding matrix, turning a discrete integer into a trainable vector of size \(d_{\text{model}}\). Because self-attention alone has no sense of order, we also add positional embeddings, giving the model information about each token’s position within the sequence.

self.tok_emb = nn.Embedding(vocab_size, d_model)   # token embeddings
self.pos_emb = nn.Embedding(block_size, d_model)   # positional embeddings

Self-Attention

Self-attention lets each token attend to all previous tokens (causally masked to prevent peeking ahead).

Mathematically:

That equation means each token computes a similarity score with all other tokens (via \(QK^\top\)), scales it by \(\sqrt{d_k}\) to stabilize gradients, turns the scores into probabilities with softmax, and then uses those probabilities to take a weighted sum of the value vectors \(V\) to produce its new representation.

Multi-head attention runs this in parallel on different projections.

class MultiHeadSelfAttention(nn.Module):
    def __init__(self, d_model, n_heads, block_size, dropout):
        super().__init__()
        assert d_model % n_heads == 0
        self.n_heads = n_heads
        self.head_dim = d_model // n_heads
        self.qkv = nn.Linear(d_model, 3 * d_model, bias=False)
        self.out_proj = nn.Linear(d_model, d_model, bias=False)
        self.attn_drop = nn.Dropout(dropout)
        self.resid_drop = nn.Dropout(dropout)
        mask = torch.tril(torch.ones(block_size, block_size, dtype=torch.bool))
        self.register_buffer("causal_mask", mask)

    def forward(self, x):
        B, T, C = x.shape
        qkv = self.qkv(x)
        q, k, v = qkv.chunk(3, dim=-1)
        def split_heads(t): return t.view(B, T, self.n_heads, self.head_dim).transpose(1, 2)
        q, k, v = split_heads(q), split_heads(k), split_heads(v)
        scores = (q @ k.transpose(-2, -1)) / math.sqrt(self.head_dim)
        scores = scores.masked_fill(~self.causal_mask[:T, :T], float("-inf"))
        att = F.softmax(scores, dim=-1)
        att = self.attn_drop(att)
        y = att @ v
        y = y.transpose(1, 2).contiguous().view(B, T, C)
        y = self.out_proj(y)
        y = self.resid_drop(y)
        return y

Feed-Forward Network

A per-token MLP, applied identically at each position.

class FeedForward(nn.Module):
    def __init__(self, d_model, mult=4, dropout=0.0):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(d_model, mult * d_model),
            nn.GELU(),
            nn.Linear(mult * d_model, d_model),
            nn.Dropout(dropout),
        )

    def forward(self, x):
        return self.net(x)

This tiny two-layer neural network can be broken down as follows:

• Input: token embedding vector (size \(d_{\text{model}}\)).
• Linear layer: expands to \(\text{mult} \times d_{\text{model}}\).
• GELU activation: introduces non-linearity.
• Linear layer: projects back to \(d_{\text{model}}\).
• Dropout: randomly zeroes some activations during training for regularization.

Transformer Block

A Transformer block applies pre-layer normalization, then runs the data through either a multi-head self-attention layer or a feed-forward network (FFN), and adds a residual connection after each. This structure is stacked multiple times to deepen the model.

class TransformerBlock(nn.Module):
    def __init__(self, d_model, n_heads, block_size, dropout):
        super().__init__()
        self.ln1 = nn.LayerNorm(d_model)
        self.ln2 = nn.LayerNorm(d_model)
        self.attn = MultiHeadSelfAttention(d_model, n_heads, block_size, dropout)
        self.ffn  = FeedForward(d_model, mult=4, dropout=dropout)

    def forward(self, x):
        x = x + self.attn(self.ln1(x))
        x = x + self.ffn(self.ln2(x))
        return x

GPT-Style Model Head & Loss

After token and position embeddings are summed, the data flows through a stack of Transformer blocks, each applying self-attention and a feed-forward transformation with residual connections.
Once all blocks have run, we apply a final LayerNorm to normalize the hidden state vectors and keep training stable.

From there, each token’s hidden vector is projected back into vocabulary space — producing a vector of raw scores (logits) for each possible token in the vocabulary.

We also use weight tying here: the projection matrix for mapping hidden vectors to logits is the same matrix as the token embedding layer’s weights.
This reduces the number of parameters, ensures a consistent mapping between tokens and embeddings, and has been shown to improve generalization.

Mathematically, weight tying can be expressed as:

where \(H\) is the matrix of hidden states from the final Transformer layer, and \(E\) is the embedding matrix from the input token embedding layer. This means the output projection reuses (shares) the same weights as the input embedding, just transposed.

class TinyGPT(nn.Module):
    def __init__(self, vocab_size, d_model=128, n_layers=2, n_heads=4, block_size=64, dropout=0.1):
        super().__init__()
        self.block_size = block_size
        self.tok_emb = nn.Embedding(vocab_size, d_model)
        self.pos_emb = nn.Embedding(block_size, d_model)
        self.drop = nn.Dropout(dropout)
        self.blocks = nn.ModuleList([
            TransformerBlock(d_model, n_heads, block_size, dropout)
            for _ in range(n_layers)
        ])
        self.ln_f = nn.LayerNorm(d_model)
        self.head = nn.Linear(d_model, vocab_size, bias=False)
        self.head.weight = self.tok_emb.weight
        self.apply(self._init_weights)

    def _init_weights(self, m):
        if isinstance(m, nn.Linear):
            nn.init.normal_(m.weight, mean=0.0, std=0.02)
            if m.bias is not None: nn.init.zeros_(m.bias)
        elif isinstance(m, nn.Embedding):
            nn.init.normal_(m.weight, mean=0.0, std=0.02)

    def forward(self, idx, targets=None):
        B, T = idx.shape
        assert T <= self.block_size
        tok = self.tok_emb(idx)
        pos = self.pos_emb(torch.arange(T, device=idx.device))
        x = self.drop(tok + pos)
        for blk in self.blocks:
            x = blk(x)
        x = self.ln_f(x)
        logits = self.head(x)
        loss = None
        if targets is not None:
            loss = F.cross_entropy(
                logits.view(B * T, -1),
                targets.view(B * T)
            )
        return logits, loss

Generation Loop

This method performs autoregressive text generation: we start with some initial tokens, repeatedly predict the next token, append it, and feed the result back into the model.

Key concepts:

• Autoregressive: generation proceeds one token at a time, conditioning on all tokens so far.
• Temperature: scales the logits before softmax; values < 1.0 make predictions sharper/more confident, > 1.0 make them more random.
• Top-k filtering: keeps only the k highest-probability tokens and sets all others to negative infinity before sampling, which limits randomness to plausible options.

Step-by-step in generate():

1. Crop context: keep only the last block_size tokens to match the model’s maximum context window.
2. Forward pass: get logits for each position in the sequence.
3. Select last step’s logits: we only want the prediction for the next token.
4. Adjust for temperature (optional).
5. Apply top-k filtering (optional).
6. Softmax: convert logits into a probability distribution.
7. Sample: randomly choose the next token according to the probabilities.
8. Append: add the new token to the sequence and repeat.

This loop continues until max_new_tokens tokens have been generated.

@torch.no_grad()
def generate(self, idx, max_new_tokens, temperature=1.0, top_k=None):
    for _ in range(max_new_tokens):
        idx_cond = idx[:, -self.block_size:]
        logits, _ = self(idx_cond)
        logits = logits[:, -1, :]
        if temperature != 1.0:
            logits = logits / temperature
        if top_k is not None:
            v, _ = torch.topk(logits, top_k)
            thresh = v[:, [-1]]
            logits = torch.where(logits < thresh, torch.full_like(logits, float("-inf")), logits)
        probs = F.softmax(logits, dim=-1)
        next_id = torch.multinomial(probs, num_samples=1)
        idx = torch.cat([idx, next_id], dim=1)
    return idx

In Practice

That concludes the entire stack that we need. We can start to ask questions of this very basic model. Just remember, this is a tiny model so results are not going to be amazing, but it will give you a sense of how these tokens are generated.

After training briefly on a small excerpt of Moby Dick plus a few Q/A lines, we can get:

Q: Why does he go to sea?
A: To drive off the spleen and regulate the circulation.

Even a tiny model learns local structure.

Conclusion

Even though this isn’t the perfect model that will challenge all of the big guys, I hope this has been a bit of a step by step walkthough on how the transformer architecture is put together.

A full version of the code referenced in this article can be found here. The code here includes the training loop so you can run it end-to-end.

D-Bus

Introduction

D-Bus (Desktop Bus) is an inter-process communication (IPC) system used on Linux and other Unix-like systems. It allows different programs — even running as different users — to send messages and signals to each other without needing to know each other’s implementation details.

Main ideas

• Message bus: A daemon (dbus-daemon) runs in the background and acts as a router for messages between applications.
• Two main buses:
  • System bus – for communication between system services and user programs (e.g., NetworkManager, systemd, BlueZ).
  • Session bus – for communication between applications in a user’s desktop session (e.g., a file manager talking to a thumbnailer).
• Communication model:
  • Method calls – like function calls between processes.
  • Signals – broadcast events (e.g., “Wi-Fi disconnected”).
  • Properties – read/write state values.
• Naming:
  • Bus names – unique or well-known IDs for services (e.g., org.freedesktop.NetworkManager).
  • Object paths – hierarchical paths (e.g., /org/freedesktop/NetworkManager).
  • Interfaces – namespaces for methods/signals (e.g., org.freedesktop.NetworkManager.Device).

Here’s a visual representation of the architecture:

User applications call methods or raise signals to a Session Bus inside the D-Bus Daemon. In turn, these messages are routed to System Services, with responses sent back to the applications via the bus.

D-Bus removes the need for each program to implement its own custom IPC protocol. It’s widely supported by desktop environments, system services, and embedded Linux stacks.

In this article, we’ll walk through some basic D-Bus usage, building up to a few practical use cases.

busctl

busctl lets you interact with D-Bus from the terminal. According to the man page:

busctl may be used to introspect and monitor the D-Bus bus.

We can start by listing all connected peers:

busctl list

This shows a list of service names for software and services currently on your system’s bus.

Devices

If you have NetworkManager running, you’ll see org.freedesktop.NetworkManager in the list.
You can query all available devices with:

busctl call org.freedesktop.NetworkManager /org/freedesktop/NetworkManager \
  org.freedesktop.NetworkManager GetDevices

Example output:

ao 6 "/org/freedesktop/NetworkManager/Devices/1" "/org/freedesktop/NetworkManager/Devices/2" "/org/freedesktop/NetworkManager/Devices/3" "/org/freedesktop/NetworkManager/Devices/4" "/org/freedesktop/NetworkManager/Devices/5" "/org/freedesktop/NetworkManager/Devices/6"

Those object paths aren’t very descriptive, so you can query one for its interface name:

busctl get-property org.freedesktop.NetworkManager \
  /org/freedesktop/NetworkManager/Devices/1 \
  org.freedesktop.NetworkManager.Device Interface

On my system:

s "lo"

The leading s tells us this is a string — here, the loopback adapter.

Introspect

You can list all properties, methods, and signals for a given object with:

busctl introspect org.freedesktop.NetworkManager \
  /org/freedesktop/NetworkManager/Devices/1

Or without the pager:

busctl --verbose --no-pager introspect org.freedesktop.NetworkManager \
  /org/freedesktop/NetworkManager/Devices/1

Desktop Notifications

Now that we can query D-Bus, we can also send messages.
For example, you could end a shell script with a visual notification on your desktop:

gdbus call --session \
  --dest org.freedesktop.Notifications \
  --object-path /org/freedesktop/Notifications \
  --method org.freedesktop.Notifications.Notify \
  "my-app" 0 "" "Build finished" "All tests passed" \
  '[]' '{"urgency": <byte 1>}' 5000

Tip: gdbus is part of the glib2 or glib2-tools package on many distributions.

This performs a method call on a D-Bus object.

• --dest — The bus name (service) to talk to.
• --object-path — The specific object inside that service.
• --method — The method we want to invoke.

This method’s signature is s u s s s as a{sv} i, meaning:

Monitoring

D-Bus also lets you watch messages as they pass through.
To monitor all system bus messages (root may be required):

busctl monitor --system

To filter for a specific destination:

busctl monitor org.freedesktop.NetworkManager

These commands stream events to your console in real time.

Conclusion

D-Bus is a quiet but powerful layer in modern Linux desktops and servers. Whether you’re inspecting running services, wiring up automation, or building new desktop features, learning to speak D-Bus gives you a direct line into the heart of the system. Once you’ve mastered a few core commands, the rest is just exploring available services and imagining what you can automate next.

Move Semantics in C++

TL;DR: std::move doesn’t move anything by itself. It’s a cast that permits moving. Real moves happen in your type’s move constructor/assignment. Use them to trade deep copies for cheap pointer swaps and to unlock container performance—provided you mark them noexcept.

The motivating example

We’ll anchor everything on a tiny heap-owning type. It’s intentionally “unsafe” (raw new[]/delete[]) so the ownership transfer is easy to see in logs.

#include <iostream>
#include <utility> // for std::move

struct my_object {
    int* data;
    size_t size;

    // Constructor
    my_object(size_t n) : data(new int[n]), size(n) {
        std::cout << "Constructed (" << this << ") size=" << size 
                  << " data=" << data << "\n";
    }

    // Copy constructor
    my_object(const my_object& other) 
        : data(new int[other.size]), size(other.size) {
        std::copy(other.data, other.data + size, data);
        std::cout << "Copied from (" << &other << ") to (" << this << ")"
                  << " data=" << data << "\n";
    }

    // Move constructor
    my_object(my_object&& other) noexcept 
        : data(other.data), size(other.size) {
        other.data = nullptr;
        other.size = 0;
        std::cout << "Moved from (" << &other << ") to (" << this << ")"
                  << " data=" << data << "\n";
    }

    // Destructor
    ~my_object() {
        std::cout << "Destroying (" << this << ") data=" << data << "\n";
        delete[] data;
    }
};

int main() {
    std::cout << "--- Create obj1 ---\n";
    my_object obj1(5);

    std::cout << "\n--- Copy obj1 into obj2 ---\n";
    my_object obj2 = obj1; // Calls copy constructor

    std::cout << "\n--- Move obj1 into obj3 ---\n";
    my_object obj3 = std::move(obj1); // Calls move constructor

    std::cout << "\n--- End of main ---\n";
}

When you run this you’ll see:

• One deep allocation
• One deep copy (new buffer), and
• One move (no allocation; just pointer steal).

The destructor logs reveal that ownership was transferred and that the moved-from object was neutered.

Try it: clang++ -std=c++20 -O0 -g move_demo.cpp && ./a.out

Having a brief look at the output (from my machine, at least):

--- Create obj1 ---
Constructed (0x7ffd8c960858) size=5 data=0x5616824336c0

--- Copy obj1 into obj2 ---
Copied from (0x7ffd8c960858) to (0x7ffd8c960848) data=0x5616824336e0

--- Move obj1 into obj3 ---
Moved from (0x7ffd8c960858) to (0x7ffd8c960838) data=0x5616824336c0

--- End of main ---
Destroying (0x7ffd8c960838) data=0x5616824336c0
Destroying (0x7ffd8c960848) data=0x5616824336e0
Destroying (0x7ffd8c960858) data=0

• Constructed: obj1 allocates a buffer at 0x5616824336c0.
• Copied: obj2 gets its own buffer (0x5616824336e0) and the contents are duplicated from obj1. At this point, both obj1 and obj2 own separate allocations.
• Moved: obj3 simply takes ownership of obj1’s buffer (0x5616824336c0) without allocating. obj1’s data pointer is nulled out (data=0), leaving it valid but empty.
• Destruction order: obj3 frees obj1’s original buffer, obj2 frees its own copy, and finally obj1 frees nothing (because it’s been neutered by the move).

The contrasting addresses make it easy to see:

• Copies produce different data pointers.
• Moves result in pointer reuse.

What problem do move semantics solve?

Before C++11, passing/returning big objects often meant deep copies or awkward workarounds. Containers like std::vector<T> also had a problem: on reallocation they could only copy elements. If copying T was expensive or forbidden, performance cratered.

Move semantics (C++11) let a type say: “If you no longer need the source object, I can steal its resources instead of allocating/copying them.” This unlocks:

• Returning large objects by value efficiently.
• Growing containers without copying payloads.
• Expressing one-time ownership transfers cleanly.

Conclusion

In this small example we only wrote a move constructor, but real-world resource-owning classes often need both move and copy operations, plus move assignment. The full “rule of five” ensures your type behaves correctly in all situations — and marking moves noexcept can make a big difference in container performance.

Move semantics solves a big problem especially when your class encapsulates a lot of data. It’s an elegant solution that C++ provides you for performance, ownership, and safety.

Building a Modular Synth in Clojure

Introduction

I’ve always liked the idea that a programming language can feel like a musical instrument.
Last night, I decided to make that idea very literal.

The result is rack — a little Clojure module that models a modular synthesizer. It doesn’t aim to be a complete DAW or polished softsynth — this is more of an experiment: what if we could patch together oscillators and filters the way Eurorack folks do, but using s-expressions instead of patch cables?

As always, the code for this article is up in my Github account to help you follow along.

Why Clojure Feels Like a Good Fit

Clojure’s s-expressions are perfect for this kind of modeling.

A synth module is, in some sense, just a little bundle of state and behavior. In OOP we might wrap that up in a class; in Clojure, we can capture it as a simple map, plus a few functions that know how to work with it.

The parentheses give us a “patch cable” feel — data and functions connected in readable chains.

A 30-Second Synth Primer

Before we dive into code, a very quick crash course in some synth lingo:

• VCO (Voltage-Controlled Oscillator): Produces a periodic waveform — the basic sound source.
• LFO (Low-Frequency Oscillator): Like a VCO, but slower, used for modulation (wobble, vibrato, etc.).
• VCA (Voltage-Controlled Amplifier): Controls the amplitude of a signal, usually over time.

That’s enough to make the examples readable. We’re here for the Clojure, not the audio theory.

Setup Audio

The first thing we need to do is open an audio output line.

Java’s javax.sound.sampled API is low-level but accessible from Clojure with no extra dependencies.

Here’s the start of our code:

(def ^:const sample-rate 48000.0)
(def ^:const bits-per-sample 16)
(def ^:const channels 1)

Three constants — a 48 kHz sample rate (good quality, not too CPU-heavy), 16-bit samples, and mono output.

Starting the Audio Engine

(defn ^SourceDataLine open-line
  ([] (open-line sample-rate))
  ([sr]
   (let [fmt (AudioFormat. (float sr) bits-per-sample channels true false) ; signed, little endian
         ^SourceDataLine line (AudioSystem/getSourceDataLine fmt)]
     (.open line fmt 4096)  ;; important: use (fmt, bufferSize) overload
     (.start line)
     line)))

Line by line:

1. Function arity: With no arguments, open-line uses our default sample rate. With one argument, you can pass a custom rate.
2. AudioFormat.: Creates a format object with:
   • sr as a float
   • bits-per-sample bits per sample
   • channels (mono)
   • true for signed samples
   • false for little-endian byte order
3. AudioSystem/getSourceDataLine: Asks the JVM for a line that matches our format.
4. .open: Opens the line with a buffer size of 4096 bytes — small enough for low latency, large enough to avoid dropouts.
5. .start: Starts audio playback.
6. Returns the SourceDataLine object so we can write samples to it.

Finding available outputs

(defn list-mixers []
  (doseq [[i m] (map-indexed vector (AudioSystem/getMixerInfo))]
    (println i ":" (.getName m) "-" (.getDescription m))))

This helper prints out all available audio devices (“mixers”) so you can choose one if your machine has multiple outputs. In cases where you’re struggling to find the appropriate sound mixer, this function can help you diagnose these problems.

Starting, Stopping, and Writing Audio

Opening an audio line is one thing — actually feeding it samples in real time is another.
This is where we start talking about frames, buffers, and a little bit of number crunching.

Writing audio frames

(defn- write-frames! [^SourceDataLine line ^floats buf nframes]
  (let [^bytes out (byte-array (* 2 nframes))]
    (dotimes [i nframes]
      (let [s (int (Math/round (* 32767.0 (double (aget buf i)))))
            idx (* 2 i)]
        (aset-byte out idx       (unchecked-byte (bit-and s 0xFF)))
        (aset-byte out (inc idx) (unchecked-byte (bit-and (unsigned-bit-shift-right s 8) 0xFF)))))
    (.write line out 0 (alength out))))

Here’s what’s happening:

1. Input:
   • buf: A float array of audio samples, each in the range \([-1.0, 1.0]\).
   • nframes: How many samples we want to send.
2. Output:
   • out: A byte array holding the samples in 16-bit little-endian PCM format.

Scaling floats to integers

Most audio hardware expects integers, not floats. In 16-bit PCM, the range is \([-32768, 32767]\).

We scale a float \(x\) by \(32767.0\):

For example:

(close enough; the exact min value is special-cased in PCM)

Breaking into bytes

Because 16 bits = 2 bytes, we split the integer into:

• Low byte: \(s \,\&\, 0xFF\)
• High byte: \((s \gg 8) \,\&\, 0xFF\)

We store them in little-endian order — low byte first — so the audio hardware interprets them correctly.

Stopping audio cleanly

(defn stop-audio! []
  (when (:running? @engine)
    (swap! engine assoc :running? false)
    (when-let [t (:thread @engine)]
      (try (.join t 500) (catch Throwable _))))
  (when-let [^SourceDataLine line (:line @engine)]
    (try (.drain line) (.stop line) (.close line)
         (catch Throwable _)))
  (reset! engine {:running? false :thread nil :line nil})
  :ok)

Stopping audio isn’t just hitting a “pause” button:

• :running? tells the audio thread to exit its loop.
• .join waits briefly for that thread to finish.
• .drain ensures any remaining samples in the buffer are played before stopping.
• .stop and .close free the hardware resources.

Starting audio in real time

(defn start-audio!
  "Start real-time audio. Call stop-audio! to end."
  ([] (start-audio! sample-rate 1024))
  ([sr block-size]
   (stop-audio!)
   (ensure-main-mixer!)
   (let [^SourceDataLine line (open-line sr)
         runner (doto
                  (Thread.
                    (fn []
                      (try
                        (let [ctx (make-ctx sr)]
                          (while (:running? @engine)
                            (let [cache (atom {})
                                  mix   ((:pull ctx) cache ctx "main-mixer" :out block-size)]
                              (write-frames! line mix block-size))))
                        (catch Throwable e
                          (.printStackTrace e))
                        (finally
                          (try (.drain line) (.stop line) (.close line)
                               (catch Throwable _))))))
                  (.setDaemon true))]
     (reset! engine {:running? true :thread runner :line line})
     (.start runner)
     :ok)))

This is where the magic loop happens:

1. block-size is how many frames we process at a time — small enough for low latency, large enough to avoid CPU overload.
2. We open the line, then spin up a daemon thread so it won’t block JVM shutdown.
3. Inside the loop:
   • make-ctx builds a context with our sample rate.
   • (:pull ctx) asks the “main mixer” module for the next block-size frames.
   • We hand those frames to write-frames! to push them to the audio hardware.
4. When :running? goes false, the loop exits, drains the buffer, and closes the line.

How block size relates to latency

Audio latency is fundamentally the time between “we computed samples” and “we hear them.” For a block-based engine, one irreducible component is the block latency:

With our defaults:

So:

That’s the one-way block scheduling delay. Real perceived latency also includes:

• Hardware/driver buffering. We opened the line with 4096 bytes. At 16-bit mono (2 bytes/sample), that’s \(4096 \div 2 = 2048\) samples, i.e.:

  \[\text{latency}_{\text{line}} = \frac{2048}{48000} \approx 42.67\ \text{ms}.\]

• OS and JVM scheduling overhead, which tends to be small but non-zero.

A rough back-of-the-envelope estimate for output-path latency is:

Lowering block-size reduces compute-to-play latency but increases CPU overhead (more wakeups, more function calls). Similarly, if your device/driver allows a smaller .open buffer, you can shave additional milliseconds — at the risk of underruns (clicks/pops). The sweet spot depends on your machine.

Keeping Track of your Patch

A modular synth is basically:

• A set of modules (oscillators, filters, VCAs…)
• A set of connections between module outputs and inputs
• Some engine state for playback

We’ll keep these in atoms so we can mutate them interactively in the REPL.

(defonce ^:private registry (atom {})) ; id -> module
(defonce ^:private cables   (atom #{})) ; set of {:from [id port] :to [id port] :gain g}
(defonce ^:private engine   (atom {:running? false :thread nil :line nil}))

• registry: All modules in the patch, keyed by ID.
• cables: All connections, each with from/to module IDs and ports, plus an optional gain.
• engine: Tracks whether audio is running, plus the playback thread and output line.

Resetting the patch

(defn reset-patch! []
  (reset! registry {})
  (reset! cables #{}))

This wipes everything so you can start a new patch. No modules. No cables.

Adding modules and cables

(defn- register! [m] (swap! registry assoc (:id m) m) (:id m))

(defn add-cable
  "Connect module output → input. Optional gain (defaults 1.0)."
  ([from-id from-port to-id to-port] (add-cable from-id from-port to-id to-port 1.0))
  ([from-id from-port to-id to-port gain]
   (swap! cables conj {:from [from-id (keyword from-port)]
                       :to   [to-id   (keyword to-port)]
                       :gain (double gain)})
   :ok))

register! stores a module in the registry and returns its ID.
add-cable creates a connection between two module ports — think of it as digitally plugging in a patch cable.

These functions are basic data structure management.

Setting parameters generically

(defn set-param!
  "Set a module parameter (e.g., (set-param! \"vco1\" :freq 440.0))."
  [id k v]
  (when-let [st (:state (@registry id))]
    (swap! st assoc k v))
  :ok)

Because each module stores its state in a map, we can update parameters without knowing the module’s internals. This is one of the joys of modeling in Clojure — generic operations fall out naturally.

Pulling Signal

Up to now we can open the device, stream audio, and keep track of a patch.

But how do modules actually produce samples for each block?

We use a pull-based model: when the engine needs N frames from a module’s output port, it asks that module to render. If the module depends on other modules (its inputs), it pulls those first, mixes/filters them, and returns a buffer.

This naturally walks the patch graph from outputs back to sources and avoids doing work we don’t need.

Connections into a port

(defn- connections-into [to-id to-port]
  (filter (fn [{:keys [to]}] (= to [to-id to-port])) @cables))

Plain data → simple query:

• We scan @cables for any connection whose :to is exactly [to-id to-port].
• The result is a (possibly empty) sequence of “incoming patch cables”.

This is intentionally tiny; the interesting part comes when we combine the sources.

Summing signals into a buffer

(defn- sum-into
  "Sum all signals connected to [id port] into a float-array of nframes."
  [cache ctx id port nframes]
  (let [conns (connections-into id port)]
    (if (seq conns)
      (let [acc (float-array nframes)]
        (doseq [{:keys [from gain]} conns
                :let [[src-id src-port] from
                      buf ((:pull ctx) cache ctx src-id src-port nframes)
                      g (float gain)]]
          (dotimes [i nframes]
            (aset-float acc i (+ (aget acc i) (* g (aget ^floats buf i))))))
        acc)
      (float-array nframes))) )

Conceptually, if \(\{x_k[i]\}\) are the input buffers (per-connection) and \(g_k\) are the per-cable gains, the mixed signal is:

Where:

• \(n =\) nframes (the block size),
• \(K =\) number of incoming connections into [id port].

Implementation notes:

• We allocate acc as our accumulator buffer and initialize it to zeros.
• For each incoming connection:
  • We pull from the source (src-id, src-port) via (:pull ctx).
  • We convert the cable’s gain to a float once (keeps the inner loop tight).
  • We add the scaled samples into acc.
• If there are no connections, we return a zeroed buffer (silence). This is a convenient “ground” for the graph.

Time complexity for this step is \(O(K \cdot n)\) per port, which is exactly what you’d expect for mixing \(K\) streams.

Rendering a port with per-block memoization

(defn- render-port
  "Render [id port] with memoization for this audio block."
  [cache ctx id port nframes]
  (if-let [cached (get @cache [id port])]
    cached
    (let [m (@registry id)]
      (when-not m (throw (ex-info (str "Unknown module: " id) {})))
      (let [outbuf ((:process m) ctx m (keyword port) nframes)]
        (swap! cache assoc [id port] outbuf)
        outbuf))))

Why memoize? Consider one VCO feeding two different modules, both ultimately ending at your main mixer. In a naive pull model, the VCO would be recomputed twice per block. We avoid that by caching the result buffer for [id port] the first time it’s pulled in a block:

• cache is an atom (a per-block memo table).
• If we’ve already computed [id port], return the cached buffer.
• Otherwise, we call the module’s :process function, stash the buffer, and return it.

This makes the pull model efficient even when the patch graph has lots of fan-out.

The context object (ctx)

;; ctx provides a way for modules to pull inputs
(defn- make-ctx [sr]
  {:sr sr
   :pull (fn [cache ctx id port nframes]
           (render-port cache ctx id port nframes))})

ctx bundles:

• :sr — the sample rate (modules often need it for phase increments, envelopes, etc.).
• :pull — the function modules call to obtain inputs. This keeps module code simple and testable.

Because :pull closes over render-port, modules don’t need to know about caching details or registry lookups — they just ask the world for “the buffer at [id port]”.

The cache sits beside the graph for the duration of a single block render. Any subsequent pulls of the same [id port] return the memoized buffer.

Numerical notes (clipping and headroom)

Mixing is a straight sum. If your sources are near full-scale and you add them, you can exceed ([-1, 1]) in the mixed float domain, which will later clip when we convert to 16-bit in write-frames!. Options to consider (later):

• Normalize or soft-clip in the mixer: ( y[i] \leftarrow \tanh(y[i]) ) or a gentle limiter.
• Encourage sub-unity gain on cables feeding into mixers.
• Keep VCO defaults conservative (e.g., amplitude (0.2) or (0.5)) to preserve headroom.

Modules

With all of the setup finished, we can finally create some modules — the building blocks of a patch.

The module shape: mk-* vs. public constructor

Each module comes in two layers:

• A maker (mk-vco, mk-lfo, mk-vca, …): returns a plain Clojure map that describes the module:
  • :id, :type, and a mutable :state atom
  • :inputs / :outputs port sets
  • a :process function with the signature
    (:process m) ctx m requested-port nframes -> float-array
• A public constructor (vco, lfo, vca, …): a thin wrapper that calls the maker and then register!s the resulting module into the global registry. This pattern keeps the module definition pure/data-first and the side‑effect (registration) explicit.

The engine always drives modules through :process. If a module needs other signals, it pulls them via sum-into (which uses the per‑block cache and respects cabling).

Voltage Controlled Oscillator (VCO)

A VCO produces periodic waveforms at audio rates. In this design:

• Base frequency is :freq (Hz).
• A control input :pitch (typically from an LFO or envelope) modulates the frequency by :pitch-depth (Hz per unit CV).
• Phase evolves per sample as
  \([ \varphi_{i+1} = \varphi_i + \frac{2\pi}{\text{sr}}\; f_i \quad\text{where}\quad f_i = \max\!\big(0,\; \text{freq} + \text{pitch_depth}\cdot \text{pitch}[i]\big). ]\)
• We render four classic shapes from the same phase accumulator: sine, square, saw, and reverse‑saw, each scaled by :amp.

Note on outputs: this VCO exposes :sine-out, :square-out, :saw-out, and :rev-saw-out. When cabling, target one of those (e.g., :sine-out), not :out.

(defn- mk-vco
  [id {:keys [freq amp pitch-depth]
       :or   {freq 220.0 amp 0.2 pitch-depth 50.0}}]
  (let [state (atom {:phase 0.0
                     :freq (double freq)
                     :amp (double amp)
                     :pitch-depth (double pitch-depth)})]
    {:id id
     :type :vco
     :state state
     :outputs #{:sine-out :square-out :saw-out :rev-saw-out}
     :inputs #{:pitch}
     :process
     (fn [ctx m port nframes]
       (let [{:keys [phase freq amp pitch-depth]} @(:state m)
             sr (:sr ctx)
             pitch-buf (sum-into (atom {}) ctx (:id m) :pitch nframes)
             two-pi (* 2.0 Math/PI)
             ;; output buffers
             sine-buf (float-array nframes)
             square-buf (float-array nframes)
             saw-buf (float-array nframes)
             rev-saw-buf (float-array nframes)]
         ;; run the block, capture final phase
         (let [final-ph
               (loop [i 0, ph phase]
                 (if (< i nframes)
                   (let [hz (max 0.0 (+ freq (* pitch-depth (aget ^floats pitch-buf i))))
                         ph2 (let [p (+ ph (/ (* two-pi hz) sr))]
                               (if (>= p two-pi) (- p two-pi) p))
                         norm-phase (/ ph two-pi) ; 0..1 based on current phase
                         sine (Math/sin ph)
                         square (if (< ph Math/PI) 1.0 -1.0)
                         saw (- (* 2.0 norm-phase) 1.0)
                         rev-saw (- 1.0 (* 2.0 norm-phase))]
                     (aset-float sine-buf i (float (* amp sine)))
                     (aset-float square-buf i (float (* amp square)))
                     (aset-float saw-buf i (float (* amp saw)))
                     (aset-float rev-saw-buf i (float (* amp rev-saw)))
                     (recur (inc i) ph2))
                   ph))]
           ;; persist the advanced phase
           (swap! (:state m) assoc :phase (double final-ph)))
         ;; return the requested port
         (case port
           :sine-out sine-buf
           :square-out square-buf
           :saw-out saw-buf
           :rev-saw-out rev-saw-buf
           (float-array nframes))))}))

(defn vco
  "Create and register a Voltage Controlled Oscillator (VCO) module.

  The VCO generates multiple waveforms and supports pitch modulation via the :pitch input.

  Inputs:
    :pitch — control signal in [-1.0 .. +1.0] range, multiplied by :pitch-depth (Hz)
             and added to :freq.

  Outputs:
    :sine-out, :square-out, :saw-out, :rev-saw-out

  Parameters:
    :freq         — base frequency in Hz (default = 220.0).
    :amp          — peak amplitude (default = 0.2).
    :pitch-depth  — Hz per unit of :pitch CV (default = 50.0).

  Example:
    (vco \"osc1\" {:freq 440.0 :amp 0.25 :pitch-depth 20.0})
    (lfo \"mod1\" {:freq 5.0 :amp 1.0})
    (add-cable \"mod1\" \"sine-out\" \"osc1\" \"pitch\")
    (add-cable \"osc1\" \"sine-out\" \"main-mixer\" \"in\")"
  ([id] (vco id {}))
  ([id params] (register! (mk-vco id params))))

Low Frequency Oscillator (LFO)

An LFO is just an oscillator that runs at control rates (typically < 20 Hz). We use it to modulate other parameters (pitch, amplitude, filter cutoff…). The math is identical to the VCO’s phase increment, just at a lower :freq, and the output is usually not sent directly to the speakers.

(defn- mk-lfo
  [id {:keys [freq amp] :or {freq 2.0 amp 1.0}}]
  (let [state (atom {:phase 0.0 :freq (double freq) :amp (double amp)})]
    {:id id
     :type :lfo
     :state state
     :outputs #{:sine-out}
     :inputs #{}
     :process
     (fn [ctx m port nframes]
       (let [{:keys [phase freq amp]} @(:state m)
             sr (:sr ctx)
             out (float-array nframes)
             two-pi (* 2.0 Math/PI)]
         (let [final-ph
               (loop [i 0, ph phase]
                 (if (< i nframes)
                   (let [ph2 (let [p (+ ph (/ (* two-pi freq) sr))]
                               (if (>= p two-pi) (- p two-pi) p))
                         s (* amp (Math/sin ph))]
                     (aset-float out i (float s))
                     (recur (inc i) ph2))
                   ph))]
           (swap! (:state m) assoc :phase (double final-ph)))
         out))}))

(defn lfo
  "Create and register a Low Frequency Oscillator (LFO) module.

  Outputs:
    :sine-out — control-rate sine in [-amp .. +amp].

  Parameters:
    :freq — Hz (default = 2.0).
    :amp  — peak amplitude (default = 1.0).

  Example:
    (lfo \"mod1\" {:freq 5.0 :amp 1.0})
    (vco \"osc1\" {:freq 220.0 :amp 0.2})
    (add-cable \"mod1\" \"sine-out\" \"osc1\" \"pitch\")"
  ([id] (lfo id {}))
  ([id params] (register! (mk-lfo id params))))

Voltage Controlled Amplifier (VCA)

A VCA scales an audio signal by a gain derived from a control voltage (CV). A common musical use is tremolo: feed a VCO into :in, an LFO into :cv, and you’ll hear periodic amplitude variation.

We map CV \(\in [-1,1]\) to gain \(\in [0,1]\) (plus an optional :bias) using: \([ \text{gain}_i = \operatorname{clamp}_{[0,1]}\!\left(\text{bias} + \tfrac{1}{2}(\text{cv}[i] + 1)\right). ]\)

The output sample is \(y[i] = \text{gain}_i \cdot x[i]\).

(defn- mk-vca [id {:keys [bias] :or {bias 0.0}}]
  (let [state (atom {:bias (double bias)})]
    {:id id
     :type :vca
     :state state
     :inputs #{:in :cv}        ;; audio in, control voltage in [-1..1]
     :outputs #{:out}
     :process
     (fn [ctx m port nframes]
       (let [in  (sum-into (atom {}) ctx (:id m) :in nframes)
             cv  (sum-into (atom {}) ctx (:id m) :cv nframes)
             out (float-array nframes)
             bias (:bias @(:state m))]
         (dotimes [i nframes]
           ;; gain = max(0, bias + 0.5*(cv+1))  -> maps cv [-1..1] to [0..1]
           (let [gain (max 0.0 (min 1.0 (+ bias (* 0.5 (+ 1.0 (aget ^floats cv i))))))
                 s (* gain (aget ^floats in i))]
             (aset-float out i (float s))))
         out))}))

(defn vca
  "Create and register a Voltage Controlled Amplifier (VCA) module.

  Inputs:
    :in   — audio signal (float samples in [-1.0..1.0]).
    :cv   — control voltage signal in [-1.0..+1.0].

  Output:
    :out  — amplified audio.

  Parameter:
    :bias — DC offset added before clamping gain to [0..1] (default 0.0).

  Example (tremolo):
    (vco \"osc\" {:freq 220 :amp 0.25})
    (lfo \"mod\" {:freq 5.0 :amp 1.0})
    (vca \"amp1\" {:bias 0.5})
    (add-cable \"osc\" \"sine-out\" \"amp1\" \"in\")
    (add-cable \"mod\" \"sine-out\" \"amp1\" \"cv\")
    (add-cable \"amp1\" \"out\" \"main-mixer\" \"in\")"
  ([id] (vca id {}))
  ([id params] (register! (mk-vca id params))))

A quick “hello patch”

Tie it together with a gentle vibrato + tremolo:

(reset-patch!)
(ensure-main-mixer!)

(lfo "vib" {:freq 6.0 :amp 1.0})
(lfo "trem" {:freq 4.0 :amp 1.0})
(vco "osc" {:freq 220.0 :amp 0.2 :pitch-depth 8.0})
(vca "amp" {:bias 0.3})

(add-cable "vib"  :sine-out "osc" :pitch)   ;; vibrato
(add-cable "osc"  :sine-out "amp" :in)
(add-cable "trem" :sine-out "amp" :cv)      ;; tremolo
(add-cable "amp"  :out      "main-mixer" :in)

(start-audio!)
;; tweak live:
;; (set-param! "osc"  :freq 330.0)
;; (set-param! "vib"  :freq 5.0)
;; (set-param! "trem" :freq 8.0)
;; (set-param! "amp"  :bias 0.5)
;; ...
(stop-audio!)

With these three modules you can already explore a surprising amount of sonic territory, and the pattern for adding more is clear: define a small :state, specify ports, and implement :process that uses sum-into for inputs and writes a block-sized buffer for outputs.

Targetting the RISC-V Core of the RP2350

Introduction

In our previous post, we got a basic “blinky” app running on the Arm Cortex-M33 side of the RP2350 using Embassy and embassy-rp. This time, we’re reworking the same application to target the RP2350’s RISC-V core instead—highlighting how to boot the RISC-V Hazard 3 with Rust and control peripherals using the rp-hal ecosystem.

This post walks through the key differences and required changes to adapt the project.

Most of this code is available in the examples section of the rp-hal repository.

What is RISC-V?

RISC-V (pronounced “risk-five”) is an open standard instruction set architecture (ISA) that emerged from the University of California, Berkeley in 2010. Unlike proprietary ISAs such as x86 or Arm, RISC-V is open and extensible—allowing anyone to design, implement, and manufacture RISC-V chips without licensing fees.

This openness has led to rapid adoption across academia, startups, and even large chipmakers. RISC-V cores can now be found in everything from tiny embedded microcontrollers to Linux-capable SoCs and even experimental high-performance CPUs.

In the RP2350, RISC-V comes in the form of the Hazard3 core—a lightweight, open-source 3-stage RV32IMAC processor developed by Raspberry Pi. It sits alongside the more familiar Arm Cortex-M33, making the RP2350 one of the first widely accessible dual-ISA microcontrollers.

For embedded developers used to the Arm world, RISC-V introduces a slightly different toolchain and runtime, but the basic concepts—GPIO control, clock configuration, memory mapping—remain very familiar.

In this post, we explore how to bring up a basic RISC-V application targeting the RP2350 Hazard3 core using Rust.

Switching to RISC-V: Overview

The RP2350’s second core is a Hazard3 RISC-V processor. To target it:

• We switch toolchains from thumbv8m.main-none-eabihf to riscv32imac-unknown-none-elf
• We drop the Embassy stack and use the rp235x-hal directly
• We write or reuse suitable linker scripts and memory definitions
• We adjust runtime startup, including clock and GPIO initialization

.cargo/config.toml Changes

We swap the build target and customize linker flags:

[build]
target = "riscv32imac-unknown-none-elf"

[target.riscv32imac-unknown-none-elf]
rustflags = [
    "-C", "link-arg=--nmagic",
    "-C", "link-arg=-Trp235x_riscv.x",
    "-C", "link-arg=-Tdefmt.x",
]
runner = "sudo picotool load -u -v -x -t elf"

Note how we invert the typical linker script behavior: rp235x_riscv.x now includes link.x instead of the other way around.

The Rust target riscv32imac-unknown-none-elf tells the compiler to generate code for a 32-bit RISC-V architecture (riscv32) that supports the I (integer), M (multiply/divide), A (atomic), and C (compressed) instruction set extensions.

The unknown-none-elf part indicates a bare-metal environment with no OS (none) and output in the standard ELF binary format. This target is a common choice for embedded RISC-V development.

Updating the Cargo.toml

Out goes Embassy, in comes rp235x-hal:

[dependencies]
embedded-hal = "1.0.0"
rp235x-hal = { git = "https://github.com/rp-rs/rp-hal", version = "0.3.0", features = [
    "binary-info",
    "critical-section-impl",
    "rt",
    "defmt",
] }
panic-halt = "1.0.0"
rp-binary-info = "0.1.0"

Main Application Rewrite

The runtime is simpler—no executor or async. We explicitly set up clocks, GPIO, and enter a polling loop.

#[hal::entry]
fn main() -> ! {
    let mut pac = hal::pac::Peripherals::take().unwrap();
    let mut watchdog = hal::Watchdog::new(pac.WATCHDOG);
    let clocks = hal::clocks::init_clocks_and_plls(...).unwrap();
    let mut timer = hal::Timer::new_timer0(pac.TIMER0, ...);
    let pins = hal::gpio::Pins::new(...);
    let mut led = pins.gpio25.into_push_pull_output();

    loop {
        led.set_high().unwrap();
        timer.delay_ms(500);
        led.set_low().unwrap();
        timer.delay_ms(500);
    }
}

Linker and Memory Layout

We swapped in a dedicated rp235x_riscv.x linker script to reflect RISC-V memory layout. This script takes care of startup alignment, section placement, and stack/heap boundaries.

The build.rs file was also extended to emit both memory.x and rp235x_riscv.x so that tooling remains consistent across platforms.

Observations and Gotchas

• Clock setup is still necessary, even though the RISC-V HAL avoids some of the abstractions of Embassy.
• Runtime and exception handling differ between Arm and RISC-V: for example, default handlers like DefaultInterruptHandler and DefaultExceptionHandler must be provided.
• The boot block and .bi_entries sections are still necessary for picotool metadata.

Conclusion

Today’s article was only a brief follow up on the first article. All of these changes are available in a risc-v branch that I’ve added to the original repository.