When simulating physical systems—whether it’s a bouncing ball, orbiting planets, or particles under gravity—accurately
updating positions and velocities over time is crucial. This process is known as time integration, and it’s the
backbone of most game physics and real-time simulations.
In this post, we’ll explore two fundamental methods for time integration: Euler’s method and Runge-Kutta 4 (RK4).
We’ll go through how each of these methods is represented mathemtically, and then we’ll translate that into code.
We’ll build a small visual simulation in Python using pygame to see how the two methods behave differently when
applied to the same system.
The Simulation
Our simulation consists of a central massive object (a “sun”) and several orbiting bodies, similar to a simplified
solar system. Each body is influenced by the gravitational pull of the others, and we update their positions and
velocities in each frame of the simulation loop.
At the heart of this simulation lies a decision: how should we advance these objects forward in time? This is where the
integration method comes in.
Euler’s Method
Euler’s method is the simplest way to update motion over time. It uses the current velocity to update position, and the
current acceleration to update velocity:
This is easy to implement, but has a major downside: error accumulates quickly, especially in systems with strong
forces or rapidly changing directions.
Here’s an example of it running:
RK4
Runge-Kutta 4 (RK4) improves on Euler by sampling the system at multiple points within a single timestep. It estimates
what will happen halfway through the step, not just at the beginning. This gives a much better approximation of curved
motion and reduces numerical instability.
Runge-Kutta 4 samples the derivative at four points:
RK4 requires more code and computation, but the visual payoff is immediately clear: smoother orbits, fewer explosions,
and longer-lasting simulations.
Here’s an example of it running:
Trade-offs
Euler is fast and simple. It’s great for prototyping, simple games, or systems where precision isn’t critical.
RK4 is more accurate and stable, especially in chaotic or sensitive systems—but it’s computationally more expensive.
In real-time applications (like games), you’ll need to weigh performance vs. quality.
Also, both methods depend heavily on the size of the timestep. Larger steps amplify error; smaller ones improve
accuracy at the cost of performance.
Conclusion
Switching from Euler to RK4 doesn’t just mean writing more code—it fundamentally changes how your simulation evolves over time. If you’re seeing odd behaviors like spiraling orbits, exploding systems, or jittery motion, trying a higher-order integrator like RK4 might fix it.
Or, it might inspire a deeper dive into the world of numerical simulation—welcome to the rabbit hole!
You can find the full code listing here as a gist,
so you can tweak and run it for yourself.
I recently decided to dip my toes into ClojureScript. As someone who enjoys exploring different language ecosystems, I
figured getting a basic “Hello, World!” running in the browser would be a fun starting point. It turns out that even
this small journey taught me quite a bit about how ClojureScript projects are wired together.
This post captures my first successful setup: a minimal ClojureScript app compiled with lein-cljsbuild, rendering
output in the browser console.
A Rough Start
I began with the following command to create a new, blank project:
lein new cljtest
First job from here is to organise dependencies, and configure the build system for the project.
project.clj
There’s a few things to understand in the configuration of the project:
We add org.clojure/clojurescript "1.11.132" as a dependency
To assist with our builds, we add the plugin lein-cljsbuild "1.1.8"
The source path is normally src, but we change this for ClojureScript to src-cljs
The output will be javascript output for a website, and all of our web assets go into resources/public
We have two different build configurations here: dev and prod.
The dev configuration focuses on being much quicker to build so that the change / update cycle during development is
quicker. Source maps, pretty printing, and no optimisations provide the verbose output appropriate for debugging.
The prod configuration applies all the optimisations. This build is slower, but produces one single output file:
main.js. This is the configuration that you use to “ship” your application.
Your First ClojureScript File
Place this in src-cljs/cljtest/core.cljs:
(ns cljtest.core)
(enable-console-print!)
(println "Hello from ClojureScript!")
Then open resources/public/index.html in your browser, and check the developer console — you should see your message.
If you want to iterate while coding:
lein cljsbuild auto dev
When you’re ready to build a production bundle:
lein cljsbuild once prod
Then you can simplify the HTML:
<script src="js/main.js"></script>
No goog.require needed — it all gets bundled.
Step it up
Next, we’ll step up to something a little more useful. We’ll put together a table of names that we can add, edit,
delete, etc. Just a really simple CRUD style application.
In order to do this, we’re going to rely on a pretty cool library called reagent.
names is the currentl list of names. next-id gives us the next value that we’ll use an ID when adding a new
record. editing-id and edit-text manage the state for updates.
Table
We can now render our table using a simple function:
The table renders all of the names, as well and handles the create case. The edit case is a little more complex and
requires a function of its own. The name-row function manages this complexity for us.
<!doctype html><html><head><metacharset="utf-8"><title>cljtest</title></head><body><h1>cljtest</h1><!-- This is our new element! --><divid="app"></div><script src="js/out/goog/base.js"></script><script src="js/main.js"></script><script>goog.require('cljtest.core');cljtest.core.init();</script></body></html>
Conclusion
This journey started with a humble goal: get a simple ClojureScript app running in the browser. Along the way, I
tripped over version mismatches, namespace assumptions, and nested anonymous functions — but I also discovered the
elegance of Reagent and the power of functional UIs in ClojureScript.
While the setup using lein-cljsbuild and Reagent 1.0.0 may feel a bit dated, it’s still a solid way to learn the
fundamentals. From here, I’m looking forward to exploring more advanced tooling like Shadow CLJS, integrating external
JavaScript libraries, and building more interactive UIs.
This was my first real toe-dip into ClojureScript, and already I’m hooked. Stay tuned — there’s more to come.
Natural language processing (NLP) has gone through several paradigm shifts:
Bag-of-Words — treated text as unordered word counts; no sequence information. We’ve spoken about this previously.
Word Embeddings (word2vec, GloVe) — learned fixed-vector representations that captured meaning. We’ve looked at these previously.
RNNs, LSTMs, GRUs — processed sequences token-by-token, retaining a hidden state; struggled with long-range dependencies due to vanishing gradients.
Seq2Seq with Attention — attention helped the model “focus” on relevant input tokens; a leap in translation and summarization.
Transformers (Vaswani et al., 2017 — “Attention Is All You Need”) — replaced recurrence entirely with self-attention, allowing parallelization and longer context handling.
Transformers didn’t just improve accuracy; they unlocked the ability to scale models massively.
In this post, we’ll walk though an understanding of the transformer architecture by implementing a GPT-style
Transformer from scratch in PyTorch, from tokenization to text generation.
The goal: make the architecture concrete and understandable, not magical.
Overview
At a high level, our model will:
Tokenize text into integers.
Map tokens to dense embeddings + positional encodings.
Apply self-attention to mix contextual information.
Use feed-forward networks for per-token transformations.
Wrap attention + FFN in Transformer Blocks with residual connections and layer normalization.
Project back to vocabulary logits.
Generate text autoregressively.
graph TD
A[Text Input] --> B[Tokenizer]
B --> C[Token Embeddings + Positional Encoding]
C --> D[Transformer Block × N]
D --> E[Linear Projection to Vocabulary Size]
E --> F[Softmax Probabilities]
F --> G[Sample / Argmax Next Token]
G -->|Loop| C
Tokenization
Before our model can process text, we need to turn characters into numbers it can work with — a process called
tokenization. In this example, we use a simple byte-level tokenizer, which treats every UTF-8 byte as its own token.
This keeps the implementation minimal while still being able to represent any possible text without building a custom
vocabulary.
classByteTokenizer:"""
UTF-8 bytes <-> ints in [0..255].
NOTE: For production models you'd use a subword tokenizer (BPE, SentencePiece).
"""def__init__(self)->None:self.vocab_size=256defencode(self,text:str)->list[int]:returnlist(text.encode("utf-8"))defdecode(self,ids:list[int])->str:returnbytes(ids).decode("utf-8",errors="ignore")
Once we have token IDs, we map them into embedding vectors — learned dense representations that capture meaning in
a continuous space. Each token ID indexes a row in an embedding matrix, turning a discrete integer into a trainable
vector of size \(d_{\text{model}}\). Because self-attention alone has no sense of order, we also add
positional embeddings, giving the model information about each token’s position within the sequence.
That equation means each token computes a similarity score with all other tokens (via \(QK^\top\)), scales it
by \(\sqrt{d_k}\) to stabilize gradients, turns the scores into probabilities with softmax, and then uses those
probabilities to take a weighted sum of the value vectors \(V\) to produce its new representation.
Multi-head attention runs this in parallel on different projections.
Linear layer: expands to \(\text{mult} \times d_{\text{model}}\).
GELU activation: introduces non-linearity.
Linear layer: projects back to \(d_{\text{model}}\).
Dropout: randomly zeroes some activations during training for regularization.
Transformer Block
A Transformer block applies pre-layer normalization, then runs the data through either a multi-head self-attention
layer or a feed-forward network (FFN), and adds a residual connection after each. This structure is stacked multiple
times to deepen the model.
graph TD
A[Input] --> B[LayerNorm]
B --> C[Multi-Head Self-Attention]
C --> D[Residual Add]
D --> E[LayerNorm]
E --> F[Feed-Forward Network]
F --> G[Residual Add]
G --> H[Output to Next Block]
After token and position embeddings are summed, the data flows through a stack of Transformer blocks, each applying
self-attention and a feed-forward transformation with residual connections.
Once all blocks have run, we apply a final LayerNorm to normalize the hidden state vectors and keep training stable.
From there, each token’s hidden vector is projected back into vocabulary space — producing a vector of raw
scores (logits) for each possible token in the vocabulary.
We also use weight tying here: the projection matrix for mapping hidden vectors to logits is the same matrix as
the token embedding layer’s weights.
This reduces the number of parameters, ensures a consistent mapping between tokens and embeddings, and has been shown
to improve generalization.
Mathematically, weight tying can be expressed as:
\[\text{logits} = H \cdot E^\top\]
where \(H\) is the matrix of hidden states from the final Transformer layer, and \(E\) is the embedding matrix
from the input token embedding layer. This means the output projection reuses (shares) the same weights as the input
embedding, just transposed.
This method performs autoregressive text generation: we start with some initial tokens, repeatedly predict the
next token, append it, and feed the result back into the model.
Key concepts:
Autoregressive: generation proceeds one token at a time, conditioning on all tokens so far.
Temperature: scales the logits before softmax; values < 1.0 make predictions sharper/more confident, > 1.0 make them more random.
Top-k filtering: keeps only the k highest-probability tokens and sets all others to negative infinity before sampling, which limits randomness to plausible options.
Step-by-step in generate():
Crop context: keep only the last block_size tokens to match the model’s maximum context window.
Forward pass: get logits for each position in the sequence.
Select last step’s logits: we only want the prediction for the next token.
Adjust for temperature (optional).
Apply top-k filtering (optional).
Softmax: convert logits into a probability distribution.
Sample: randomly choose the next token according to the probabilities.
Append: add the new token to the sequence and repeat.
This loop continues until max_new_tokens tokens have been generated.
That concludes the entire stack that we need. We can start to ask questions of this very basic model. Just remember,
this is a tiny model so results are not going to be amazing, but it will give you a sense of how these tokens are
generated.
After training briefly on a small excerpt of Moby Dick plus a few Q/A lines, we can get:
Q: Why does he go to sea?
A: To drive off the spleen and regulate the circulation.
Even a tiny model learns local structure.
Conclusion
Even though this isn’t the perfect model that will challenge all of the big guys, I hope this has been a bit of a step
by step walkthough on how the transformer architecture is put together.
A full version of the code referenced in this article can be found here.
The code here includes the training loop so you can run it end-to-end.
D-Bus (Desktop Bus) is an inter-process communication (IPC) system
used on Linux and other Unix-like systems. It allows different programs — even running as different users — to send
messages and signals to each other without needing to know each other’s implementation details.
Main ideas
Message bus: A daemon (dbus-daemon) runs in the background and acts as a router for messages between applications.
Two main buses:
System bus – for communication between system services and user programs (e.g., NetworkManager, systemd, BlueZ).
Session bus – for communication between applications in a user’s desktop session (e.g., a file manager talking to a thumbnailer).
Communication model:
Method calls – like function calls between processes.
Interfaces – namespaces for methods/signals (e.g., org.freedesktop.NetworkManager.Device).
Here’s a visual representation of the architecture:
flowchart LR
subgraph AppLayer[User Applications]
A1[App 1]
A2[App 2]
end
subgraph DBusDaemon[D-Bus Daemon Message Bus]
D1[System Bus]
D2[Session Bus]
end
subgraph SysServices[System Services]
S1[NetworkManager]
S2[BlueZ Bluetooth]
S3[systemd-logind]
end
%% Connections
A1 --method calls or signals--> D2
A2 --method calls or signals--> D2
S1 --method calls or signals--> D1
S2 --method calls or signals--> D1
S3 --method calls or signals--> D1
%% Cross communication
D1 <-->|routes messages| A1
D1 <-->|routes messages| A2
D2 <-->|routes messages| A1
D2 <-->|routes messages| A2
%% System bus to service connections
D1 <-->|routes messages| S1
D1 <-->|routes messages| S2
D1 <-->|routes messages| S3
User applications call methods or raise signals to a Session Bus inside the D-Bus Daemon. In turn,
these messages are routed to System Services, with responses sent back to the applications via the bus.
D-Bus removes the need for each program to implement its own custom IPC protocol. It’s widely supported by desktop
environments, system services, and embedded Linux stacks.
In this article, we’ll walk through some basic D-Bus usage, building up to a few practical use cases.
busctl
busctl lets you interact with D-Bus from the terminal. According to the man page:
busctl may be used to introspect and monitor the D-Bus bus.
We can start by listing all connected peers:
busctl list
This shows a list of service names for software and services currently on your system’s bus.
Devices
If you have NetworkManager running, you’ll see org.freedesktop.NetworkManager in the list.
You can query all available devices with:
Tip:gdbus is part of the glib2 or glib2-tools package on many distributions.
This performs a method call on a D-Bus object.
--dest — The bus name (service) to talk to.
--object-path — The specific object inside that service.
--method — The method we want to invoke.
This method’s signature is s u s s s as a{sv} i, meaning:
Code
Type Description
Example Value
Meaning
s
string
"my-app"
Application name
u
uint32
0
Notification ID (0 = new)
s
string
""
Icon name/path
s
string
"Build finished"
Title
s
string
"All tests passed"
Body text
as
array of strings
'[]'
Action identifiers
a{sv}
dict<string, variant>
'{"urgency": <byte 1>}'
Hints (0=low, 1=normal, 2=critical)
i
int32
5000
Timeout (ms)
Monitoring
D-Bus also lets you watch messages as they pass through.
To monitor all system bus messages (root may be required):
busctl monitor --system
To filter for a specific destination:
busctl monitor org.freedesktop.NetworkManager
These commands stream events to your console in real time.
Conclusion
D-Bus is a quiet but powerful layer in modern Linux desktops and servers. Whether you’re inspecting running services,
wiring up automation, or building new desktop features, learning to speak D-Bus gives you a direct line into the heart
of the system. Once you’ve mastered a few core commands, the rest is just exploring available services and
imagining what you can automate next.
TL;DR: std::move doesn’t move anything by itself. It’s a cast that permits moving. Real moves happen in your
type’s move constructor/assignment. Use them to trade deep copies for cheap pointer swaps and to unlock container
performance—provided you mark them noexcept.
The motivating example
We’ll anchor everything on a tiny heap-owning type. It’s intentionally “unsafe” (raw new[]/delete[]) so the
ownership transfer is easy to see in logs.
#include<iostream>
#include<utility> // for std::movestructmy_object{int*data;size_tsize;// Constructormy_object(size_tn):data(newint[n]),size(n){std::cout<<"Constructed ("<<this<<") size="<<size<<" data="<<data<<"\n";}// Copy constructormy_object(constmy_object&other):data(newint[other.size]),size(other.size){std::copy(other.data,other.data+size,data);std::cout<<"Copied from ("<<&other<<") to ("<<this<<")"<<" data="<<data<<"\n";}// Move constructormy_object(my_object&&other)noexcept:data(other.data),size(other.size){other.data=nullptr;other.size=0;std::cout<<"Moved from ("<<&other<<") to ("<<this<<")"<<" data="<<data<<"\n";}// Destructor~my_object(){std::cout<<"Destroying ("<<this<<") data="<<data<<"\n";delete[]data;}};intmain(){std::cout<<"--- Create obj1 ---\n";my_objectobj1(5);std::cout<<"\n--- Copy obj1 into obj2 ---\n";my_objectobj2=obj1;// Calls copy constructorstd::cout<<"\n--- Move obj1 into obj3 ---\n";my_objectobj3=std::move(obj1);// Calls move constructorstd::cout<<"\n--- End of main ---\n";}
When you run this you’ll see:
One deep allocation
One deep copy (new buffer), and
One move (no allocation; just pointer steal).
The destructor logs reveal that ownership was transferred and that the moved-from object was neutered.
Having a brief look at the output (from my machine, at least):
--- Create obj1 ---
Constructed (0x7ffd8c960858) size=5 data=0x5616824336c0
--- Copy obj1 into obj2 ---
Copied from (0x7ffd8c960858) to (0x7ffd8c960848) data=0x5616824336e0
--- Move obj1 into obj3 ---
Moved from (0x7ffd8c960858) to (0x7ffd8c960838) data=0x5616824336c0
--- End of main ---
Destroying (0x7ffd8c960838) data=0x5616824336c0
Destroying (0x7ffd8c960848) data=0x5616824336e0
Destroying (0x7ffd8c960858) data=0
Constructed: obj1 allocates a buffer at 0x5616824336c0.
Copied: obj2 gets its own buffer (0x5616824336e0) and the contents are duplicated from obj1.
At this point, both obj1 and obj2 own separate allocations.
Moved: obj3 simply takes ownership of obj1’s buffer (0x5616824336c0) without allocating.
obj1’s data pointer is nulled out (data=0), leaving it valid but empty.
Destruction order: obj3 frees obj1’s original buffer, obj2 frees its own copy, and finally obj1 frees nothing (because it’s been neutered by the move).
The contrasting addresses make it easy to see:
Copies produce different data pointers.
Moves result in pointer reuse.
What problem do move semantics solve?
Before C++11, passing/returning big objects often meant deep copies or awkward workarounds. Containers like
std::vector<T> also had a problem: on reallocation they could only copy elements. If copying T was expensive or
forbidden, performance cratered.
Move semantics (C++11) let a type say: “If you no longer need the source object, I can steal its resources
instead of allocating/copying them.” This unlocks:
Returning large objects by value efficiently.
Growing containers without copying payloads.
Expressing one-time ownership transfers cleanly.
Conclusion
In this small example we only wrote a move constructor, but real-world resource-owning classes often need both move
and copy operations, plus move assignment. The full “rule of five” ensures your type behaves correctly in all
situations — and marking moves noexcept can make a big difference in container performance.
Move semantics solves a big problem especially when your class encapsulates a lot of data. It’s an elegant solution that
C++ provides you for performance, ownership, and safety.