Event sourcing is simpler than it's often made out to be. Here's the core idea:
Don't store current state, store what happened.
Your bank doesn't store "Balance: $527" anywhere except in a cache maybe. They
store every deposit and every withdrawal and compute the balance from the
history on demand.
That's it, that's event sourcing, and it's what every mature industry does.
A mental model
A database as most people know it stores what we call a snapshot:
users table
| id | email | name | updated_at |
|-----|-----------------|-------|------------|
| 123 | bob@example.com | Bob | 2024-01-22 |
You see the current state, it's there but you don't know how it got there. Was
the email address always bob@example.com? When did it change? Who changed it?
In event sourcing we store the history:
events table
| id | aggregate_id | type | data | timestamp |
|----|--------------|-----------------|-------------------------------|------------|
| 1 | user_123 | was_created | {email: "bob@old.com", ...} | 2024-01-01 |
| 2 | user_123 | had_email_changed | {email: "bob@example.com"} | 2024-01-22 |
Current state is derived by replaying these events in order. You get the same
queryable state, plus the complete history.
An example
We'll build a classic shopping cart example. You'll probably need a little more
code if you want to compete with amazon but it'll get the idea across:
{
"aggregate_types": {
"cart": {
"events": {
"was_created": {
"schema": { "type": "object" },
"handler": [
{ "set": { "target": "items", "value": [] } },
{ "set": { "target": "status", "value": "active" } }
]
},
"had_item_added": {
"schema": {
"type": "object",
"properties": {
"sku": { "type": "string" },
"name": { "type": "string" },
"price": { "type": "number" },
"quantity": { "type": "integer" }
},
"required": ["sku", "name", "price", "quantity"]
},
"handler": [
{ "append": { "target": "items", "value": "$.data" } }
]
},
"was_checked_out": {
"handler": [
{ "set": { "target": "status", "value": "checked_out" } }
]
}
}
}
},
"agent_types": ["user", "admin"]
}
I don't know about you but my eyes tend to glaze over looking at a large json
blob. Let's take it piece by piece:
aggregate_types - The things (nouns) your system knows about, like users,
carts, products, etc. For more information see the full spec documentation
hereevents - a list (array) of all the things (verbs) that can happen to
that type of thing (noun). In this case being created, having items added, and
being checked out. We're not implementing removing items or changing quantities
in this example. Who needs it?handlers - what we should do when we come across this event while
reading a stream of events for this aggregate type, defined here in terms of
Tick handler operations, see full documentation here for details.schemas - The shape that an event of this type must have in order to be
written to the permanent record, defined in a subset of the JSON-Schema
format. Please see full documentation for details.agent_types - a list of the kinds of aggregates which can take action in
your system. Don't worry about this for now.
Upload a spec like this to J17 via the dashboard or API and you'll have a
(small) working system.
Writing events
Create a cart:
curl -X POST https://myapp.j17.dev/cart/550e8400-e29b-41d4-a716-446655440000/was_created \
-H "Authorization: Bearer $J17_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"data": {},
"metadata": {
"actor": { "type": "user", "id": "550e8400-e29b-41d4-a716-446655440001" }
}
}'
Add an item:
curl -X POST https://myapp.j17.dev/cart/550e8400-e29b-41d4-a716-446655440000/had_item_added \
-H "Authorization: Bearer $J17_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"data": {
"sku": "WIDGET-1",
"name": "Blue Widget",
"price": 29.99,
"quantity": 2
},
"metadata": {
"actor": { "type": "user", "id": "550e8400-e29b-41d4-a716-446655440001" }
}
}'
Add another:
curl -X POST https://myapp.j17.dev/cart/550e8400-e29b-41d4-a716-446655440000/had_item_added \
-H "Authorization: Bearer $J17_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"data": {
"sku": "GADGET-5",
"name": "Red Gadget",
"price": 49.99,
"quantity": 1
},
"metadata": {
"actor": { "type": "user", "id": "550e8400-e29b-41d4-a716-446655440001" }
}
}'
So what did we just do? We created a cart and added a couple of items to it. In
production you would probably post the create in a batch operation with the
first item rather than create an empty cart proactively but let's not get bogged
down in the details.
Reading state
curl https://myapp.j17.dev/cart/550e8400-e29b-41d4-a716-446655440000 \
-H "Authorization: Bearer $J17_API_KEY"
{
"ok": true,
"data": {
"items": [
{ "sku": "WIDGET-1", "name": "Blue Widget", "price": 29.99, "quantity": 2 },
{ "sku": "GADGET-5", "name": "Red Gadget", "price": 49.99, "quantity": 1 }
],
"status": "active"
},
"metadata": {
"length": 3,
"created_at": 1705914000,
"updated_at": 1705914120
}
}
j17 replayed the three events and computed current state. metadata.length tells
you how many events have been applied. created_at and updated_at are
injected automatically by the engine. By default the whole stream is read every
query, but we can talk about performance concerns, caching, bypassing caching,
and all that another time. For now, computers are fast so don't worry about it
kitten.
Reading history
Here's the real superpower; if you need to debug something, or provide an audit
trail, or refute a claim, or if you're just curious:
curl https://myapp.j17.dev/cart/550e8400-e29b-41d4-a716-446655440000/events \
-H "Authorization: Bearer $J17_API_KEY"
{
"ok": true,
"events": [
{
"stream_id": "1705914000000-0",
"key": "cart:550e8400-e29b-41d4-a716-446655440000",
"type": "was_created",
"data": {},
"metadata": {
"timestamp": 1705914000,
"actor": { "type": "user", "id": "550e8400-e29b-41d4-a716-446655440001" }
}
},
{
"stream_id": "1705914060000-0",
"key": "cart:550e8400-e29b-41d4-a716-446655440000",
"type": "had_item_added",
"data": { "sku": "WIDGET-1", "name": "Blue Widget", "price": 29.99, "quantity": 2 },
"metadata": {
"timestamp": 1705914060,
"actor": { "type": "user", "id": "550e8400-e29b-41d4-a716-446655440001" }
}
},
{
"stream_id": "1705914120000-0",
"key": "cart:550e8400-e29b-41d4-a716-446655440000",
"type": "had_item_added",
"data": { "sku": "GADGET-5", "name": "Red Gadget", "price": 49.99, "quantity": 1 },
"metadata": {
"timestamp": 1705914120,
"actor": { "type": "user", "id": "550e8400-e29b-41d4-a716-446655440001" }
}
}
]
}
Complete audit trail. You know exactly what happened, when, and who did it (this
is where those actor types come in).
More superpowers
Want to travel through time? Query the aggregate with at_length or at_time
parameters and see what that aggregate looked like last tuesday at 4:29 PM, or
after the first 4 events. No rollback system or backups required, just stop the
replay early.
Why it matters
Audit trails are free - Every state change is recorded with timestamp and
context. Compliance, debugging, retrospective analytics you haven't thought to
ask for yet; it's all built in.
Bug isolation - Read-side bugs are easily fixed, change the handlers and
replay. Write-side bugs are trickier, but that domain is clearly separated and
you've got the power of JSON-Schema on your side. There are known strategies if
a write-side bug does sneak into production, I'll write about them later.
History enables features - Undo function? Basically free. Client-visible
time-travel/revision tree? Easy.
Simple concurrency model - Event streams are append only. No locking, no
mutexes, no update conflicts. Optimistic concurrency by default.
The tradeoffs
"There are no solutions, only tradeoffs" - Event sourcing isn't a free lunch.
Storage - Storing every event can add up if you have many thousands of
aggregates with many thousands of events each, but there are good patterns for
dealing with this. See this post for more information.
Perf - Replaying every event is always going to be slower than reading
cached state. Computers are SHOCKINGLY fast though, so you may never need the
caching and indexing features we've implemented to combat this.
Query complexity - Want "all users with email ending in @gmail.com", without
something called projections you'd have to read every user and check each one.
Luckily, we have projections for that.
Mental shift - It's not trivial to do things in a fundamentally different
way. CRUD development is etched in the brainstem of most developers. We think
it's worth knowing both.
What we've learned
Events over snapshots - Store what happened, derive current (or historical)
state from events.
Handlers define state transformations - Each event type has a handler that
builds the aggregate one tick at a time.
History is queryable - Now is just a special case of any other time in the
past.
Immutability buys features - Undo, debugging, analytics; all built in.
That's event sourcing, basically. No PhD required. A practical pattern for any
system where history matters.
Ready to try it? Create a free j17 account now→