← Blog

The Most Expensive Code You Ship Does Nothing

· 10 min read
databases surrealdb architecture enterprise

Here's something that should bother you more than it does: the majority of backend code in production right now doesn't compute anything. It doesn't make decisions. It doesn't transform data in meaningful ways. It moves data. It reshapes it. It checks whether you're allowed to see it, then hands it to you in a slightly different format than how it was stored.

We've built an entire industry around writing, deploying, monitoring, and scaling code whose only job is to sit between a client and a database and say "yes, you can read that" or "no, that field isn't for you."

I've shipped this code. Lots of it. And after years of building backend services at insurance companies and startups alike, I'm convinced most of it shouldn't exist.

The Middleware Tax

Think about what a typical backend API actually does. A request comes in. You authenticate the caller. You authorize the specific operation. You validate the input. You translate it into a database query. You run the query. You reshape the response for the client's specific needs. You send it back.

That's six layers of code for what is, fundamentally, a SELECT statement with a WHERE clause. And every one of those layers has to be written, tested, deployed, versioned, and maintained. Every one introduces latency. Every one is a place where bugs hide.

Now multiply that by every consumer. The web app needs the data one way. The mobile client needs it another. The analytics pipeline wants a denormalized dump. The partner API needs a filtered subset. So you build an API gateway or a GraphQL aggregator. Each of which is more middleware, more code that exists only to reshape data and enforce access.

Toggle to compare architectures

The integration surface grows combinatorially. N producers times M consumers, each with its own API contract, its own auth check, its own data mapping. This is the middleware tax, and it's the most expensive code you ship because it does nothing except exist as connective tissue between the thing that has the data and the thing that needs it.

And before someone says "but you still need defense in depth": yes, you do. But defense in depth doesn't mean writing the same auth check in three different services. Put Cloudflare in front for DDoS protection, TLS termination, and connection pooling via Hyperdrive. Let SurrealDB handle authentication, row-level permissions, and query execution with its built-in ACL system, backed by pluggable storage (SurrealKV, TiKV, RocksDB). That's two layers, each doing something the other can't, instead of five layers all doing variations of the same permission check.

I've Lived This

At a large enterprise company, I watched this pattern play out at scale. We had data living in the ticketing platform: tickets, incidents, change records. Useful data. But nobody queried it directly. Instead, we ran nightly ETL jobs to copy everything into the data warehouse. Not because it was better for the workload, but because it was the only place the analytics team had credentials to query.

So every night, a batch job would snapshot the ticketing platform's state, transform it, and dump it into warehouse tables. By morning, the data was already hours stale. But it got worse: other teams would copy from the warehouse into their own downstream systems. Another ETL hop, another copy, another layer of staleness. Data that was real-time at the source was now 12-24 hours behind by the time anyone made a decision from it.

The actual flow: Ticketing Platform → nightly ETL → Data Warehouse → downstream ETL → team dashboards. Each arrow is a batch job, a cron schedule, a transformation script, and a prayer that the schema hasn't changed upstream.

Here's an even more absurd example. I needed a webhook from the ticketing platform, a notification when certain tickets changed state. Three buttons in the admin UI. The team that owned the instance scheduled a meeting with ten people to discuss it. They wanted a follow-up meeting after that. The timeline they quoted? Six to eight weeks. To click three buttons and create a webhook.

This isn't incompetence. It's the logical consequence of a world where every data access path requires a custom integration, a compliance review, a maintenance owner, and a runbook. The organizational overhead of mediating access through APIs and middleware doesn't just slow down the software, it slows down the people. When direct, scoped access to the data exists, all of that ceremony evaporates. You define the permission once, and anyone with the right scope can query it.

This wasn't a one-company problem. This is the enterprise data problem. Every large company I've talked to has some version of this: authoritative data living in one system, consumed through copies in another, with a graveyard of ETL scripts maintaining the illusion that everything is in sync.

Each ETL hop adds hours of staleness. Live queries fan out to all consumers in real-time.

Why Does This Persist?

It's not because engineers are lazy or dumb. It persists because security is the bottleneck that forces the API layer to exist. You can't give the mobile app direct database access because you can't express "user X can only see their own orders" at the database level. You can't let the partner integration query your tables because you can't scope their credentials to only the rows they're entitled to.

So you build an API. The API checks the JWT, looks up the user's roles, constructs a query with the appropriate filters, runs it, and returns the results. The API is the security boundary. Without it, you'd have to trust every client to self-enforce access rules, which is obviously insane.

This is why "just give them database access" has never been a serious answer. Traditional databases authenticate at the connection level. You get a username and password, and that grants you access to tables. Maybe you can set up row-level security policies in Postgres, but they're clunky, hard to test, disconnected from your application's identity model, and certainly don't understand that "this user is a member of organization X and should only see organization X's data."

So the middleware lives on. Not because it adds value, but because the database can't do the one thing that would make it unnecessary: understand who is asking and what they're allowed to see.

The Pivot: What If the Database Handled Auth?

Imagine a database where you don't connect as "postgres_admin" with a shared password. Instead, you authenticate as a specific user or a scoped token. The database knows who you are. Your identity travels with your session, and every query is evaluated against permissions that are defined on the data itself.

Not "this user can access the orders table." More like: "this user can read orders where order.customer_id = $auth.id." Or: "this scope can read product listings but not modify pricing." Or: "users with the admin role can see all records, but viewer scopes see only records where org = $auth.org."

If the database enforced this natively, at the query engine level, before results are even returned, what would happen to all that middleware?

It would disappear. The auth service? Replaced by database-level authentication. The aggregation layer? Unnecessary. Clients query the data they need directly. The API gateway? Reduced to a thin proxy, if you even need one. The ETL pipelines? Replaced by live queries that push changes as they happen.

SurrealDB's Model

This is what SurrealDB actually does, and I'm not saying that as a pitch. I'm saying it because I've built production systems on it and watched the middleware melt away.

SurrealDB has a layered permission system built into the query engine. You define permissions at the table level and field level using the same query language you use for everything else:

SurrealQL
DEFINE TABLE order
  PERMISSIONS
    FOR select
      WHERE customer = $auth.id,
    FOR create
      WHERE $auth.role = 'customer',
    FOR update, delete
      WHERE $auth.role = 'admin';

This isn't a separate config file. It's part of the schema. It lives next to the data. It's version-controlled with the rest of your definitions.

Scoped authentication means you can issue tokens with different levels of access. A mobile app gets a user-scoped token that automatically limits every query to that user's data. An admin dashboard gets a broader scope. A partner integration gets a scope that only exposes the specific tables and fields they need.

Record-level permissions mean the database evaluates access on every individual record, not just at the table level. You don't need to write WHERE customer_id = :current_user in your API. The database does it for you, automatically, on every query.

And then there are live queries. Instead of polling an API or running batch ETL jobs, clients subscribe to changes and get pushed updates in real-time. The staleness problem from those source-to-warehouse pipelines? It just doesn't exist in this model.

Modularity as Architecture

What makes this viable as a backend replacement isn't just that SurrealDB does auth and queries. It's that the entire system is designed to be modular at every layer. The storage engine is pluggable: SurrealKV for embedded single-node deployments, TiKV for distributed clusters, RocksDB for high-throughput local storage, or even in-memory for testing. You pick the storage backend that matches your deployment, not the one the database vendor locked you into.

The protocol layer is heading the same direction. With PostgreSQL wire protocol support landing, SurrealDB becomes a drop-in target for any Postgres client, ORM, or BI tool. That single addition unlocks the entire Postgres ecosystem without sacrificing the multi-model query engine, the permission system, or the graph/document capabilities underneath. Your Grafana dashboard, your Prisma ORM, your legacy reporting tool: they all just connect.

This is the key insight: the reason backends exist is because databases were monolithic and inflexible. They couldn't authenticate users, couldn't enforce row-level access, couldn't push real-time updates, couldn't speak the protocols your tools expected. So you built services to bridge every gap. But when the database itself becomes modular, when it handles auth, permissions, real-time, storage, and protocol adaptation natively, those bridges become redundant. The backend doesn't shrink. It disappears.

Put Cloudflare Hyperdrive in front for global connection pooling and query caching at the edge. SurrealDB handles everything behind that: identity, scoping, record-level ACL, query execution, live subscriptions. Two components, zero middleware, and every piece is independently swappable.

Traditional: 5 hops, 5 services to maintain. SurrealDB: token verification, scoping, ACL, and query execution in one engine.

Building on It

I'm not theorizing here. At Inertia Social, where I was a founding engineer, we built a Go-based blockchain indexer backed by SurrealDB. The indexer processed on-chain events in real-time and stored them in SurrealDB, and the frontend queried the database directly using scoped tokens. No REST API in between. No BFF layer. No GraphQL server translating between the client's query and the database's query.

That experience also exposed gaps. SurrealDB doesn't have built-in rate limiting, which matters when your indexer is hammering the database during high-throughput windows. I proposed a solution to Tobie and the SurrealDB team directly: WASM-based middleware scripts that could track usage and enforce limits at the database level, preventing DoS-style attacks and data exfiltration without needing an external proxy. It's still an open problem a year later, but the collaboration gave me a deep understanding of where the query engine's boundaries actually are.

I also built SurrealGuard, a compile-time validated ORM for SurrealDB in Rust. The motivation was exactly the problem I'm describing: if the database is your security boundary, your queries need to be correct by construction. SurrealGuard validates queries at compile time, ensuring type safety and schema conformance before anything hits the database. It's the kind of tooling that makes direct database access practical in production.

What Actually Disappears

Let me be concrete about what a database-native security model eliminates:

  • Auth middleware: the database verifies tokens and enforces permissions natively
  • Aggregation layers: clients query the data they need directly, shaped by their scope
  • Row-level filtering in application code: record-level permissions handle this automatically
  • ETL pipelines for data distribution: live queries push changes to subscribers in real-time
  • Per-consumer API contracts: each consumer queries what they need with their own scope
  • Most of your integration tests: the permission logic is declarative and co-located with the schema

What you're left with is application logic that actually matters: business rules, computations, workflows that genuinely require code. The plumbing that used to be 60-80% of your backend just isn't there anymore.

The Future is Less Code

The best code is the code you don't write. Not because writing code is hard, but because every line you ship is a line you maintain. Every API endpoint is a versioning commitment. Every ETL pipeline is a fragile chain of assumptions about upstream schemas.

The shift I'm describing isn't "use this specific database." It's a deeper architectural change: move the security boundary into the data layer. When the database understands identity, enforces permissions per-record, and pushes changes in real-time, the entire category of middleware that exists only to mediate access becomes unnecessary.

I've seen this work in practice. Not just in demos or toy apps, but in production systems handling real-time blockchain data at scale. The middleware tax is real, and it's optional. The most expensive code you ship is the code that does nothing, and you can stop shipping it.