PlanetScale has unveiled Neki, a new platform that extends Postgres databases by implementing sharding to handle hundreds of millions of queries per second and petabytes of data. Unlike traditional Postgres deployments limited by single-machine capacity, Neki distributes data across multiple real Postgres shards. It incorporates a router, sidecars, and a control plane to manage scaling seamlessly without downtime.
Neki supports dynamic resharding by splitting hot shards as workloads increase, replicating data to new shards, and rerouting traffic without interrupting database availability. This allows organizations to rebalance capacity and adjust shard layouts online, avoiding maintenance windows or application changes.
Each shard operates as a fully functional Postgres instance, with Neki managing health monitoring, replica promotion, and routing updates. Applications continue to connect through a consistent Postgres endpoint, simplifying integration.
The platform also coordinates schema changes across shards, applying appropriate methods per shard and tracking progress to ensure consistency. Connection pooling is handled within the sharded layer, with routing decisions informed by topology, shard health, and session state.
Neki enables workload isolation by assigning tables or data to different shard groups optimized for specific hardware profiles. High availability is maintained by running each shard as a Postgres cluster across availability zones, minimizing failure impact.
Data placement is explicitly defined and reviewable, facilitating controlled resharding. Neki supports distributed transactions that span multiple shards, ensuring atomic commits.
Operational workflows for imports, resharding, schema changes, and migrations are repeatable and automated. The platform also supports online upgrades to new Postgres versions by replicating data to new shard groups and performing seamless cutovers.
Change data capture remains compatible with standard Postgres patterns, with each shard maintaining its own write-ahead log and logical replication stream.
Neki aims to provide enterprises with a scalable, resilient Postgres solution capable of handling demanding workloads without sacrificing compatibility or requiring downtime.