Prefab’s Cloud Spanner with PostgreSQL: Adaptable, dependable, and reasonably priced for any size
PostgreSQL is a fantastic OLTP database that can serve the same purposes as Redis for real-time access, MongoDB for schema flexibility, and Elastic for data that doesn’t cleanly fit into tables or SQL. It’s like having a Swiss Army knife in the world of databases. PostgreSQL manages everything with elegance, whether you need it for analytics queries or JSON storage. Its transaction integrity is likewise flawless.
NoSQL databases, such as HBase, Cassandra, and DynamoDB, are at the other end of the database spectrum. Unlike PostgreSQL’s adaptability, these databases are notoriously difficult to set up, comprehend, and work with. However, their unlimited scalability compensates for their inflexibility. NoSQL databases are the giants of web-scale databases because they can handle enormous amounts of data and rapid read/write performance.
However, is there a database that can offer both amazing scale and versatility?
It might have it both ways after its experience with Spanner.
Why use the PostgreSQL interface from Spanner?
At Prefab, Google uses dynamic logging, feature flags, and secrets management to help developers ship apps more quickly. To construct essential features, including evaluation charts, that aid in it operations, scaling, and product improvement, it employ Cloud Spanner as a data store for its customers’ setups, feature flags, and generated client telemetry.
The following are some of the main features that attracted to Spanner:
- 99.99% uptime by default (multi-availability zone); if you operate in many regions, you can reach up to 99.999% uptime.
- Robust ACID transactions
- Scaling horizontally, even for writes
- Clients, queries, and schemas in PostgreSQL
To put it another way, Spanner offers the ease of use and portability that make PostgreSQL so alluring, along with the robustness and uptime of a massively replicated database on the scale of Google.
How Spanner is used in Prefab
Because Prefab’s architecture is divided into two sections, it made perfect sense for us to have a separate database for each section. This allowed us to select the most appropriate technology for the task. The two aspects of its architecture are as follows:
- Using Google’s software development kits (SDKs), developers can leverage its core Prefab APIs to serve their clients.
- Google Cloud clients utilize a web application to monitor and manage their app settings.
In addition to providing incredibly low latency, Google’s feature flag services must be scalable to satisfy the needs of the developers’ downstream clients. With Spanner’s support, Java and the Java virtual machine (JVM) are the ideal options for this high throughput, low latency, and high scalability sector. Although it has a much lower throughput, the user interface (UI) of its program must still enable us to provide features to its clients quickly. It uses PostgreSQL, React, and Ruby on Rails for this section of its architecture.
Spanner in operation
The backend for Google Cloud’s dynamic logging’s volume tracking is one functionality that currently makes use of Cloud Spanner. Its SDK transmits the volume for each log level and logger to Spanner after detecting log requests in its customers’ apps. Then, using the Prefab UI, Google Cloud leverages this information to assist users in determining how many log statements will be output to their log aggregator if they enable logging at different settings.
It need a table with the following shape in order to enable this capture:
CREATE TABLE logger_rollup
(
id varchar(36) NOT NULL,
start_at timestamptz NOT NULL,
end_at timestamptz NOT NULL,
project_id bigint NOT NULL,
project_env_id bigint NOT NULL,
logger_name text NOT NULL,
trace_count bigint NOT NULL,
debug_count bigint NOT NULL,
info_count bigint NOT NULL,
warn_count bigint NOT NULL,
error_count bigint NOT NULL,
fatal_count bigint NOT NULL,
created_at spanner.commit_timestamp,
client_id bigint,
api_key_id bigint,
PRIMARY KEY (project_env_id, logger_name, id)
);
As clients provide the telemetry for Google Cloud’s dynamic logging, this table scales really quickly and erratically. Yes, a time series database or some clever windowing and data removal techniques might potentially be used for this. However, for the sake of this post, this is a simple method to show how Spanner aids in performance management for a table with a large amount of data.
Get 100X storage with no downtime for ⅓ of the cos
It must duplicate Prefab’s database among several zones during production. Because feature flags and dynamic configuration systems are single points of failure by design, reliability is crucial.
Here, Google adopts a belt and suspenders strategy, but its “belt” is robust with Spanner’s uptime SLA and multi-availability zone replication. You would need to treble the cost of a single instance of PostgreSQL to accomplish this. However, replication and automatic failover are included in Cloud Spanner pricing right out of the box. Additionally, you only pay for the bytes you use, and each node has a ton of storage space up to 10TB with Spanner’s latest improvements. This gives the comparison the following appearance for:
The best practice of having a database instance for each environment can become exorbitantly costly at small scales. This was a problem when I initially looked into Spanner a few years back because the least instance size was 1,000 PUs, or one node. Spanner’s scale has since been modified to scale down to less than a whole node, which makes our selection much simpler. Additionally, it allows us to scale up anytime we need to without having to restructure our apps or deal with outages.
Recent enhancements to the Google Cloud ecosystem with Spanner
When we first started using the PostgreSQL interface for Spanner, we encountered several difficulties. Nonetheless, we are thrilled that the majority of the first issues we ran into have been resolved because Google Cloud is always developing and enhancing its goods and services.
Here are a few of our favorite updates:
Query editor: Having a query editor in the Google Cloud console is quite handy as it enables us to examine and optimize any queries that perform poorly.
Key Visualizer: Understanding row keys becomes crucial when examining large-volume NoSQL databases with HBase. It can identify typical problems that lead to hotspots and examine Cloud Spanner data access trends over time with the Key Visualizer.
In brief
Although it has extensive prior experience with HBase and PostgreSQL, it is quite with its choice to use Spanner as Prefab’s preferred horizontally scalable operational database. For its requirements, it has found it to be simple to use, offering all the same scaling capabilities as HBase without the hassles of developing it yourself. It saves time and money when there are fewer possible points of failure and fewer items to manage.
Consider broadening your horizons if you’re afraid of large tables but haven’t explored options other than PostgreSQL. Spanner’s PostgreSQL interface combines the dependable and scalable nature of Cloud Spanner and Google Cloud with the portability and user-friendliness of PostgreSQL.
Start Now
Spanner is available for free for the first ninety days or for as low as $65 a month after that. Additionally, it would be delighted to establish a connection with you and would appreciate it if you could learn more about its Feature Flags, Dynamic Logging, and Secret Management, which are components of the solution built on top of Cloud Spanner.
