Tuesday, November 5, 2024

Prefab Cloud Spanner And PostgreSQL: Flexible And Affordable

- Advertisement -

Prefab’s Cloud Spanner with PostgreSQL: Adaptable, dependable, and reasonably priced for any size

PostgreSQL is a fantastic OLTP database that can serve the same purposes as Redis for real-time access, MongoDB for schema flexibility, and Elastic for data that doesn’t cleanly fit into tables or SQL. It’s like having a Swiss Army knife in the world of databases. PostgreSQL manages everything with elegance, whether you need it for analytics queries or JSON storage. Its transaction integrity is likewise flawless.

- Advertisement -

NoSQL databases, such as HBase, Cassandra, and DynamoDB, are at the other end of the database spectrum. Unlike PostgreSQL’s adaptability, these databases are notoriously difficult to set up, comprehend, and work with. However, their unlimited scalability compensates for their inflexibility. NoSQL databases are the giants of web-scale databases because they can handle enormous amounts of data and rapid read/write performance.

However, is there a database that can offer both amazing scale and versatility?

It might have it both ways after its experience with Spanner.

Why use the PostgreSQL interface from Spanner?

At Prefab, Google uses dynamic logging, feature flags, and secrets management to help developers ship apps more quickly. To construct essential features, including evaluation charts, that aid in it operations, scaling, and product improvement, it employ Cloud Spanner as a data store for its customers’ setups, feature flags, and generated client telemetry.

- Advertisement -

The following are some of the main features that attracted to Spanner:

  • 99.99% uptime by default (multi-availability zone); if you operate in many regions, you can reach up to 99.999% uptime.
  • Robust ACID transactions
  • Scaling horizontally, even for writes
  • Clients, queries, and schemas in PostgreSQL

To put it another way, Spanner offers the ease of use and portability that make PostgreSQL so alluring, along with the robustness and uptime of a massively replicated database on the scale of Google.

How Spanner is used in Prefab

Because Prefab’s architecture is divided into two sections, it made perfect sense for us to have a separate database for each section. This allowed us to select the most appropriate technology for the task. The two aspects of its architecture are as follows:

  • Using Google’s software development kits (SDKs), developers can leverage its core Prefab APIs to serve their clients.
  • Google Cloud clients utilize a web application to monitor and manage their app settings.

In addition to providing incredibly low latency, Google’s feature flag services must be scalable to satisfy the needs of the developers’ downstream clients. With Spanner’s support, Java and the Java virtual machine (JVM) are the ideal options for this high throughput, low latency, and high scalability sector. Although it has a much lower throughput, the user interface (UI) of its program must still enable us to provide features to its clients quickly. It uses PostgreSQL, React, and Ruby on Rails for this section of its architecture.

How Prefab uses Spanner
Image credit to Google Cloud

Spanner in operation

The backend for Google Cloud’s dynamic logging’s volume tracking is one functionality that currently makes use of Cloud Spanner. Its SDK transmits the volume for each log level and logger to Spanner after detecting log requests in its customers’ apps. Then, using the Prefab UI, Google Cloud leverages this information to assist users in determining how many log statements will be output to their log aggregator if they enable logging at different settings.

It need a table with the following shape in order to enable this capture:

CREATE TABLE logger_rollup
(
id varchar(36) NOT NULL,
start_at timestamptz NOT NULL,
end_at timestamptz NOT NULL,
project_id bigint NOT NULL,
project_env_id bigint NOT NULL,
logger_name text NOT NULL,
trace_count bigint NOT NULL,
debug_count bigint NOT NULL,
info_count bigint NOT NULL,
warn_count bigint NOT NULL,
error_count bigint NOT NULL,
fatal_count bigint NOT NULL,
created_at spanner.commit_timestamp,
client_id bigint,
api_key_id bigint,
PRIMARY KEY (project_env_id, logger_name, id)
);

As clients provide the telemetry for Google Cloud’s dynamic logging, this table scales really quickly and erratically. Yes, a time series database or some clever windowing and data removal techniques might potentially be used for this. However, for the sake of this post, this is a simple method to show how Spanner aids in performance management for a table with a large amount of data.

Get 100X storage with no downtime for ⅓ of the cos

It must duplicate Prefab’s database among several zones during production. Because feature flags and dynamic configuration systems are single points of failure by design, reliability is crucial.

Here, Google adopts a belt and suspenders strategy, but its “belt” is robust with Spanner’s uptime SLA and multi-availability zone replication. You would need to treble the cost of a single instance of PostgreSQL to accomplish this. However, replication and automatic failover are included in Cloud Spanner pricing right out of the box. Additionally, you only pay for the bytes you use, and each node has a ton of storage space up to 10TB with Spanner’s latest improvements. This gives the comparison the following appearance for:

Spanner and PostgreSQL at Prefab
Image credit to Google Cloud

The best practice of having a database instance for each environment can become exorbitantly costly at small scales. This was a problem when I initially looked into Spanner a few years back because the least instance size was 1,000 PUs, or one node. Spanner’s scale has since been modified to scale down to less than a whole node, which makes our selection much simpler. Additionally, it allows us to scale up anytime we need to without having to restructure our apps or deal with outages.

Recent enhancements to the Google Cloud ecosystem with Spanner

When we first started using the PostgreSQL interface for Spanner, we encountered several difficulties. Nonetheless, we are thrilled that the majority of the first issues we ran into have been resolved because Google Cloud is always developing and enhancing its goods and services.

Here are a few of our favorite updates:

Query editor: Having a query editor in the Google Cloud console is quite handy as it enables us to examine and optimize any queries that perform poorly.

Key Visualizer: Understanding row keys becomes crucial when examining large-volume NoSQL databases with HBase. It can identify typical problems that lead to hotspots and examine Cloud Spanner data access trends over time with the Key Visualizer.

In brief

Although it has extensive prior experience with HBase and PostgreSQL, it is quite with its choice to use Spanner as Prefab’s preferred horizontally scalable operational database. For its requirements, it has found it to be simple to use, offering all the same scaling capabilities as HBase without the hassles of developing it yourself. It saves time and money when there are fewer possible points of failure and fewer items to manage.

Consider broadening your horizons if you’re afraid of large tables but haven’t explored options other than PostgreSQL. Spanner’s PostgreSQL interface combines the dependable and scalable nature of Cloud Spanner and Google Cloud with the portability and user-friendliness of PostgreSQL.

Start Now

Spanner is available for free for the first ninety days or for as low as $65 a month after that. Additionally, it would be delighted to establish a connection with you and would appreciate it if you could learn more about its Feature Flags, Dynamic Logging, and Secret Management, which are components of the solution built on top of Cloud Spanner.

- Advertisement -
Thota nithya
Thota nithya
Thota Nithya has been writing Cloud Computing articles for govindhtech from APR 2023. She was a science graduate. She was an enthusiast of cloud computing.
RELATED ARTICLES

Recent Posts

Popular Post

Govindhtech.com Would you like to receive notifications on latest updates? No Yes