Overview of At-Rest Encryption Methods

Let’s start by defining some terms. There are four primary ways to encrypt your data while it is at rest:

OS-Level and Filesystem Encryption

Operating system or disk-level encryption protects entire file systems or disks. This method is application-agnostic and offers encryption with minimal overhead. Think technologies like luks in Linux or FileVault in MacOS.

Pros:

• Transparent to applications and the database
• Simplifies management by applying encryption to the entire storage layer
• Offloads encryption and decryption processing to the OS
• Minimal performance and operational impact
• Widely understood and implemented technology

Cons:

• Less granular control over specific databases or tables
• Backups are not encrypted by default
• Additional overhead is required to ensure encryption keys are properly managed

Storage Device Encryption

Encryption is directly implemented on storage devices such as hard disk drives or SSDs which automatically encrypt all of the data written to their storage.

Pros:

• Suitable for environments with hardware security requirements
• Minimal performance and operational impact
• Offloads encryption and decryption processing to the hardware layer

Cons:

• Less granular control over specific databases or tables
• Additional overhead is required to ensure encryption keys are properly managed

Transparent Disk Encryption (TDE)

In the context of Postgres, TDE means offloading encryption and decryption to the Postgres application. TDE encrypts the entire database, its associated backup files, and the transaction log files, using a database encryption key. This process is transparent to applications, meaning they operate without any changes, as the encryption and decryption happen at the database engine level.

This introduces some complexity and performance overhead as the database must handle all encryption and decryption tasks. Postgres does not currently have this capability built in. Generally, TDE in Postgres is accomplished by forking Postgres, applying patches, and re-building the forked TDE-enabled version.

Pros:

• Encryption at the database level

Cons:

• The database must handle all encryption and decryption for every disk read and write
• Moving away from TDE can be complex and requires an expensive dump and restore process
• Risk of total data loss if keys are not accessible
• Additional overhead is required to ensure encryption keys are properly managed
• Functionality is not native to Postgres and currently requires a forked version of the code
• Data in memory is not encrypted

Application-Level Encryption

Encryption logic is implemented directly within your application code. This method can impact performance and add complexity but offers the most flexibility. You can use tools like pgcrypto alongside your application code to encrypt data at the columnar level, ensuring that your most sensitive data is stored safely. You can create a “dual key” system, where one key unlocks access to the database, and the second key unlocks access to sensitive data stored in the database. The database need never be aware of the second key, as the application uses it to encrypt and decrypt data before it sends it to the database.

Pros:

• Offloads encryption and decryption processing to the application layer
• Allows fine-grained encryption down to the field level
• Enables encryption tailored to the application's specific security needs
• Can be used to give a level of separation of duties and access
• Protects data from non-business users thus giving a level of separation of duties between admins and users

Cons:

• Greatly increases application complexity
• Additional management overhead is required to ensure encryption keys are properly managed
• Can hinder database search, sorting and indexing capabilities

How We Think About Data Encryption

First and foremost, are you encrypting your data in flight? That's just table stakes in our mind. When it comes to at-rest data encryption, you need to think about the options available and your real requirements.

1. Assess Compliance Requirements

Understanding regulatory frameworks and internal requirements is crucial in deciding which encryption strategy to implement. Your specific requirements should guide your decision-making process and help you reach a decision around which technology is the best fit for your environment.

2. Evaluate Existing Architecture

When selecting an encryption strategy, evaluate the existing architecture and available resources. Consider OS support, hardware resources, and storage devices to ensure compatibility and minimal disruption. Think about any operational burden you may incur.

3. Balance Complexity and Security

Finding the right balance between security and performance is critical. Encrypting highly sensitive data with more intensive methods is justified, but less critical data might require less intensive methods. Testing and understanding your requirements can help identify acceptable trade-offs in complexity.

4. Minimize Management Complexity

Encryption solutions should not overwhelm existing management workflows. Effective key management, version compatibility, and alignment with current security operations are essential to minimize additional management overhead.

5. Combine Strategies for Layered Security

Consider combining encrypted storage with in-database encryption for highly sensitive data. Layered security can provide additional safeguards and may enhance overall data protection.

Finding the Right Solution

For General Implementations or Medium to High Security Environments

Most implementations fall into this category. Think about OS-level or filesystem encryption for ease of deployment and compatibility across multiple applications. You can leverage tools like Tablespaces to target more secure underlying storage systems for tables with more sensitive data. Design a solution leveraging filesystem, disk and application level encryption that addresses your specific needs.

For when Disk Encryption is not an option

If you really require transparent disk encryption, evaluate whether storage-level encryption is sufficient. For workloads with stringent data classification requirements that necessitate TDE, ensure that you undertake a proof-of-concepts with clear goals and measurable outcomes. Test and create playbooks for events like data restore to an off-site database. Ensure that you have built operation expertise in key management, security and backup.

Conclusion

Encrypting your data while it is at rest is essential for data security in Postgres environments. By understanding your needs and balancing various encryption approaches, you can achieve optimal data protection without overcomplicating your workflows. With the right strategy, you can ensure security, compliance, and performance, providing a robust solution tailored to each environment's needs. At Crunchy Data we have deep expertise in helping enterprises and agencies navigate these requirements and design secure solutions that meet their requirements while also keeping maintenance and operational complexity low. We have helped countless organizations navigate these challenges and would be happy to discuss this further with you. Reach out to info@crunchydata.com for more information. ]]> 2e728b8128c0f4367956b1f6f19996408d56ffcbfe2ca60fad15ddb30f453f9b Thu, 30 May 2024 06:00:00 EDT 2024-05-30T10:00:00.000Z 2024-05-30T10:00:00.000Z https://www.crunchydata.com/blog/crunchy-postgres-for-kubernetes-5.3-release We are excited to announce the release of Crunchy Postgres for Kubernetes version 5.3. We have been hard at work on a lot of new features that we cannot wait to get into your hands. You can get started on version 5.3 from our Developer Portal or the getting started tutorial. We have decided to highlight a few of our favorite new features and changes today.

Hot off the Presses: Postgres 15 and Kubernetes 1.25

With the latest release of Crunchy Postgres for Kubernetes, we are excited to now natively support Postgres 15. Please note that TimescaleDB and pgAdmin 4 are not currently supported for use with Postgres 15. We are hard at work enabling that functionality for Crunchy Postgres for Kubernetes.

We are also excited to announce that Crunchy Postgres for Kubernetes now offers support for Kubernetes 1.25. Be sure to check your apps’ usage of CronJob, PodDisruptionBudget, or PodSecurityPolicy before upgrading; some of these API versions have been removed in Kubernetes 1.25.

Some of the notable updates in Kubernetes 1.25 that Crunchy Postgres for Kubernetes now supports include:

• Pod Security Admission is stable and Crunchy Postgres for Kubernetes is generally compliant with its baseline policy. It is also compliant with the restricted policy in OpenShift.
• cgroup v2 is stable and works with the bundled pgMonitor v4.8.0 container metrics.

TLS for Exporter

In our ongoing quest to ensure that Crunchy Postgres for Kubernetes remains the default for securely and safely managing Postgres in Kubernetes, we are hard at work to enable safe and sane defaults and settings everywhere we can. We are excited to announce that in Crunchy Postgres for Kubernetes 5.3 you can now enable full TLS for the Postgres Exporter, the tool that exports metrics from the Postgres pods.

Helm Charts

We know a lot of our users like using Helm charts to install Crunchy Postgres for Kubernetes and we are excited to announce that we are now hosting install charts for Crunchy Postgres for Kubernetes 5.3 in our own OCI registry. You can find instructions for using these charts here.

$ helm show chart oci://registry.developers.crunchydata.com/crunchydata/pgo
Pulled: registry.developers.crunchydata.com/crunchydata/pgo:5.3.0
Digest: sha256:7f50f74b0bb4fde32348af87cc740b79ed7be965f90002bcb2425e27fb02a9e3
apiVersion: v2
appVersion: 5.3.0
description: Installer for PGO, the open source Postgres Operator from Crunchy Data
name: pgo
type: application
version: 5.3.0

$ helm install oci://registry.developers.crunchydata.com/crunchydata/pgo
Pulled: registry.developers.crunchydata.com/crunchydata/pgo:5.3.0

IPv6 for pgBackRest

We are excited to announce that IPv6 is now supported for pgBackRest. Users can now successfully deploy Postgres clusters to Kubernetes environments that are configured for IPv6 only!  You can find directions for setting that up in the Backup Configuration section of the docs.

Open Source Contributions

Of course a large thank you should go out to our Open Source contributors who help make the Postgres Operator better for everyone. During the last quarter these are the highlights of their contributions:

• JIT is now explicitly disabled for the monitoring user, allowing users to opt-into using JIT elsewhere in the database without impacting exporter functionality. Contributed by Kirill Petrov (@chobostar).
• PGO now logs both stdout and stderr when running a SQL file referenced via spec.databaseInitSQL during database initialization. Contributed by Jeff Martin (@jmartin127).
• Limit the monitoring user to local connections using SCRAM authentication. Contributed by Scott Zelenka (@szelenka)
• Skip a scheduled backup when the prior one is still running. Contributed by Scott Zelenka (@szelenka)

If you would like to lend a hand with PGO development, get started by reviewing the contributing guidelines.

And More…

We are very excited to bring you this next version of Crunchy Postgres for Kubernetes. This is just a sampling of the new features and fixes that we have shipped with this version. We hope that you enjoy using it, and as always we value our community's feedback. The full feature notes are available in our documentation. ]]> 1baa02b6e2a4226d7112b73c5bfbb3d649327168860b26d0534e7cb7bb7fbf10 Wed, 21 Dec 2022 10:00:00 EST 2022-12-21T15:00:00.000Z 2022-12-21T15:00:00.000Z https://www.crunchydata.com/blog/crunchy-postgres-for-kubernetes-launch-5.2 We are excited to announce the release of Crunchy Postgres for Kubernetes version 5.2. We have been hard at work on a lot of new features we cannot wait to get into your hands. You can get started on version 5.2 from our container portal or the get started tutorial. We have decided to highlight a few of our favorite features today.

Command Lines for Everyone

First, we are very excited to release the first iteration of our CLI for PGO v5, pgo. Our CLI is designed as a kubectl plugin. You can get it here and you can use it today with PGO 5.2. We have taken steps to align our CLI very closely to kubectl for ease of use. pgo also works as a plugin with the OpenShift oc CLI. This ensures that developer experience matches the rest of Kubernetes.

For example, you can now use the following command to create a cluster:

pgo create postgrescluster hippo

Or this command to backup a cluster:

pgo backup hippo --repoName="repo1”

This is a preview release of the CLI, and we will be working hard on adding more commands and workflows. We welcome feedback on what you would like to see included.

Streaming Replication

Enabling opinionated and safe streaming replication for Kubernetes workloads was one of our goals in 5.2 to increase the choices for achieving disaster recovery. Streaming replication can be faster than log shipping and does not rely on a single s3 storage area. You can read more about how to turn this feature on in the documentation.

This feature allows new architecture choices for replicas and standbys, with low latency direct interconnectivity. Now we support streaming replication as well as the use of both streaming and a cloud-based WAL archive in conjunction.

Feature Gates and Sidecars

We have added two feature gates related to sidecars. You can now add any container you want to Postgres or PgBouncer pods through the PostgresCluster spec. It can be an observability agent like DataDog or New Relic, a network proxy like poolers or service meshes, or an API endpoint like Hasura. The possibilities are endless and we’re excited to see what our users build. We relaxed the runAsNonRoot security setting so you can run more images from your favorite registries. This opens up more custom use cases in a safe and opinionated fashion. This will allow folks on 5.2 additional flexibly to design a system that works well for their needs. You can read more in the documentation.

We will be working with customers and partners to release documented sidecar patterns in the coming months, and we welcome your pull requests if you want to share.

Easy Upgrades

With the 5.x version of PGO, we have focused on making the upgrade process easier. One improvement that we have delivered with 5.2 is the new Pause Reconcile feature. This enables customers to pause execution of a reconcile until such a time as it is convenient. This will allow users to stage changes, and execute them when it is convenient.

Upgrading your PGO version requires just a few simple non-destructive steps to. For example, if you are on 5.1 series, updating to 5.2 can be as easy as:

kubectl apply --server-side -k kustomize/install/default

And More…

We are very excited to bring you this next version of Postgres for Kubernetes. This is just a sampling of the new features and fixes that we have shipped with this version. We hope that you enjoy using it, and as always we value our community's feedback. The full feature notes are available in our documentation.



co-authored with Andrew L'Ecuyer and Chris Bandy ]]> c12bd2d81b2118a88f3316d97507adf672128e9baf8e18d0857364a8f18b4c70 Fri, 09 Sep 2022 11:00:00 EDT 2022-09-09T15:00:00.000Z 2022-09-09T15:00:00.000Z