Protecting Our Customers' DNS Credentials with a Stateless Go Proxy

In this article, we show you how we protect the API credentials used to integrate with our customers' DNS providers. We are sharing this not just to showcase SSLMate's security engineering practices, but also because we hope to encourage other SaaS companies to apply this technique when integrating with their customers' accounts at other providers.

Background

Before you can get a publicly-trusted SSL certificate, you have to demonstrate control over the domains in the certificate. One of the ways to demonstrate control is to publish a special DNS record under each domain. The certificate authority looks up the records, and only issues the certificate if the records are correct.

SSLMate integrates with several popular DNS providers to automatically publish the necessary DNS record when a customer requests a certificate through the SSLMate API or CLI, as well as when a certificate renews. This makes it easy to get certificates through SSLMate.

To make this work, SSLMate needs API credentials to access the DNS accounts of our customers. These are powerful credentials, and accordingly we consider them among the most sensitive information we hold. We can't just store them in our database unprotected. Even though SSLMate employs best practices, like using prepared statements to prevent SQL injection, and applying regular security updates to our infrastructure, the entire SSLMate application, including third party dependencies and web server software, contains a lot of code, some of it written in memory-unsafe languages. We want to keep our customers' DNS credentials safe even if we or one of our third party software providers make a mistake.

Goals

Ideally, we'd satisfy the following goals:

An attacker who can access the SSLMate database shouldn't be able to access our customers' DNS accounts.
An attacker who can execute arbitrary code with the permissions of the SSLMate backend should only be able to add/remove DNS records related to SSL certificate issuance - not arbitrary records (such as the A record for a website).
All access to our customers' DNS accounts should be logged.

A popular approach to protecting sensitive credentials is to encrypt them at rest, possibly using a service such as AWS Key Management Service. When the backend needs to access the DNS service, it would retrieve the encrypted credential from the database, decrypt it, and use it to access the DNS provider's API.

This satisfies the first goal - an attacker who can access the database, such as via a SQL injection, would see only encrypted credentials. However, it does not satisfy goals 2 and 3. An attacker who gets arbitrary code execution in the backend, such as from a memory safety vulnerability in the web server, could decrypt DNS credentials and do whatever they want with them.

To satisfy goals 2 and 3, the SSLMate backend can't have access to the unencrypted DNS credentials. That means it can't talk directly to the DNS provider. Instead, a separate service, which we call the Customer Gateway, makes requests to the DNS provider. It accepts requests from the SSLMate backend, logs them, validates them, and if they are related to certificate issuance, forwards them to the DNS provider with the appropriate API credentials attached.

The Customer Gateway runs on separate infrastructure from the rest of SSLMate, which means an attacker who can execute arbitrary code with the permissions of the SSLMate backend can't get the unencrypted DNS credentials. They could make requests to the Customer Gateway, but the Customer Gateway would only permit access to certificate-related records and would log all requests to aid in remediation.

The Customer Gateway is a small, stateless service written in Go that exposes a JSON API over HTTP. It provides the following endpoints (click on an endpoint to show the inputs/outputs, which have been slightly simplified):

POST /make_integration

Input:

DNS provider type (e.g. "route53", "dnsimple")
DNS provider credentials

Output:

Integration handle

POST /get_zones

Input:

Integration handle

Output:

List of DNS zones

POST /add_record

Input:

Integration handle
DNS record details (FQDN, type, value)

Output:

Record handle

POST /remove_record

Input:

Integration handle
Record handle

Adding an Integration

When an SSLMate customer adds a DNS integration through the SSLMate web console, they submit their DNS provider credentials. The SSLMate backend forwards these credentials to the make_integration endpoint of the Customer Gateway. The Gateway encrypts the credentials using a symmetric key known only to the Gateway, and returns the encrypted credentials to the SSLMate backend in the form of an "integration handle." The backend stores the integration handle in the database.

(Crypto details: the handle is encrypted using NaCl's secretbox construction, which uses XSalsa20 for encryption, Poly1305 for authentication, and a random 192-bit nonce which is prepended to the handle. The key is derived from the Gateway's root key using HKDF, with a context string identifying the type and version of handle.)

The exact process for adding an integration depends on the type of authentication used by the DNS provider. The DNS providers supported by SSLMate can be divided into three categories:

Shared bearer token: When the DNS provider uses a simple bearer token to authenticate to their API, the SSLMate customer enters their token into the SSLMate website. The backend forwards the token to the Customer Gateway's make_integration endpoint, which encrypts the bearer token and returns it in the integration handle.
OAuth: When the DNS provider uses OAuth, the authorization code is submitted to the SSLMate backend from the customer's browser after they complete the OAuth flow to authorize SSLMate. The backend forwards the authorization code to the Customer Gateway via the make_integration endpoint. The Customer Gateway exchanges the authorization code for a long-term access token, which it encrypts and returns in the integration handle. The long-term access token is never exposed unencrypted outside of the Customer Gateway.
Service account authorization: With Amazon Web Services and Google Cloud, no secrets are exchanged at all. Instead, the SSLMate customer authorizes SSLMate's AWS or Google Cloud account to access their own account. Then they enter the ID of their AWS or Google Cloud account into the SSLMate website. The backend forwards the customer account ID to the make_integration endpoint, which encrypts and returns the ID in the integration handle. The credentials for SSLMate's AWS and Google Cloud accounts are only accessible by the Customer Gateway. Care is taken to avoid confused deputy attacks; for details, see the AWS docs or this post about Google Cloud.

Using an Integration

When the SSLMate backend needs to list the customer's DNS zones, add a DNS record, or remove a DNS record, it retrieves the integration handle from the database and sends it to the appropriate Customer Gateway endpoint. The Customer Gateway decrypts the integration handle to get the real credentials, makes a request to the DNS provider, and returns the result to the SSLMate backend.

Before adding a DNS record, the Customer Gateway confirms that the record is related to certificate issuance. If the record can co-exist with other records with the same name, it must be a TXT or CAA record. If the record needs to replace existing records with the same name, it must be a TXT, CAA, CNAME, or NS record and its first node must be _acme-challenge, or start with an underscore and not be one of the names in IANA's Underscored and Globally Scoped DNS Node Names List.

After adding a DNS record, the Customer Gateway puts details about the record in an encrypted record handle that it returns to the SSLMate backend. The SSLMate backend stores the record handle in the database, and when it needs to remove the record, passes the record handle to the Customer Gateway's remove_record endpoint.

The remove_record endpoint decrypts the record handle to determine what record to remove. Since these details come from a handle that was created by the add_record endpoint, the Gateway can only remove records that it added.

The add_record and remove_record endpoints log all changes made to the customer's zone. If the SSLMate backend is ever compromised, the logs can be analyzed to determine the impact of the compromise.

Conclusion

Thanks to the Customer Gateway, we can offer our customers a high assurance that their DNS credentials are protected from misuse. We think the Customer Gateway provides customers a higher assurance than placing their DNS credentials on every one of their TLS endpoints, which is required when using standalone ACME clients.

Although SSLMate's Customer Gateway is currently oriented around DNS providers, the concept is applicable to any type of integration, and we plan to leverage it in the future for installing certificates on AWS load balancers and for discovering infrastructure to monitor with Cert Spotter. It will just be a matter of adding new endpoints such as install_certificate or list_servers, which will work similarly to add_record and list_zones.

We hope that other SaaS providers will consider a similar approach to securing their customers' integrations. With more and more companies using SaaS, there is a growing need for integrations between different providers. If not done carefully, these integrations are a serious security liability and can greatly expand the impact of a security compromise. But with careful security engineering like SSLMate's Customer Gateway, the risk can be manageable.