Sinter (now dbt Cloud) is the best way to deploy dbt in production at most organizations. It features rich scheduling functionality, accessible run logs and error alerting, and customizable job definitions. With today’s release of Sinter, we’re making it even better, with fine-grained GitHub permissions, build on Pull Request / Continuous Integration, and a rich set of API endpoints to orchestrate your deployment of dbt.
Fishtown Analytics strongly believes that analytics is a subfield of software engineering, so we feel an obligation to provide analysts with the same kinds of tools software engineers use every day. A key part of software engineering is deployment. For dbt users, deployment means making dbt models available for use by the rest of the organization.
To this end, our new release of Sinter makes it really easy to deploy dbt projects directly from GitHub. If your project is hosted on GitHub, then you can install Sinter as a GitHub App with just a couple of clicks. During the GitHub authentication flow, you’ll be prompted to give Sinter access to your specific dbt project repository, and in moments, you’ll have a fully functional production deployment of dbt! This deep integration with GitHub means that analysts can readily deploy their own code in production without needing to wrestle with tools like EC2 instances, virtual environments, or cron.
In practice, the GitHub authorization flow looks something like this:
Once installed, Sinter will be able to clone your repository automatically at the beginning of every run, ensuring that your production models always use the latest code available. This can be a scary proposition though — what if the new code is buggy, or breaks an existing contract with your users?
dbt’s schema tests and custom data tests can go a long way to providing confidence that your models keep functioning as expected while you’re developing new code. Crucially though, you need to actively run `dbt test` to determine if you’ve broken an existing contract. While everyone should of course always test their code, Sinter can help out as a last line of defense against bugs.
With this new release, Sinter’s “build on PR” functionality is out of beta and generally available to users on a paid plan. If enabled, Sinter will run and test your code in a sandbox schema (e.g.
sinter_pr_12345), reporting on any errors directly in the GitHub Pull Request. With this additional check, you can either revisit areas of the code that need more work, or deploy with confidence knowing that your code is in good shape.
Finally, we know that many organizations deploy complex data stacks to power their data-driven applications or ML pipelines. To support these users, we’ve just released a Sinter Operator for Airflow, which we’ve been piloting with several clients for the last few months. If you’re interested in using the Sinter API to trigger jobs from Airflow (or another framework), drop me (connormcarthur) a line in Slack!
Sinter is the best way to deploy dbt into a production environment. As we continue development of Sinter, keep an eye out for tighter integrations with dbt and GitHub, as well as greater control over the administration of your project in Sinter. We’re going to continue to invest heavily in Sinter in the coming months, so you can expect to see many more of these release notes in the future. If you’re interested in giving Sinter a spin, you can create a free account.