I created my dbt Slack account in January 2017 — at the time, this meant joining a community of just 58 other data practitioners. The dbt community of 2017 was genuinely special to me — community members generously gave their time to help me grow in my role and become more impactful, so much so that I decided to move to the US in 2018 to lead the dbt community.
In 2017, everything in the dbt community happened in Slack — and that made sense! It allowed people to meet each other, form connections, and figure out the best way to use dbt together!
Today, our Slack community now has over 6000 members, and continues to grow — as Tristan wrote in his four years of Fishtown Analytics post, we’ve now gone mainstream. The dbt community is still genuinely special — I love seeing people join and share how excited they are to have discovered dbt, or be grateful for the help someone gives them.
That being said, in 2020, having everything happen in Slack doesn’t make sense anymore — the conversation can feel unfocused as we try to cover everything from “help, I’m stuck installing dbt!”, to “what books should I read in 2020?”. Further, answering questions in Slack doesn’t scale — the message history gets lost, and there’s little hope of getting help unless you have an account.
So, we’re taking some steps to bring back the ✨magic ✨ in the dbt community. First stop — focusing the different ways the dbt community interacts with each other. In particular, we're being more conscious of the breakdown between synchronous and asynchronous information sharing, and the specific mediums where these different types of information live. To do this, we’re:
- Investing deeply in documentation
- Redefining the most appropriate mediums for sharing information
- Creating social norms that promote positive Slack etiquette
Let’s dive in to what each of these look like!
1. Investing deeply in documentation
Some people’s quarantine project has been making sourdough. My quarantine project has been restructuring docs.getdbt.com.
Truth-be-told, it had been a while since we’d really given docs.getdbt.com some love. For each feature, we had a habit of using only one article for all the information we documented. Take sources as an example — whether you were a newer dbt user trying to figure out what sources are, or an existing user who just wanted to look up how far to indent the
freshness field, there was one single article on “Using Sources” to find the answer.
This approach meant that adding more information to each article (e.g. documenting advanced use cases and examples) became a balancing act — either we’d add the information at the risk of overwhelming a new user, or (more commonly) we’d just leave it out.
As a result, community members would ask questions in Slack that should have been served by documentation:
This meant more effort for both the person asking the question and the person answering. Further, if anyone else wants to know this in the future (likely!) they would have to ask in Slack too.
So, we undertook a restructure. In doing so, we considered:
- How easy is it for a new user to understand a concept without getting bogged down in the details?
- How easy is it for power users to find the information they are looking for?
- How easy is it to extend these docs with additional examples?
Now, when navigating the docs, you’ll find three separate types of articles:
- Docs: Guides that introduce each concept (e.g. the guide for Sources) and show how it can be used. Most of these guides were re-written as part of this project.
- Reference: Exact usage syntax once you understand the concept (e.g. the docs on Source properties). A lot of these configurations were previously undocumented. Each page also includes examples which we’ll continue to add to over time.
- FAQs: Commonly asked questions, that are embedded in the guides to help you get to the right answers quickly. Bonus: when embedded, these FAQs expand — preserving that sweet sweet skimmability.
Documentation is never finished, but from here on, most of the remaining work can be reasonably scoped into smaller pieces, and the structure is in a place where we can add more examples to docs as they arise. For example, that above question about adding images to your project documentation? Now it’s part of docs.getdbt.com for everyone to find.
Next time you go to ask (or answer!) a question in Slack, check if the answer already exists in docs.getdbt.com! You might find yourself pleasantly surprised. Or you might find yourself making a Pull Request to include it!
2. Redefining the most appropriate mediums for sharing information
Restructuring docs.getdbt.com was just the first step — now that they’re in much better shape, we can start to leverage other mediums to answer questions. Whilst Slack is wonderful, it’s not searchable, and we’re blowing through our 10k message limit in a matter of weeks!
Going forward, here’s how we’re thinking about the different mediums we use to communicate:
|docs.getdbt.com: Docs||Understanding oriented
Introduce the core concepts of dbt, with examples.
|docs.getdbt.com: Reference||Information oriented
The technical reference for dbt configurations. These docs assume that you have a basic understanding of key concepts.
|docs.getdbt.com: Tutorial||Learning oriented
Provide an way for a newcomer to get started
|docs.getdbt.com: FAQs||Problem oriented
Easily indexed common questions that link back to relevant guides and reference docs.
|Stack Overflow||Problem oriented
Specifically, “I’m stuck and don’t know what to do”
How analytics engineers use dbt to solve their tactical problems, e.g.:
- Version controlling UDFs
- Writing a custom schema test for not null
- Snowflake shares + dbt
- Permission schemes in a data warehouse
|Blog posts||Strategy oriented
Bigger picture approaches, not just for dbt practitioners
Create connections with other analytics engineers. Discuss ideas that require opinions, or push the boundaries of what has been done before.
(This thinking was strongly informed by this article — thank you to Taylor “320” Murphy for sharing it with me.)
There’s some new things to note here:
- Stack Overflow will become the place to ask support-style questions. If someone is stuck, chances are someone else will get stuck there too. Keeping this information in Slack doesn’t scale, and it doesn’t add a ton of value to the conversation. Plus, Stack Overflow has much better mechanisms (upvotes, resolved buttons) for marking things as solved.
- Discourse (which currently is in less-than-perfect shape) will become a place for people to share their tactics. Think articles like “How to use Snowflake shares with dbt”, or “Publishing dbt docs to Netlify”. Usually these are writeups where there is no one perfect answer (unlike the “I’m stuck” questions on Stack Overflow), instead, you might need to dig into the “why” or discuss tradeoffs of your approach in these articles.
While adding more channels (more prominently) into the mix feels like it could be overwhelming, I think this is an important step to help us maintain long-term health of the community.
3. Creating social norms that promote positive Slack etiquette
To facilitate these changes, I’m asking you, as community members, to help out with a few things.
1. Promoting Stack Overflow for support
We’ve created a #stack-overflow-feed channel in dbt Slack that subscribes to the RSS feed for dbt questions. If you’re interested in helping out, please join the channel!
Further, if someone asks a question that is more suited to Stack Overflow (rule of thumb: the question can be summarized as “I’m seeing this error message and don’t understand it”), then nudge them to SO instead. You can use the words
stack overflow bot to trigger this Slackbot response.
Hi there! This question would be useful for others to learn from. Since our Slack history is short-lived, and not indexed by a search engine, we’d prefer if you posted this on Stack Overflow instead. Make sure you tag the question with
dbtso we pick it up, and/or add the link to this thread when it’s posted.
2. Promoting Discourse for interesting write-ups
Lots of valuable discussion happens in Slack, and then we lose it to the Slack ether (and unfortunately, the economics of paying for Slack for 6k people don’t really work out). If you read an answer you think would be interesting for others to learn from, nudge the person to write this up on Discourse
3. Using semantic emojis
It’s going to take a fair bit to create this cultural change, but we’re going to give it our best shot. Please use emoji-reactions to help people sort through threads more quickly! The most important one for now is using the ✅ to mark questions as resolved. If you think a question is resolved, use the words
resolved bot to get this response:
Hi there! dbt Slack is pretty big these days, and messages go by quickly. Since this message looks like it’s resolved, please add a ✅ reaction to your original message so that others know you’re no longer in need of help.
Thanks to Stephen Whitworth for first making me aware of semantic emojis (you can read more about them here).
There’s more to come
This post is just the few steps on part of a bigger journey to making the dbt community as magical as possible — expect to hear more on this front. And if you’re interested in having a say on the future of the dbt community, let me know in the #community-strategy channel in Slack!