Materialize setup
Certain core functionality may vary. If you would like to report a bug, request a feature, or contribute, you can check out the linked repository and open an issue.
Overview of dbt-materialize
- Maintained by: Materialize Inc.
- Authors: Materialize team
- GitHub repo: MaterializeInc/materialize/blob/main/misc/dbt-materialize
- PyPI package:
dbt-materialize
- Slack channel: #db-materialize
- Supported dbt Core version: v0.18.1 and newer
- dbt Cloud support: Not Supported
- Minimum data platform version: v0.28.0
Installing dbt-materialize
pip is the easiest way to install the adapter:
pip install dbt-materialize
Installing dbt-materialize
will also install dbt-core
and any other dependencies.
Configuring dbt-materialize
For Materialize-specifc configuration, please refer to Materialize Configuration.
For further info, refer to the GitHub repository: MaterializeInc/materialize/blob/main/misc/dbt-materialize
Connecting to Materialize
Once you have set up a Materialize account, adapt your profiles.yml
to connect to your instance using the following reference profile configuration:
materialize:
target: dev
outputs:
dev:
type: materialize
host: [host]
port: [port]
user: [user@domain.com]
pass: [password]
dbname: [database]
cluster: [cluster] # default 'default'
schema: [dbt schema]
sslmode: require
keepalives_idle: 0 # default: 0, indicating the system default
connect_timeout: 10 # default: 10 seconds
retries: 1 # default: 1, retry on error/timeout when opening connections
Configurations
cluster
: The default cluster is used to maintain materialized views or indexes. A default
cluster is pre-installed in every environment, but we recommend creating dedicated clusters to isolate the workloads in your dbt project (for example, staging
and data_mart
).
keepalives_idle
: The number of seconds before sending a ping to keep the Materialize connection active. If you are encountering SSL SYSCALL error: EOF detected
, you may want to lower the keepalives_idle value to prevent the database from closing its connection.
To test the connection to Materialize, run:
dbt debug
If the output reads "All checks passed!", you’re good to go! Check the dbt and Materialize guide to learn more and get started.
Supported Features
Materializations
Because Materialize is optimized for transformations on streaming data and the core of dbt is built around batch, the dbt-materialize
adapter implements a few custom materialization types:
Type | Supported? | Details |
---|---|---|
source | YES | Creates a source. |
view | YES | Creates a view. |
materializedview | YES | Creates a materialized view. |
table | YES | Creates a materialized view. (Actual table support pending #5266) |
sink | YES | Creates a sink. |
ephemeral | YES | Executes queries using CTEs. |
incremental | NO | Use the materializedview materialization instead. Materialized views will always return up-to-date results without manual or configured refreshes. For more information, check out Materialize documentation. |
Indexes
Materialized views (materializedview
), views (view
) and sources (source
) may have a list of indexes
defined.
Seeds
Running dbt seed
will create a static materialized view from a CSV file. You will not be able to add to or update this view after it has been created.
Tests
Running dbt test
with the optional --store-failures
flag or store_failures
config will create a materialized view for each configured test that can keep track of failures over time.