Skip to main content

Firebolt

Availability:
Available Airbyte CloudAvailable Airbyte OSSAvailable PyAirbyte
Support Level:
Marketplace
Connector Version:
0.2.25(Last updated a day ago)
CDK Version:
0.73.0
Sync Success Rate:
Usage Rate:

This page guides you through the process of setting up the Firebolt destination connector.

Prerequisites​

This Firebolt destination connector has two replication strategies:

  1. SQL: Replicates data via SQL INSERT queries. This leverages Firebolt SDK to execute queries directly on Firebolt Engines. Not recommended for production workloads as this does not scale well.

  2. S3: Replicates data by first uploading data to an S3 bucket, creating an External Table and writing into a final Fact Table. This is the recommended loading approach. Requires an S3 bucket and credentials in addition to Firebolt credentials.

For SQL strategy:

  • Host
  • Username
  • Password
  • Database
  • Account
  • Engine

Airbyte automatically picks an approach depending on the given configuration - if S3 configuration is present, Airbyte will use the S3 strategy.

For S3 strategy:

  • Username
  • Password
  • Database
  • Account
  • S3 Bucket Name
    • See this to create an S3 bucket.
  • S3 Bucket Region
    • Create the S3 bucket on the same region as the Firebolt database.
  • Access Key Id
  • Secret Access Key
    • Corresponding key to the above key id.
  • Host (optional)
    • Firebolt backend URL. Can be left blank for most usecases.
  • Engine (optional)
    • If connecting to a non-default engine you should specify its name or url here.

Setup guide​

  1. Sign up to Firebolt following the guide
  2. Follow the getting started tutorial to setup a database.
  3. Create a service account.
  4. Create an engine as described in here
  5. (Optional) Create a staging S3 bucket (for the S3 strategy).
  6. (Optional) Create an IAM with programmatic access to read, write and delete objects from an S3 bucket.

Supported sync modes​

The Firebolt destination connector supports the following sync modes:

  • Full Refresh
  • Incremental - Append Sync

Connector-specific features & highlights​

Output schema​

Each stream will be output into its own raw Fact table in Firebolt. Each table will contain 3 columns:

  • _airbyte_ab_id: a uuid assigned by Airbyte to each event that is processed. The column type in Firebolt is VARCHAR.
  • _airbyte_emitted_at: a timestamp representing when the event was pulled from the data source. The column type in Firebolt is TIMESTAMP.
  • _airbyte_data: a json blob representing the event data. The column type in Firebolt is VARCHAR but can be be parsed with JSON functions.

Changelog​

Expand to review
VersionDatePull RequestSubject
0.2.252024-11-2548672Update dependencies
0.2.242024-10-2947780Update dependencies
0.2.232024-10-2847100Update dependencies
0.2.222024-10-1246841Update dependencies
0.2.212024-10-0546420Update dependencies
0.2.202024-09-2846144Update dependencies
0.2.192024-09-2145744Update dependencies
0.2.182024-09-1445562Update dependencies
0.2.172024-09-0745245Update dependencies
0.2.162024-08-3144991Update dependencies
0.2.152024-08-2444698Update dependencies
0.2.142024-08-2244530Update test dependencies
0.2.132024-08-1744239Update dependencies
0.2.122024-08-1043682Update dependencies
0.2.112024-08-0343143Update dependencies
0.2.102024-07-2742703Update dependencies
0.2.92024-07-2042211Update dependencies
0.2.82024-07-1341789Update dependencies
0.2.72024-07-1041602Update dependencies
0.2.62024-07-0941118Update dependencies
0.2.52024-07-0640854Update dependencies
0.2.42024-06-2740578Replaced deprecated AirbyteLogger with logging.Logger
0.2.32024-06-2540494Update dependencies
0.2.22024-06-2240078Update dependencies
0.2.12024-06-0639157[autopull] Upgrade base image to v1.2.2
0.2.02024-05-0836443Service account authentication support
0.1.12024-03-0535838Un-archive connector
0.1.02022-05-1813118New Destination: Firebolt
Was this page helpful?