Skip to main content

​
Overview

An export writes one dataset to one of your destinations, on a schedule you choose — as Parquet files for an S3 or GCS destination, or loaded directly into a table for a Snowflake or BigQuery destination. This guide covers the create flow that’s common to every dataset; each dataset page documents the columns and any dataset-specific settings.

​
Prerequisites

​
Create the export

1

Open the Exports table

In the dashboard, go to Integrations → Amazon S3, Integrations → Google Cloud Storage, Integrations → Snowflake, or Integrations → BigQuery — the Exports table lists every export regardless of destination type, and appears on all four pages. Select Create export.
2

Name the export

Give it a recognizable name, for example “Touchpoints to warehouse”.
3

Choose a destination

Select the destination to write to. The dropdown only lists destinations matching the integration page you opened it from — Amazon S3 destinations from Integrations → Amazon S3, Google Cloud Storage destinations from Integrations → Google Cloud Storage, Snowflake destinations from Integrations → Snowflake, BigQuery destinations from Integrations → BigQuery.
4

Set a path prefix or target table

For an S3 or GCS destination, optionally set a path prefix (for example cometly/) — files are written under this path within the bucket. For a Snowflake or BigQuery destination, name the target table Cometly creates and loads into: it must start with a letter or underscore, followed by any number of letters, digits, or underscores. Snowflake table names may also contain $; BigQuery table names may not.
5

Pick a dataset

Choose the dataset to export. Each dataset has its own columns and quirks — see the dataset reference.
6

Choose the columns

Pick which columns the export carries from the dataset’s catalog. Locked columns are always included; the rest are optional. Your selection is frozen at creation (see below).
7

Set the cadence and window

Choose how often the export runs and how much data each run covers:
  • One-time — runs once. Pick an absolute start and end date.
  • Daily — runs once a day at a scheduled hour you choose. Pick a trailing window preset (up to 90 days).
  • Hourly — runs every hour. Pick a trailing window preset (up to 14 days; up to 7 days for ad-analytics datasets).
See Cadence and windows for how trailing windows behave.
8

Configure dataset-specific settings

Some datasets ask for more at creation — the Attributed conversions dataset takes an attribution model, window, type, and optional source filter; the Contacts dataset asks how custom-field columns should be named. Most datasets need only a column selection.
9

Create

Select Create export. For recurring cadences the first run is scheduled; for one-time it runs right away. Cometly writes the data to your destination and records each run in the export’s run history.
Each export run is recorded with its status and timing. Open an export to view its run history.

​
Exports are immutable

Once created, an export’s name is the only thing you can change. To alter the cadence, window, destination, dataset, or columns, delete the export and create a new one. This keeps every run’s output faithful to the configuration that produced it — a run’s columns and shape never drift underneath your pipeline.

​
File layout (S3 and GCS destinations)

Files are written to your bucket under a Hive-style partitioned path — the same layout for both S3 and GCS destinations:
  • <dataset> is the dataset’s path segment — touchpoints, raw_conversions, raw_hits, attributed_conversions, contacts, companies, or <channel>_ad_analytics (for example facebook_ad_analytics). Each dataset page lists its segment.
  • exported_date is the date the export ran (not the event date) and is sortable, so your pipeline can always find the most recent export.
  • run_id makes every run unique, so runs never overwrite each other and retries are safe.
  • _SUCCESS is written last. Only ingest a run folder once its _SUCCESS marker is present, so you never read a partially written run.

​
Loading the data (S3 and GCS destinations)

Point your ingestion tooling — Snowflake, BigQuery, Databricks Auto Loader, Athena, and others — at the …/<dataset>/space_id=<id>/ prefix to pick up new partitions automatically.
Each recurring run re-exports the entire window, not just new rows — so the most recent exported_date is the current snapshot of that window. Deduplicate on a stable key (such as the row’s id, or the id plus metric date for ad analytics) when loading.
For a Snowflake or BigQuery destination, there’s no file layout to manage — Cometly loads each run directly into the target table you named at creation, appending rows (Snowflake) or running an append-only load job (BigQuery). Cometly does not de-duplicate on your behalf — the same re-export and deduplication behavior applies: each run re-loads the entire window, so rows repeat across runs and you deduplicate on a stable key on your side. See De-duplicating loaded rows for a BigQuery example.