Overview
An export writes one dataset to one of your destinations as Parquet files, on a schedule you choose. This guide covers the create flow that’s common to every dataset; each dataset page documents the columns and any dataset-specific settings.Prerequisites
- An S3 destination configured. See Set up an S3 destination.
- A space administrator role in Cometly.
Create the export
Set a path prefix (optional)
Optionally set a path prefix (for example
cometly/). Files are written under this path within
the bucket.Pick a dataset
Choose the dataset to export. Each dataset has its own columns and quirks — see the
dataset reference.
Choose the columns
Pick which columns the Parquet files carry from the dataset’s catalog. Locked columns are always
included; the rest are optional. Your selection is frozen at creation (see below).
Set the cadence and window
Choose how often the export runs and how much data each run covers:
- One-time — runs once. Pick an absolute start and end date.
- Daily — runs once a day at a scheduled hour you choose. Pick a trailing window preset (up to 90 days).
- Hourly — runs every hour. Pick a trailing window preset (up to 14 days; up to 7 days for ad-analytics datasets).
Configure dataset-specific settings
Some datasets ask for more at creation — the Attributed conversions
dataset takes an attribution model, window, type, and optional source filter; the
Contacts dataset asks how custom-field columns should be
named. Most datasets need only a column selection.
Each export run is recorded with its status and timing. Open an export to view its run history.
Exports are immutable
Once created, an export’s name is the only thing you can change. To alter the cadence, window, destination, dataset, or columns, delete the export and create a new one. This keeps every run’s output faithful to the configuration that produced it — a run’s columns and shape never drift underneath your pipeline.File layout
Files are written to your bucket under a Hive-style partitioned path:<dataset>is the dataset’s path segment —touchpoints,raw_conversions,raw_hits,attributed_conversions,contacts,companies, or<channel>_ad_analytics(for examplefacebook_ad_analytics). Each dataset page lists its segment.exported_dateis the date the export ran (not the event date) and is sortable, so your pipeline can always find the most recent export.run_idmakes every run unique, so runs never overwrite each other and retries are safe._SUCCESSis written last. Only ingest a run folder once its_SUCCESSmarker is present, so you never read a partially written run.
Loading the data
Point your ingestion tooling — Snowflake, BigQuery, Databricks Auto Loader, Athena, and others — at the…/<dataset>/space_id=<id>/ prefix to pick up new partitions automatically.
Each recurring run re-exports the entire window, not just new rows — so the most recent
exported_date is the current snapshot of that window. Deduplicate on a stable key (such as the
row’s id, or the id plus metric date for ad analytics) when loading.