Weekly live AEP workshops every Friday - reserve your seat →
Student question · Ingestion

AEP ingestion methods: batch vs streaming

Which AEP ingestion methods are batch and which are streaming: source connections, workflows and data collection datastreams explained.

Quick answer

AEP has three common ingestion routes. Source connections can be batch or streaming depending on the source - cloud storage is batch, Kafka-style streams are streaming. Workflows are batch because they ingest files such as CSV or Parquet. Data collection through datastreams is streaming because Web and Mobile SDKs send events in real time.

Summary

RouteModeTypical use
Source connectionsBatch or streamingCloud storage, CRM, databases, streaming platforms
WorkflowsBatchCSV and Parquet files mapped to XDM
Datastreams (data collection)StreamingWeb SDK, Mobile SDK, Edge Network

File formats that trip people up

Drag-and-drop upload to a dataset expects JSON or Parquet. CSV files fail there; use a workflow to map CSV to XDM.

Transform before you load

Clean, standardise and model data before it reaches AEP. When transformation happens upstream, Profile stays fast, identities stay clean and activation is simpler.

Frequently asked questions

Is Data Landing Zone batch or streaming?

Batch. Files land in cloud storage and are picked up by a batch source connection.

Why did my CSV upload fail?

Direct dataset upload expects JSON or Parquet; ingest CSV through a mapping workflow.

Last updated:

Ready to move your Adobe MarTech program forward?

Consulting for enterprise teams, training for professionals. Tell us where you are today.

WhatsApp