Acquire Tasks (or acquisition tasks) define the inbound movement of data, i.e. data moving from a data source into your centralized, enterprise data Lakehouse.

In Empower, you can create as many acquire tasks as you want, while configuring each of them to acquire data however often you want.

📘

Quick Links

This page only covers an overview of Acquisition Tasks and how to use them

• Logging and Monitoring: defines how to use the log page to monitor tasks ("View Runs").
• Schedules and Triggers: describes how to trigger tasks to run on demand ("Preview Run") and schedule tasks to run on a repeating cadence ("Schedule").

Overview

The Acquire module is accessible from the left navigation menu.

From here, you can view all Acquisition Tasks within your selected environment.

There are a few restrictions when it comes to acquisition tasks.

1. An Acquisition Task can only ever be affiliated with one Data Source Connection at a time.
   1. This means you can't bring data in from multiple data sources under the same task! In case you have this need, create a separate acquisition task for each of your data sources, and schedule them to run at the same time.
2. Archiving a Data Source Connection will also force the archiving of all affiliated Acquisition Task.
   1. You will not be able to edit, schedule, or trigger these tasks until you restore the underlying Data Source Connection.
3. Deleting a Data Source Connection will permanently archive all affiliated Acquisition Tasks.
   1. You will never again be able to edit, schedule, or trigger these tasks.

Tasks

From the default view for the Acquire module, you can see all the Acquisition Tasks within your current environment.

You can search these tasks by name, view configurations or historical run logs, trigger a task to run on demand, view/set/activate scheduling, and create new tasks.

Creation

📘

Fivetran Connections - Task Creation

Note that when you create a new Connection with Fivetran as the connection provider, an Acquire Task is automatically created for you. Each Fivetran Connection can only have one Acquire Task associated with it. Learn more about how Fivetran provided Connections work here.

Creating an Acquisition Task can be done by simply clicking on "+ New Task" at the top of the screen on the homepage for the Acquire module.

Once you fill out the required fields (Name and Data Source), click "Create" to complete the creation process.

You will now see your newly created task at the top of the page.

📘

Advanced Features

For a peak at the advanced features you may configure to optimize Acquisition, take a look at our guide on Batch Extraction.

📘

INFO: 1 to Many - Data Source Connections and Acquisition Tasks

Connections and Acquisition Tasks have a 1:many relationship.

This means that a single Acquisition Task can only ever be associated with one Data Source Connection at any moment in time. However, a specific Data Source Connection may be associated with many different Acquisition Tasks, all with different schedules and inclusion configurations.

Editing

Clicking on the name of a Task will bring you to the Task tab view as shown below.

From this view, you can click on the edit pencil icon to update the name, connection, or advanced options.

🚧

WARNING: changing the Data Source Connection field will affect your configuration!

Changing a Acquisition's data source will completely wipe any previous object inclusion configurations. Be mindful of this action when you modify a Acquisition Task!

Deletion

You can delete any existing Acquisition Task by clicking the Delete button from the Task overview tab. Doing so will bring up a menu with one option being to Delete the task.

A confirmation modal will pop up. You must confirm you wish to delete the task in order to complete the deletion process.

❗️

DANGER: Deletion is Permanent

Task deletion is a permanent action. Deleting a task will also remove the entire historical log of that task. You will not be able to reverse a task's deletion, so make sure you actually want to perform this action!

Scheduling and Triggering Tasks

To read about how to schedule and trigger Acquisition Tasks or any other task type, visit Scheduling Data Tasks.

Configuration

Clicking on the name of a Task and then select the "Configuration" tab on any existing Acquisition Task to visit its configuration page.

The configuration page contains a list of all available objects from the task's associated Data Source. You may toggle Include to configure this object to be included as part of data acquisition when this task is triggered.

You may search objects by name, as well as filter objects by Include, Estimated Rows and Schema Name.

When you include/exclude an object on this page, the change is automatically saved to the task's configuration.

🚧

WARNING: Global Enablement

The Enable toggle on the object within the Data Source's Metadata Catalog will impact the ability of this task to acquire the object.

If the object is globally disabled it will not be acquired when this task runs.

1. You will not be able to include globally disabled objects it in the task.
2. Even if the disabled object is already included in the task, it will not be acquired when the task runs.

How Data Acquisition Works

Data Acquisition is an umbrella term that links together three data movement concepts.

Metadata Extraction

Acquisition automatically refreshes the associated data source's metadata to ensure Schema Drift protection. Read up about Metadata Extraction and how Metadata Catalogs work.

Extraction

Extraction defines the process of collecting data from a data source and writing it to the open-source parquet file format in the Empower Landing Zone.

📘

Supported Data Sources in Empower

Empower is configured to support over 200+ data sources for extraction. Some of these are entirely self-serviceable from the UI. Others require more configuration by the Empower delivery team.

For a full list of sources, check out our Source System Support page.. Talk to your delivery manager for more information.

Ingestion

Data ingestion takes the data from the Landing Zone and ingests it into the open-sourced Delta Table format into the Bronze Layer of the Medallion Architecture.

All Empower-ingested data is tracked as part of a Type 2 data contract. This means that all changes made to the data from the original data source, including modifications and deletions, are stored as new data points in the Bronze Layer.

Bronze Layer ingested data is further ingested into the Silver Layer as the most current version of every data point.