Custom Flows

BETA The Custom Flows feature is currently in BETA mode.

Custom Flows offer the ability to execute a single notebook within your deployment. This feature allows you to run any logic as needed within that notebook providing added flexibility beyond the specialized flow types already available (Data Acquisition, Analytics Engineering, and Publishing flows).

Recommendations

While Custom Flows offer great flexibility, we strongly recommend using the specialized flow types (Data Acquisition, Analytics Engineering, and Publishing) whenever possible. These flow types come with additional functionalities, such as advanced logging capabilities, that Custom Flows do not provide.

However, if you need a straightforward way to run a notebook, Custom Flows are the ideal solution.

How to Use Custom Flows

NOTE: custom flows are currently feature flagged in the UI by default. Talk to your delivery team today for how to turn this on for your deployment.

Once custom flows have been enabled in your environment, you can create a new custom flow by following these steps:

1. Add: Click on the blue plus sign button to start a new flow.
2. Configure: Choose a unique name for your custom flow and specify the filepath to the notebook located in your lakehouse then hit Create.
3. Run: Once added, you can now run the flow manually or define a schedule for it to run on automatically. The specified notebook will be executed each time the custom flow runs.