Empower Data Platform v1.40
Key Changes and Improvements
We have A LOT of great updates coming out in v1.40:
- Self-Service Error Reset: You can now reset acquisition tasks on your own from the Dashboard module.
- Email Notifications (BETA): Beta rollout of email notifications for key events and workflows.
- Fivetran Connections GA: Fivetran Connections are now generally available. 100+ new connections are now instantly available for all in the Connections module!
- Task Page Redesigns - Task pages have been redesigned from card-based to table-based layouts.(excluding Extend, coming soon!)
- Model Entity Navigation Update: Modeling has been moved out of the Transform section into its own top-level menu item for better discoverability.
- Transform Task Autocomplete: Model fields now autocomplete when configuring transform tasks for ease of use.
- Improved Task Visibility on Connect Pages: Associated tasks are now shown in an accordion view for better clarity and navigation.
Self-Service Error Reset
You can now reset acquisitions on your own, reducing dependency on support. At the top of the Dashboard module, you can now see notifications for unaddressed errors during data acquisition. You can click into this section to expand a table of all errors in your environment.
You can reset individual objects or bulk-reset multiple objects simultaneously, clearing their error states and enabling future runs with the Reset Error State button.
Learn more on the Object Error Reset page!
Email Notifications (BETA)
The Email Notifications module enables users to stay informed about critical issues in their data workflows—even when they’re not logged into the platform. This feature is currently released in BETA, which means we do not recommend it yet for production-critical workflows.
This BETA feature will automatically be available to any deployment v1.40+. Read more about how to use it on the Email Notifications page.
Fivetran Connections GA
Fivetran integrations are now generally available! We are making over 160+ Fivetran connectors available for you to try in the v1.40 release, including:
- Azure Devops
- Google (Cloud Functions, Analytics 4 Export, Business Profile, Campaign Manager 360, etc)
- Snowflake
- Qualtrics
- Github
- and 160+ more!
Check out our docs about Fivetran to learn more!
Task Page Redesigns
We’ve reimagined the Task Page experience to make it faster, clearer, and more scalable.
Instead of the previous card-based layout, tasks are now displayed in a streamlined table view. The new table view will make it easier to scan, sort, and manage large volumes of tasks at a glance. Acquire, Transform, and Publish tasks will reflect this new design change!
Why the change? The card layout was visually rich, but it did not scale all that well for power users managing dozens (or even hundreds) of tasks.
What's new?
- Sort and filter tasks by key attributes like name, connection type, schedule, and last run info.
- Compact layout provides better visibility and more info on the page.
- Improved UI performance on large sets of tasks.
📆 Note: Extend redesign coming in v1.41!
Model Entity Navigation Update
Models, Entity, Entity Column, and Entity Dependency have been moved out of the Transform section and now live under their own dedicated top-level menu item in the Models module.
This change reflects how modeling has evolved into a core part of the platform, deserving its own space for easier access and better organization. Whether you're defining models, managing relationships, or reviewing schema changes, everything is now just one click away, grouped into the model it belongs to.
Transform Task Model Autocomplete
We’ve added autocomplete support for model fields when configuring Transform tasks to make it faster and easier to associate models their tasks.
This is a small but helpful improvement that helps make Transform tasks more intuitive, especially for newer users or those working across multiple models in their environment.
Improved Associated Task Visibility with Connections
We’ve made it significantly easier to understand what’s happening behind the scenes when working with connections in the Connect module. With this update, all associated tasks—those that power acquisition from and publishing to your data sources—are now clearly displayed in an expandable accordion view.
This enhancement gives you a more transparent and intuitive way to:
- See which tasks are tied to a connection.
- Understand the flow of data from source to the lakehouse.
- Quickly identify and troubleshoot issues without having to constantly jumping between modules.
Itemized Changelog
Enhancements and Features
- Enable Notifications for 1.40 release
- Enhance Logging for Extend tasks
- Implement MAR Service for utilization tracking
- Deploy Data Factory Pipelines for Fivetran acquisition
- Create new version of OnPremise Oracle Connector
- Remove databricks_account_id, databricks provider from external-organization in 1.40
- Enhance UC Terraform application to support existing metastore
- Add support for enable_databricks_private_endpoint for enabling databricks private endpoint
- Add support for enable_storage_account_sftp for Datalake storage account
- Add new task/model side panel
- Unfeature flag entity dependency table toggle
- Update breadcrumb to scroll to location on grid
- Unfeature flag new task views
- Create edit form for tasks and model pages
- Create Overview page and tabs for all Task types (minus Extend tasks)
- Add configuration tab for all task types
- Add Scheduling tab for each task type
- Add runs tabs for task type
- Add column filters to each of the task main page grids
- Add Task Display in Connector pages
- Add scheduling for Fivetran tasks
- Update Overview and Config pages for Fivetran tasks
- Integrate API and unfeature flag notifications
- Integrate API affiliated task endpoint to failure modal
- Clean up loading of Publish and Acquire pages
- Update size of model icon in sidenav
- Update overview page action behavior
- Unfeature flag object failure
- Add Fivetran docs route
- Include Fivetran Tasks in Dashboard Analytics
- Update UI endpoints to match change in notification endpoint
- Data Migration : Validate data migration between the environment with different versions
- CoreAPI Integration to notification service for email alerts
- Create Acquire Task in API automatically when Fivetran Connector is made
- Update GET/POST endpoint recipients for notification service
- Enhance Logging for flows_flowrun
- New Fivetran task type creation experience for acquisition tasks
- Put required proxy endpoints into the fixture
- Allow filtering tasks by multiple task types
Bugfixes
- Fixed 400 error in custom flows for Oracle and MariaDB
- Properly configured UTF-8 values in code
- Resolved Quickbase CData Connector failures
- Fixed JSON parse errors in ADF custom flow history
- Fixed JSON parse errors in ADF data acquisition history
- Fixed PostgresSQL connection template
- Fixed DB2 batch extraction failures
- Improved logging for ADF custom flows when Databricks jobs don't start
- Added error messages for failed Logic App custom flows
- Attached Empower Policy to interactive cluster
- Fixed Databricks extraction retries in Acquire
- Resolved extraction failure for Workday RAAS
- Fixed infinite "in progress" state in Acquire when task status is Success
- Set proper value in
url
column for failed flow runs - Corrected DAQ failed object count in notification service
- Enabled separate notification toggles for the same user
- Fixed inconsistent subscription behavior across workflow levels
- Displayed correct table when toggling entity dependency table
- Standardized time format in notification emails
- Ensured back button is visible when toggling entity dependency table
- Refreshed affiliated tasks when switching between models
- Updated "New Record" button when switching to Entity Column tab
- Resolved image rendering issues for source types
- Aligned grids in Preview Run and Schedule modals
- Fixed incorrect back navigation from run history
- Corrected schema names in Fivetran filter dropdown
- Made columns visible without scroll in run history
- Updated archived chip display in Acquire grid
- Renamed "View Affiliated Flows" to "View Affiliated Tasks"
- Displayed tooltip for disabled Reset Error State button
- Added "Cleared for Retry" to Error State filter
- Disabled Run Error State button when no objects selected
- Displayed toast when editing metadata catalog options
- Prevented toggle jumping in bulk edit banner
- Highlighted watermark column when using dedupe merge strategies
- Bolded text in metadata catalog bulk edit banner
- Fixed repeated sentence in confirmation modal
- Removed white banner when opening new task
- Enabled Archive/Restore on Tasks Overview
- Redirected after deleting task from Overview
- Added tooltip for Metadata Sync button
- Displayed breadcrumbs in Publish/PowerBI Overview
- Corrected tooltips for archived/deleted tasks or connections
- Enabled Save for non-Spark workers in Acquire without advanced options
- Fixed styling of chips in Add Team Members modal
- Fixed comment styling in Data Migration
- Disabled cell/row editing in archived Publish task
- Highlighted correct row in Extend history when URL has runId
- Added left border in Preview grid of Schedule page
- Fixed breadcrumb routing from Acquire schedule to correct task type
- Enabled Schedule button for "Repeat on NONE"
- Fixed validation for start/end dates in Schedule tab
- Prevented past date selection for Start Date
- Removed extra columns in Dynamic CE Publish config
- Fixed new schedule saving issue in Schedule tab
- Standardized time zone display in Flow Scheduling
- Removed already-added users from Add New Email search
- Enabled Run and Schedule buttons in new views
- Allowed deleting duplicate email entries
- Displayed underscore in disabled Fivetran schedule column
- Disabled Schedule button without required fields
- Made Transform config grid read-only
- Prevented object selection in archived tasks
- Sorted Est Rows correctly
- Fixed incorrect labels in model edit form
- Opened correct docs from Model Overview
- Showed correct tooltip for Include toggle on archived tasks
- Reset filters in Acquire config
- Displayed filter data in Publish config
- Aligned Fivetran Acquire run history
- Displayed correct icons on Publish main page
- Enabled Fivetran scheduling from Overview page
- Fixed 404 when viewing field details
- Made last row visible in Acquire config tab
- Resolved small UI issues on Task Overview
- Made Schedule toggle clickable
- Fixed routing for workflow link in Dashboard modal
- Prevented creating/updating tasks with archived parent connections
- Moved Archived chip to correct location in Runs tab
- Corrected time format in Runs tab
- Displayed correct task name in Create Task panel
- Updated Last Modified when changing name/schedule/archive
- Fixed layout in PowerBI Overview
- Removed Fivetran connections from new/edit forms
- Fixed polling in Run tab
- Removed debug log of typed values in New Task form
- Stopped infinite skeleton loader in Connect
- Pre-filled connection field when navigating from "Create New Task"
- Loaded task grid properly from "Create New Task"
- Fixed issues with task scheduling form
- Re-enabled Re-Authorization for Fivetran connections
- Enabled/disabled scheduling in Fivetran Schedule tab
- Reflected Schedule toggle changes for Fivetran tasks
- Fixed 404 on Transform History page navigation
- Enabled Publish Configuration button
- Enabled model enable/disable toggle
- Added search suggestions in Publish config new/edit forms
- Fixed 502 error during merge promotion
- Fixed 400 error on Fivetran source creation
- Prevented 500 error when fetching recipients by level
- Automatically archived task on Fivetran connector deletion