👍

Key Changes and Improvements

• Publishing Flow Monitoring and Logging: Data publishing flows now provide robust logging with entity-by-entity run status granularity.
• Data Flow Run Details Filtering & Search: Advanced filtering capabilities added to the object/entity details table in data flow run details views.
• Metadata Catalog Merge Strategies: Users can now choose a merge strategy from the metadata catalog on a per-object level for tailored data ingestion.
• Flow Error Upgrades: Two large improvements: stack trace URL links to debug Azure services and providing error messages at a global flow level for easier debugging and triage.
• Databricks - Self-Service Connector: You can now setup Databricks Unity Catalog connections from the Empower UI.

📘

Publishing Flow Monitoring and Logging

Empower now offers comprehensive logging capabilities for your data publishing flows!

This enhancement provides detailed, entity-by-entity run status granularity, allowing you to monitor each aspect of your publishing flows with greater precision. Check out the Empower Docs to explore how to effectively monitor and log your data publishing processes.

Start leveraging these advanced capabilities by accessing the detailed run status of your publishing flows directly within your Empower instance.

📘

Data Flow Run Details Filtering & Search

We've added advanced filtering capabilities to the object details table in data flow run views. This allows you to more efficiently monitor and locate specific information within each column.

📘

Metadata Catalog Merge Strategies

Empower now enables users to choose a merge strategy from the data source's metadata catalog on a per-object level. This feature allows tailored data ingestion strategies for each source object, addressing specific data management scenarios and needs more effectively.

Read more on the merge strategy options in the Empower Docs.

📘

Flow Error Upgrades

We have introduced several error message upgrades in this release!

• Error message details can now contain stack trace links in their message body. You can use this link to quickly navigate to the Azure service where the error occurred (such as an Azure Data Factory pipeline or an Azure Databricks notebook).

• Data flows now contain their own error messages to better identify issues not tied to specific objects. When a data flow run encounters an flow-level error, a general message now appears above the stoplight bars in the flow run details with the option to view the full error message.

📘

Databricks - Self Service Connector

Databricks is now available as a self-service connector. Setup connections to a Databricks Unity Catalog using the Empower UI.

For more details, read our documentation on this new data source type.

Itemized Changelog

Enhancements and New Features

• Updated flow history page to include Target and Schedule for Publish Flows.
• Created a pop-out blade for detailed flow history in Publish Flows.
• Introduced stoplight progress bars for Object Publish logs.
• Updated pop-out blade to contain flow run information for Publish Flows.
• Developed grid view for Object Publish logs on Flow Runs.
• Implemented Object Publish fail modal.
• Added tooltip hover states on Publish Logs.
• Tied custom flows to advanced options toggle.
• Displayed External Id instead of Id for flow runs.
• Connected Refresh Button on Publish Logs.
• Connected details blade to API in Publish Flows.
• Connected stoplight progress bar to API.
• Connected table and tooltips to API.
• Enabled merge strategies in the Metadata Catalog UI.
• Updated merge strategy setting to utilize isIncremental.
• Updated load strategy visibility to advanced options only.
• Updated run modal to display merge strategy in DAQ Flows.
• Implemented error trace for high-level flow history failures in DAQ Flows.
• Provided error modal for DAQ flows if error is provided.
• Developed data flow-related workflows modal.
• Implemented schema search/sort filters on DAQ flow details blade.
• Implemented name search/sort filters on DAQ flow details blade.
• Implemented extraction sort filters on DAQ flow details blade.
• Implemented ingestion search/sort filters on DAQ flow details blade.
• Assumed load strategy from Merge Strategy against old API without IsIncremental.
• Hid merge strategies by advanced options in Metadata Catalog.
• Created new Databricks self-service connector and UI capability.

Bug Fixes

• Resolved issue where running or scheduling a Data Publishing flow shows disabled entities in the run plan modal.
• Fixed issue where "Objects" text is displayed near entity count at the pop-out blade for the Flow Log History in AE Flow.
• Corrected behavior where Flows unhighlight during polling.
• Fixed display issue for the longest possible Source Notebook path in Custom Flow.
• Corrected default schedule issue when clearing schedule in Workflow/Flow.
• Removed unnecessary text in the Schedule modal in Workflow/Flows.
• Fixed issue where start time is not populated for entities returned from GET: /details endpoint in Analytics Engineering.
• Resolved issue where breadcrumbs disappear during navigation in UI.
• Fixed required fields validation issue when selecting "Full Dedupe Merge" for Merge Strategy column in Metadata Catalog.
• Corrected issue where Load Strategy value is not saving after selection in Metadata Catalog.
• Fixed issue where Clear Filters button is displayed enabled when nothing is selected at the filter dropdown in DAQ.
• Resolved issue where selected filtering remains when switching between flow runs in DAQ.
• Corrected issue where not all filter options are displayed at the Filter dropdown in DAQ.
• Fixed issue where search component is present at the filter dropdown for the Extraction column in DAQ Flows.
• Corrected issue where sorting is not working for the Extraction column at the DAQ Flow Logs pop-out blade.
• Fixed issue where "Merge Strategy" column is not displayed by default at the Run/Schedule modal in DAQ Flows.
• Fixed Databricks METADATA extraction issue.
• Resolved issue where Start/End Time is cleared out after closing Extraction/Ingestion filter dropdown at the DAQ Flow Run Logs pop-out blade.
• Fixed issue where Publishing Error modal is opened when clicking on Succeeded Records at Flow Run Logs pop-out in Data Publishing.
• Corrected required Fields validation issues when selecting Merge Strategy column in Metadata Catalog.