Empower Data Platform v1.38

👍

Key Changes & Improvements

  • Quick Help - In-App Documentation: Instantly access page-specific documentation guides directly within Empower via a new help button.
  • Change Tracking for SQL Server and Azure SQL DB: The ability to choose change tracking as the merge strategy from the metadata catalog for SQL Server and Azure SQL DB connections.
  • Group Role Assignment: RBAC functionality has been expanded by enabling the assignment of roles to entire user groups, simplifying role management and accelerating user onboarding.
  • View Trace Link for Errors: Easily debug the root cause of errors with a new "View Trace" link included directly within error message pop-up, providing quick access to detailed trace information.
  • DB2 Self-Service Connector: You can now setup DB2 connections from the Empower UI.

📘

Quick Help Button: In-App Documentation

Empower now features a "Quick Help" button, accessible via the question mark icon in the header beside your name.

When clicked, this button opens a side panel directly in the UI displaying documentation about the page you are viewing, ensuring you have the guidance you need no matter where you are.

Learn more about Quick Help in the Docs here.

📘

Change Tracking: SQL Server & Azure SQL DB

We’ve added support for native Change Tracking in SQL Server and Azure SQL Database source types. This new merge strategy is available to all users—no need to enable Advanced Options.

When selected, the Change Tracking strategy uses SQL’s built-in capabilities to ingest only records that have changed since the last load. Empower will automatically configure the object for incremental loading, and you’ll see the watermark type set to “Change Tracking”—this setting will be locked to ensure correct behavior.

Learn more about Change Tracking in the Docs here.

📘

Group Role Assignment

We’ve expanded RBAC capabilities—you can now assign roles to whole user groups, making it easier to manage permissions and onboard users faster.

Navigate to Settings Access Control. Click Add Groups to open the search modal. You can now search for and add entire user groups—just start typing a name to get started.

Learn more about RBAC and access control in the Docs here.

📘

View Trace Link for Errors

Troubleshooting just got easier. Error messages now include a “View Trace” link that takes you directly to detailed trace information—so you can quickly identify the root cause and resolve issues faster, without digging through logs.

You can also "Copy Trace URL" to quickly share logs with your team.

Learn more about error messages and logging in the Docs here.

📘

DB2 Self-Service Connector

You can now easily set up DB2 connections directly from the Empower interface.

This makes it faster and easier to integrate with one of the most widely used enterprise databases, especially in financial services, insurance, and legacy systems. Whether you're modernizing mainframe data or unlocking insights from critical systems of record, DB2 support helps bring more of your core data into the Lakehouse.

To learn more about DB2 connections, visit our Docs here.

Itemized Changelog

Enhancements & New Features

  • Displayed Empower documentation directly within the application interface during onboarding.
  • Added sorting functionality for the Active column in Workflows.
  • Implemented an empty state view for the side panel in Workflows.
  • Removed pagination for large object lists in the Metadata Catalog.
  • Updated Watermark field options in the Metadata Catalog.
  • Refined requirements for merge strategies in the Metadata Catalog.
  • Integrated UTF-8 options into Spark job pool configurations.
  • Deployed default Blob Tiering policies for Datalake environments.
  • Developed the DB2 Connector for IBM DB2 database integration.
  • Enabled persistence of configuration values when switching batch extraction options in Cluster Config.
  • Unfeatured flag-based group management from the roles page in Access Control.
  • Created template for converting Gold Layer Object to Liquid Clustering in Spark.
  • Added a view trace button to the error modal in Flows.
  • Updated descriptions in Cluster Config for clarity.
  • Hid Cluster Config options for non-spark data sources.
  • Implemented various UI updates on the Connect page.
  • Updated PowerBI flow styles within Workflows.
  • Enabled jobLink data in flow details endpoint returns.
  • Provided a list of Databricks cluster names through Core API for Cluster Config.
  • Surface trace links for ingestion, modeling, and publishing processes in Troubleshooting tools.
  • Included ObjectPublish.job_link in flow history detail responses.

Bugfixes

  • Fixed Last Runs alignment issue in Workflows when Running state is included.
  • Corrected status update failure for objects on error due to JSON message issues.
  • Resolved tooltips inconsistency during Workflow run.
  • Fixed display of '-optional' values in Cluster Config form for Acquire flow edits.
  • Corrected console errors for stack size exceeding when toggling advanced options.
  • Addressed incorrect default cluster type display in Cluster Config for Acquire flows.
  • Resolved breadcrumb navigation issues upon switching to settings in Metadata Catalog.
  • Removed unwanted horizontal scrollbars in side navigation for small screens.
  • Fixed metadata extraction errors with SAP Hana and PostgresQL.
  • Resolved visual issue with breadcrumbs application-wide.
  • Updated field label from "Connection Name" to "Display Name" on Connect screen.
  • Fixed alignment of "Source Notebook" information on Workflows Configuration page.
  • Corrected pointer cursor behavior for "?" in object failures modal.
  • Ensured correct documentation redirection via Need Help button.
  • Addressed incorrect default documentation display from Need Help button.
  • Removed unexpected text from "New Publish Flow" form.
  • Fixed "Node Size" field display in edit mode for old flows.
  • Resolved '-Optional' label issues when switching from non-spark to spark data sources.
  • Fixed incorrect API calls in Metadata Catalog connections.
  • Corrected cluster config mismatch for custom flows from Regression workflows.
  • Resolved navigation issue in error modal with "View Trace" button.
  • Fixed UI issues with trace buttons for null jobLinks.
  • Corrected error styling for Watermark columns in Metadata Catalog.
  • Enabled inactive connections to be selectable.
  • Resolved publishing errors due to schema column issues.
  • Ensured archived connections appear in the connections list.
  • Fixed missing "JobLink" population after flow runs.