What is Unity Catalog?

Unity Catalog is a unified security and governance feature set built by Databricks. It provides several critical features that make Empower a more robust data estate solution.

UC enables Delta Sharing, providing the ability to share read-only copies of your data securely both internally and with your partners and associates without requiring them to be on the Databricks platform. UC also provides the ability to create row-filters and column masks for different users and groups in SQL, giving you fine-grained control over your data security and access.

Unity Catalog comes with the 1.14 Empower release to all customers, and as such there are a few required steps necessary to ready the foundation for 1.14's deployment.

Initial Configuration Values

Steps

1. Provide the Empower-Service service principal with Databricks Account Admin access

   1. Note: You will need an account with either Databricks Account Admin access or Azure Global Access Administrator permissions.
   2. Login to the account console at https://accounts.azuredatabricks.net.
   3. Perform this action through the Databricks Account Console:
      1. In a Databricks workspace, navigate to the top right drop-down menu and select "Manage Account."
      2. Under the "User Management" tab, click on the "Service Principals" tab and click the blue button on the right-hand side of the screen to add a new service principal.
      3. Name the service principal "Empower-Service" and use the service principal's Object ID for the UUID field. (this will be different for each directory, please look it up in AAD).
      4. Once the service principal is added, click on its name and navigate to the "Roles" tab. Then enable "Account Admin."

2. Create a resource group for the Databricks Metastore

📘

This step is optional; you may ask Hitachi Solutions to create this on your behalf.

• This step must be repeated for every Azure region where your company has an Empower solution deployment.

1. Keep in mind the following considerations:
   1. The metastore will be global for all Databricks workspaces in a given Azure region. Therefore, the resource group name need not be Empower-specific e.g: [company name]-[region]-unitycatalog.
   2. The name will be immutable. Once deployed, the Unity Catalog cannot be moved.
   3. The name should conform to your company's resource group naming conventions.
2. Please provide a hitachi team member with at least reader on this resource group.
3. Please provide Empower-Service with Owner over this resource group.

Required Values for Deployment

Please provide the following values to your Hitachi Empower contact for each metastore deployment:

1. Resource group name created in Step 2
2. Subscription ID that holds the resource group
3. Metastore naming: The metastore will require three names provided, which will be deployed by our team.
   1. Metastore name (lowercase alphanumeric characters and hyphens). Suggested name: {location}-metastore
   2. Metastore storage account name (lowercase alphanumeric characters only) Suggested name: {location}metastore
   3. Metastore storage account container name (lowercase alphanumeric characters and hyphens) Suggested name: {location}-metastore
4. A /24 block of IP addresses for the unity catalog Vnet. The Vnet itself does not need to be deployed. We require an unused block of IP addresses to use in our automated deployment.