Getting Started with Dynamics V2

Initial Setup and Configuration

The Dynamics V2 library provides a powerful interface for managing data synchronization with Dynamics CRM. This guide walks you through the essential setup process and core configuration concepts.

Basic Installation and Setup

To begin using the Dynamics V2 library, first import the main Dynamics class:

from empowerspark.target.type.dynamics_v2 import Dynamics

# Create a new Dynamics client instance
dynamics_client = Dynamics()

After creating the client instance, you'll need to configure two critical database settings that determine where the library stores its operational data and logs:

# Configure the artifact schema that will hold the logs for this package
dynamics_client.__artifact_db__ = "empower_ddu"

# Set the name of the high-level batch log table for tracking execution metrics
dynamics_client.__log_table_name__ = "batch_log"

Secret Management and Authentication

The library uses Azure Key Vault for secure credential management. Configure your client with the appropriate secrets using:

# Configure the client with your connection string secret
dynamics_client.configure("dataverse-connection-string")

Secret Structure and Requirements

Your base secret in Azure Key Vault should contain a JSON object with the following required fields:

{
    "dynamics_url": "https://your-instance.crm.dynamics.com",
    "azure_tenant": "your-azure-tenant-id",
    "client_id": "your-client-id",
    "client_secret": "your-client-secret",
    "dynamics_erp": "your-erp-name",
    "extra_clients_secret_name": "secret-containing-additional-service-principals"
}

For enhanced performance with multiple service principals, store additional credentials in a separate secret using the format:

OAuthClientID=<client-id>;OAuthClientSecret=<client-secret>;

Performance Configuration

Worker Thread Management

The library uses a thread pool for parallel processing. The optimal number of worker threads depends on your service principal configuration:

The base calculation is: number_of_service_principals * 50
Each service principal supports up to 52 concurrent connections
For example, with 32 service principals, configure: 32 * 50 = 1,600 workers

base_config = {
    "advanced.performance.num_workers": 1600  # Adjust based on your service principals
}

Basic Usage Example

Here's a complete example showing how to perform a basic upsert operation:

# Initialize and configure the client
dynamics_client = Dynamics()
dynamics_client.__artifact_db__ = "empower_ddu"
dynamics_client.__log_table_name__ = "batch_log"
dynamics_client.configure("dataverse-connection-string")

# Configure performance settings
dynamics_client.update_config({
    "advanced.performance.batch_size": 50_000,  # see note below
    "advanced.performance.num_workers": 1600,  # see note below
})

# Perform an upsert operation
result = dynamics_client.load(
    source_table="your_source_table",
    key_columns=["id"],  # Primary key or business key columns
    target_table="contact",  # Dynamics target table
    mode="upsert"
)

NOTE: For detailed information on configuration settings see Advanced Configuration Guide