Skip to main content
Procore

Export to Fabric Lakehouse Using Data Factory

  1. Last updated
    Jan 31, 2025
  2. Save as PDF

Overview

Integrating Delta Sharing with Microsoft Fabric Data Factory enables seamless access and processing of shared Delta tables for your analytics workflows with Procore Analytics 2.0. Delta Sharing, an open protocol for secure data collaboration, ensures organizations can share data without duplication.

Prerequisites

  •  Procore Analytics 2.0 SKU
  • Delta Sharing Credentials:
    • Obtain the share.json (or equivalent) Delta Sharing credentials file from your data provider.
    • This file should include:
      • Endpoint URL: The Delta Sharing Server URL.
      • Bearer Token: Used for secure data access.
  • Microsoft Fabric Setup:
    • A Microsoft Fabric tenant account with an active subscription.
    • Access to a Microsoft Fabric-enabled workspace.

Steps

Switch to the Data Factory Experience

  1. Navigate to your Microsoft Fabric Workspace.
    clipboard_efd065fed3e214b84c9b311d011d8ae67.png
     
  2. Select New, then choose Dataflow Gen2.
    clipboard_e73cb72aa551cc8730795b5bd53bb8a7a.png

Configure the Dataflow

  1. Go to the dataflow editor.
  2. Click Get Data and select More.
  3. Under New source, select Delta Sharing Other as the data source.
    clipboard_efb6b95e01a2211ba291f186035b8b4ab.png
  4. Enter the following details:
    • URL: From your Delta Sharing configuration file.
    • Bearer Token: Found in your config.share file.
      clipboard_eb997dd0c67cf6e2bcf47cfc9d7e5d71c.png
  5. Click Next and select the desired tables.
  6. Click Create to complete the setup.

Perform Data Transformations

After configuring the dataflow, you can now apply transformations to the shared Delta data. Choose your Delta Sharing Data option from the list below:

  • Add Data Destination
  • Create/Open Lakehouse

Add Data Destination

  1. Go to Data Factory.
  2. Click Add Data Destination.
  3. Select Lakehouse as the target and click Next.
  4. Choose your destination target and confirm by clicking Next.

Create-Open Lakehouse

  1. Create/open yourLakehouse and click Get data.
  2. Select New Dataflow Gen2.
  3. Click Get Data, then More and find Delta Sharing.
  4. Enter the URL bearer token from your config.share file and then select Next.
  5. Choose your data/table/s to download and click Next.
  6. After these manipulations, you should have all the selected data in your Fabric Lakehouse.

Validation and Monitoring

Test your data pipelines and flows to ensure smooth execution.Use monitoring tools within Data
Factory to track progress and logs for each activity.