Setting Up Azure Access for UCX Table Migration | Part 6

Databricks · Beginner ·🛠️ AI Tools & Apps ·1y ago

Key Takeaways

This video tutorial demonstrates how to set up Azure access for UCX table migration, covering essential Azure CLI commands, permissions setup, secure storage access configuration, and troubleshooting common access issues using tools like Azure CLI and Databricks Unity Catalog.

Full Transcript

hey everyone welcome to this next video on the ucx series and the second part of the table migration flow I'm Andrea and in this video we'll be covering the next commands required to be executed before proceeding with the table migration these commands are going to be focused on authentication and access so we need to deploy and configure a lot of things on the cloud small disclaimer here this video will be focused specifically on the aser Cloud functionality and usage of commands will vary depending on the cloud that you're using so make sure to follow the correct guide let's get started first it's useful to understand where we're at that is what we have done so far what we are going to do in this step and what's missing for the next video as you can see here we have already completed the table mapping so ucx knows what our desired design is so now we need to execute the four commands in the red box let's understand what each of these is going to do our first stop is the principal prefix access command to execute this and the following terminal commands will need the appropriate authentication in this case the Asher CLI with the proper credentials this command will identify all of the storage accounts used on the tables of the workspace for later grants once we have executed this command we can move on to the next one the migrate credentials command this like the rest will work differently on each Cloud for ashure it will create a new access connector for each of the storage accounts detected previously this is optional but recommended this will also migrate any aure service principles with storage blob data contributor data reader data owner roles on any ADLs Gen 2 locations being used on data braks to Unity catalog storage credentials after this process is completed those respective storage credentials will be created on the DAT site and available for use moving on to the migrate locations command this one will take the output of the assessment and the migrate credentials command to create all of the required external locations pointing to the Right Storage accounts and storage credentials used and created in the previous step you can validate the external locations that will be created by using the validate external locations command the final step is to execute the create Uber principle command this will create an asro service principle with read only access to the detected storage accounts the migration workflows will use this service principle to ensure ucx has access to the data now that we understand the process and what it does let's start executing the commands first let's run the principle prefix command again I'm using custom authentication profile that has access to my workspace for this set of commands we also have to specify the subscription ID or IDs where our storage accounts are located I executed the command so now it's using my Asher CLI authentication making sure that I have access and then executing the command commands in this case once it's finished executing we can move on to the next one the migrate credentials command so let's give this a second again this will take some time depending on your configuration of the workspace how many assets you have and how many objects it has to go through so now we can move on to the next one migrate credentials command again I'm specifying the subscription IDs and my authentication profile one once this command is completed we'll be able to see a new access connector or multiple in a resource Group as well as the storage credential on the DAT R site first we need to confirm if we need if we want to create an access connector for each storage account this is what I mentioned that it's optional but recommended so let's see yes this is going to proceed with that step creating those access connectors in my resarch group in this case just one and then it's going to ask me if I want to migrate my service principles as well in this case I don't have any so it will say that it found zero so I can just proceed saying that I don't want to M anything and then the execution is complete so now if we check we can see on the workspace we have a new storage credential and this storage credential is pointing to a new access connector so if we go to Asher and refresh we'll be able to see that storage access connector created on the AER side with a tag that says create by ucx great now we have the storage credentials so we need to create the external locations let's run the appropriate command in this case we have to execute the migrate locations using the correct subscription again and the profile for authentication of the workspace for this one we can also check the workspace once execution is finished and we should be able to see a new external location pointing to this storage account that I'm using for the demo and using the storage credential that was created in the previous step so let's check it out let's go back and we'll see now that we have a new exter location that's pointing to my container and storage account and that it's using my storage credential that I created in the previous step finally the last command that we will use in this video will be the create duber principle so let's run that for this command we can also check that the correct changes were performed on the Asher side as I said and explain on the presentation this will create a new service principle on the tenant so if we go to the tenant after this is created right now it's asking me for a name I'll just leave it as default after this is created we can see the confirmation so we can go to the tenant refresh and now we have that service principle again this service principle will have access to my storage account or storage accounts and it's the one that's going to be used for the migration of the tables in the migration workflows in the next video we'll finish the table migration flow to complete a migration from hive metas store into Unity catalog thank you

Original Description

Master Azure storage access for UCX workflows in Unity Catalog migrations. This tutorial covers: • Essential Azure CLI commands for UCX storage access • Setting up proper permissions for UCX in Azure • Best practices for secure storage access configuration • Troubleshooting common access issues GitHub repository for UCX: https://github.com/databrickslabs/ucx/blob/main/README.md
Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from Databricks · Databricks · 8 of 60

1 Building AI Agent Systems with Databricks
Building AI Agent Systems with Databricks
Databricks
2 Databricks Workflows
Databricks Workflows
Databricks
3 Automate Unity Catalog Upgrade with UCX Part 1: Overview
Automate Unity Catalog Upgrade with UCX Part 1: Overview
Databricks
4 Automate Unity Catalog Upgrade with UCX Part 2: Installation
Automate Unity Catalog Upgrade with UCX Part 2: Installation
Databricks
5 Automate Unity Catalog Upgrade with UCX Part 3 - Assessment
Automate Unity Catalog Upgrade with UCX Part 3 - Assessment
Databricks
6 Automate Unity Catalog Upgrade with UCX  Part 4 - Group Migration
Automate Unity Catalog Upgrade with UCX Part 4 - Group Migration
Databricks
7 Table Migration and Catalog Design with UCX | Part 5
Table Migration and Catalog Design with UCX | Part 5
Databricks
Setting Up Azure Access for UCX Table Migration | Part 6
Setting Up Azure Access for UCX Table Migration | Part 6
Databricks
9 UCX Table Migration: Creating Catalogs and Schemas | Part 7
UCX Table Migration: Creating Catalogs and Schemas | Part 7
Databricks
10 Automate Unity Catalog Upgrade with UCX  Part 8: Code Migration
Automate Unity Catalog Upgrade with UCX Part 8: Code Migration
Databricks
11 Streaming to Kafka Just Got Easier with DLT Pipelines
Streaming to Kafka Just Got Easier with DLT Pipelines
Databricks
12 Data Engineering From Data to Dashboards with DABs: Crunching the Cookies Dataset
Data Engineering From Data to Dashboards with DABs: Crunching the Cookies Dataset
Databricks
13 Epsilon helps businesses connect with their consumers using Databricks Data Intelligence Platform
Epsilon helps businesses connect with their consumers using Databricks Data Intelligence Platform
Databricks
14 Unilever transforms operations with GenAI using the Databricks Data Intelligence Platform
Unilever transforms operations with GenAI using the Databricks Data Intelligence Platform
Databricks
15 ActionIQ enables businesses to unlock customer data with the Databricks Data Intelligence Platform
ActionIQ enables businesses to unlock customer data with the Databricks Data Intelligence Platform
Databricks
16 Mixed Attention & LLM Context | Data Brew | Episode 35
Mixed Attention & LLM Context | Data Brew | Episode 35
Databricks
17 Inside Databricks SQL: Engineering innovation with Hans
Inside Databricks SQL: Engineering innovation with Hans
Databricks
18 Inside Databricks: Engineering innovation with Michael Armbrust
Inside Databricks: Engineering innovation with Michael Armbrust
Databricks
19 The Money Team at Databricks: driving revenue and customer growth
The Money Team at Databricks: driving revenue and customer growth
Databricks
20 Unity Catalog unveiled: engineering data governance at scale
Unity Catalog unveiled: engineering data governance at scale
Databricks
21 Create a view in Databricks and share it with Power BI using Delta Sharing
Create a view in Databricks and share it with Power BI using Delta Sharing
Databricks
22 NDUS leverages Databricks Data Intelligence Platform to revolutionize higher education management
NDUS leverages Databricks Data Intelligence Platform to revolutionize higher education management
Databricks
23 Démo Databricks de AI/BI
Démo Databricks de AI/BI
Databricks
24 EMEA Data + AI World Tour 2024
EMEA Data + AI World Tour 2024
Databricks
25 GenAI: The Shift to Data Intelligence - Customer Panel on Industry Use Cases
GenAI: The Shift to Data Intelligence - Customer Panel on Industry Use Cases
Databricks
26 GenAI: The Shift to Data Intelligence - Ft. Ash Jhaveri, VP of Reality Labs Partnerships at Meta
GenAI: The Shift to Data Intelligence - Ft. Ash Jhaveri, VP of Reality Labs Partnerships at Meta
Databricks
27 Virtue Foundation leverages the Databricks Data Intelligence Platform to advance global health
Virtue Foundation leverages the Databricks Data Intelligence Platform to advance global health
Databricks
28 Announcing Synthetic Data Generation in Mosaic AI Agent Evaluation
Announcing Synthetic Data Generation in Mosaic AI Agent Evaluation
Databricks
29 AI/BI Dashboards Embedding - A tutorial
AI/BI Dashboards Embedding - A tutorial
Databricks
30 Bayer transforms global data management with the Databricks Data Intelligence Platform
Bayer transforms global data management with the Databricks Data Intelligence Platform
Databricks
31 Databricks at AWS re:Invent 2024
Databricks at AWS re:Invent 2024
Databricks
32 Hive Metastore and AWS Glue Federation in Unity Catalog
Hive Metastore and AWS Glue Federation in Unity Catalog
Databricks
33 Data + AI World Tour Paris 2024
Data + AI World Tour Paris 2024
Databricks
34 Retail reimagined: Currys data-first strategy to driving growth and improving operations
Retail reimagined: Currys data-first strategy to driving growth and improving operations
Databricks
35 Mixture of Memory Experts (MoME) | Data Brew | Episode 36
Mixture of Memory Experts (MoME) | Data Brew | Episode 36
Databricks
36 Verana Health Data Curation and Innovation with Databricks and AWS
Verana Health Data Curation and Innovation with Databricks and AWS
Databricks
37 Securing SaaS Applications: Obsidian Security on Their Journey with Databricks and AWS
Securing SaaS Applications: Obsidian Security on Their Journey with Databricks and AWS
Databricks
38 Twilio Eng VP on Data Intelligence & AI at AWS re:Invent 2024
Twilio Eng VP on Data Intelligence & AI at AWS re:Invent 2024
Databricks
39 Chegg Eng SVP on Data-Driven Approach to Student Success with Databricks and AWS
Chegg Eng SVP on Data-Driven Approach to Student Success with Databricks and AWS
Databricks
40 Ibotta Personalized Rewards Innovation with Databricks and AWS
Ibotta Personalized Rewards Innovation with Databricks and AWS
Databricks
41 Simplify AI governance with #databricks AI Gateway
Simplify AI governance with #databricks AI Gateway
Databricks
42 Databricks SQL and Power BI Integration
Databricks SQL and Power BI Integration
Databricks
43 Databricks Serverless SQL Warehouses
Databricks Serverless SQL Warehouses
Databricks
44 7 West powers audience growth with the Databricks Data Intelligence Platform
7 West powers audience growth with the Databricks Data Intelligence Platform
Databricks
45 Secret to Production AI: Tools & Infrastructure | Data Brew | Episode 37
Secret to Production AI: Tools & Infrastructure | Data Brew | Episode 37
Databricks
46 Skyflow CEO on Data Privacy with Databricks at AWS re:Invent
Skyflow CEO on Data Privacy with Databricks at AWS re:Invent
Databricks
47 Databricks Clean Rooms Product Demo
Databricks Clean Rooms Product Demo
Databricks
48 Dun & Bradstreet Enrichment & Monitoring, powered by Delta Sharing & Databricks Marketplace
Dun & Bradstreet Enrichment & Monitoring, powered by Delta Sharing & Databricks Marketplace
Databricks
49 Unpacking Libraries in Databricks
Unpacking Libraries in Databricks
Databricks
50 Providence uses an AI agent system from Databricks to help doctors improve their communication
Providence uses an AI agent system from Databricks to help doctors improve their communication
Databricks
51 How State Street Uses AI to Transform Millions of Trades Daily
How State Street Uses AI to Transform Millions of Trades Daily
Databricks
52 Vevo Therapeutics CEO on Curing Disease with Data at AWS re:Invent
Vevo Therapeutics CEO on Curing Disease with Data at AWS re:Invent
Databricks
53 Over Architected with Nick & Holly: Databricks updates for Feb 2025
Over Architected with Nick & Holly: Databricks updates for Feb 2025
Databricks
54 The Power of Synthetic Data | Data Brew | Episode 38
The Power of Synthetic Data | Data Brew | Episode 38
Databricks
55 Use Databricks Lakehouse Federation to break down data silos
Use Databricks Lakehouse Federation to break down data silos
Databricks
56 AI's rugby score: National Rugby League rallies fans with analytics and unified data
AI's rugby score: National Rugby League rallies fans with analytics and unified data
Databricks
57 Open Variant Data Type in Delta Lake and Apache Spark
Open Variant Data Type in Delta Lake and Apache Spark
Databricks
58 How would you sort Ætheldred in the alphabet using Databricks?
How would you sort Ætheldred in the alphabet using Databricks?
Databricks
59 A guide on how to operationalize the Databricks AI Security Framework (DASF)
A guide on how to operationalize the Databricks AI Security Framework (DASF)
Databricks
60 Future-Proof Your Asset Performance Management with Generative AI - Field Assistant Live Demo
Future-Proof Your Asset Performance Management with Generative AI - Field Assistant Live Demo
Databricks

This tutorial teaches how to set up Azure access for UCX table migration, covering essential Azure CLI commands, permissions setup, and secure storage access configuration. By following this tutorial, viewers can master Azure storage access for UCX workflows in Unity Catalog migrations. The tutorial is part of a series and provides hands-on experience with Azure and Databricks tools.

Key Takeaways
  1. Install Azure CLI
  2. Configure UCX storage access
  3. Set up proper permissions for UCX in Azure
  4. Implement secure storage access configuration
  5. Troubleshoot common access issues
  6. Verify UCX workflow functionality
💡 Proper permissions setup and secure storage access configuration are crucial for successful UCX table migration in Azure.

Related Reads

Up next
How to Get Your Brand Cited by ChatGPT, Claude and Gemini
Arvow
Watch →