Spreadsheets in the modern data stack: security, governance, AI, and self-serve analytics

Databricks · Beginner ·📊 Data Analytics & Business Intelligence ·1y ago

Key Takeaways

The video discusses the role of spreadsheets in modern data analytics, covering security, governance, AI, and self-serve analytics, with a focus on Databricks and Row Zero solutions.

Full Transcript

Uh thank you everyone for coming. Uh my name is Nick End. I'm one of the founders of Row Zero. Uh so Row Zero is a spreadsheet for the modern data stack and I'll get into the details of the product uh later on in the presentation. Um yeah, glad to see some faces here. Spreadsheets are not the sexiest topic in the era of AI, but there are those of us that care a lot about spreadsheets. So, I suspect you're here because you either love spreadsheets or you're on the data team and spreadsheets are a pain in the neck. Uh, so we'll address some of those issues. Uh, so if you take a look at this diagram, I I suspect that it is a familiar workflow uh for many of you. Uh, you have a data warehouse. You guys are using data bricks. You put BI tools on top of your data warehouse. So Tableau, PowerBI, uh Looker, Quicksite, your team will build dashboards that your business partners consume. And uh the most common action on those dashboards, I suspect is export to CSV so the data can be opened in a spreadsheet. Um and we hear this from from tons of our customers and and every company we talk to. Uh for 20 years, BI tools have been selling the promise of self-s serve analytics and it still has not materialized. Spreadsheets are the most ubiquitous data analysis tool in the world. Supposedly, Google Sheets and Excel have billions of users. Uh and so spreadsheets aren't going away. Uh there are many problems with this workflow. Uh which I will start to dive into. Uh so you might be wondering why why are people exporting data from BI tools? Every BI tool claims it's self-s serve. You can get your business people to work with data, but but that's just not the reality. Uh we hear this from from every company. If you talk to consultants, it's a a running joke in the data world that every BI every BI dashboard is just a very fancy CSV export. Now the reason BI tools evolved, this happened 20 years ago, is data sets started to outgrow the capabilities of Excel. And uh you know when Excel was invented 40 years ago and even for the first 20 years, a 1 million row data set was not uh not very common. But today, every business user regularly comes in contact with data sets that either exceed the Excel row limit or cause Excel to crash. Um, and and for those of you who are not aware, Excel has a 1,48,000 row limit. Uh, it it hits performance constraints long before you actually get to that point. Google Sheets is about 10 times uh less powerful. It has a 10 million cell limit, so it will crash much much faster than Excel. Uh, BI tools offer power. So, you can put them on top of giant data sets, create these dashboards. Doesn't matter if your data set is a billion rows. Uh, your BI tool can handle it. they connect to the cloud data warehouse. When Excel uh was first designed, cloud computing wasn't a thing. Uh the same goes for Google Sheets. Google Sheets launched in uh 2007, which I think is the same year that S3 launched. And so cloud computing had not been invented. Um and then the the nice thing about BI tools is they offer better security. Uh you don't have data floating around on laptops. You can enforce uh security from your data warehouse, arbback, and RLS. The problem that dashboards and BI tools have created is every time a business business partner comes to a data team and asks uh for access to a data set or a different slice of data, the data team immediately creates another dashboard and the first thing that business person does is export to Excel. And so what ends up happening is data teams will create thousands of dashboards. Uh, our largest customer has a data team of 200 people and they've created over 3,000 uh, Tableau and Quicksite dashboards. I thought that was impressive and I was talking to a guy earlier today who said they did an audit last year. This is a telecom company and uh, they had created over 9,000 dashboards in the past five years. Uh, and so not only is that a waste of time if people are exporting to Excel, uh, but it's hard to maintain. Like that's a a burden on your team. Uh the challenge with dashboards and BI tools is they're rigid. They're only as good as the use cases and analyses that are pre-anticipated by the creator. Uh so you have to anticipate the filters or the group buys or the aggregations that your business teams want and it's impossible to anticipate all the different configurations which is why business teams come back to the data team again and again for new slices of data, new dashboards and you just end up in dashboard overload. Um, and then lastly, despite what BI companies say about self-s serve analytics, business people don't create dashboards. They will consume them, export from them, but they don't want to learn Tableau. They don't know SQL. Uh, they just want to get their data in a tool they know how to use. Spreadsheets aren't going away. I assume you've guessed that based on the subject of this presentation. Uh they're ubiquitous for the main reason is because of the UI. Everybody knows how to use a spreadsheet. You're introduced to them very early on in your career, oftentimes in high school, college. Uh it's just it lets you see your data. It lets you scroll through your data, interact with it on a cellbyell basis. And it's the most intuitive UI we have for analyzing data. your business partners in finance, operations, sales, and marketing uh only want spreadsheets. Despite uh the proliferation of BI tools, uh they're still going back to the spreadsheet time and time again. But spreadsheets have problems. So, like I've said, I love Excel and Google Sheets. Uh but they they come with some some real heartache. So, the the issues with Excel and Google Sheets are not the UI. The spreadsheet UI is great, but spreadsheets have this stigma because of their power, their connectivity, and their security. So, as as I've expounded upon, uh, Excel and Google Sheets have these small row limits, but outside of power, they're not connected. Uh, so Excel obviously can connect to data bricks, but you're using an OBBC driver, which is tied to a laptop, which means you cannot share that file with other people. Uh, it's also really slow and fragile. So with modern data sets will take a ton of time to load data uh in an ODBC driver. Uh so they're not connected. Google Sheets also has similar connectivity challenges. Um there have been some advancements but generally they're really slow connections and brittle. Uh and then the last is security. And I think this one probably um gives data teams the most constrnation. Uh, anytime somebody is downloading a CSV or opening an Excel file on their laptop, that's now locally stored on someone's computer. It could be emailed to the wrong person. The laptop could be lost. Uh, it's now totally untraceable. So, you've you've lost that data. And the security risk with spreadsheets is this latent risk that many many big companies have just accepted. Uh we've talked to a number of companies that have said every time they do a security review, a data security review, the top risk is spreadsheets stored on people's laptops. Uh we talked to a Fortune 500 company recently whose CEO's laptop was stolen in a major airport and their their biggest concern about that theft was the spreadsheets stored on that laptop. Um one of our customers is again a big company. uh they have had CSV exports from their BI tools walk away and go to competitors and they want to eliminate the CSV risk and the Excel risk from their company. Uh but the UI is not the problem and so there should not be this stigma around spreadsheets. There should be a stigma around locally hosted uh insecure unconnected disconnected and and uh unpowerful spreadsheets. And so if you could imagine a spreadsheet in the modern data stack, it would be powerful enough to handle modern data set sizes. Uh so it would open billions of rows. You would have the same formulas as you get in Excel and Google Sheets. You could pivot, graph, filter, do everything people are used to doing. So there'd be no learning curve. It would be built in the cloud, so it could connect natively to data bricks and other cloud data warehouses. Uh you can read from it. You could also write back to it. So complete interactivity with your data store. Uh and then you would have enterprisegrade security. So you'd enforce arbback, rls, inherit those from uh your data warehouse. You could restrict data export. Uh maybe you could configure it so data can't leave the spreadsheet, but business teams still get the spreadsheet they want. Uh and you'd allow sharing, collaboration, but with full traceability. And that is what we've built at row zero. Uh the diagram on top is the one I started with and I've sort of covered all the risks with that. You've got way too many dashboards. CSV exports are a security nightmare. Uh and the legacy spreadsheets have performant issues. You can't do modern data analysis in legacy spreadsheets. Now with row zero, we sit on top of data bricks. You can import data from data bricks, write it back to uh your cloud data warehouse and we live alongside BI tools. Uh when you think about ways to handle spreadsheet proliferation, data teams essentially have two options. You can either acknowledge that it exists and let people download uh from from your BI tools or you try to clamp down on it and then you end up creating thousands of dashboards. And in a row zero world, you maintain your executive scorecards, your highle dashboards. uh those are still great for reporting and we're not a replacement for those. But all the other ad hoc analysis that your business teams are doing can fall into row zero. You can share connected data sources with them where the data is just dropped into a spreadsheet and they can perform all the analysis they need without the help and support of a data team. Now you might be wondering in this era of AI, do we actually need spreadsheets? Uh valid question. Uh we believe yes. Yes, you do need spreadsheets in the era of AI. Um for two reasons. Uh so LLMs are great, but there are two inherent challenges. Uh so first is LLMs are not yet good enough to just give us an answer that we inherently believe. Uh if you use a natural language prompt, get an answer from one of the many AI analytics tools. Let's say the answer is 42, you don't know if it's accurate, right? AI often often makes mistakes and so you need some way to audit that response. And the majority of AI analytics tools today are performing analysis in Python or SQL. They're like writing code to interrogate and analyze data and then spitting out a response. Sometimes some visuals, graphics, pivot tables, uh subsets of data. The problem is the business people we've been talking about do not know Python or SQL. So they have no way to look at this code and assess whether the result is accurate. Uh and so so like what's the solution there? Uh the spreadsheet is the UI that business people can audit and can augment. And so if an AI agent produced a spreadsheet with pivot tables, cell references, formulas, graphs, uh any average business person can take a look at that data, take a look at the graphs and assess whether they think the analysis is valid. If not, you can try again. The other issue is performance. Uh so context windows for LLMs are not big enough to feed in entire data sets. Uh, and even if you could, it's not clear that feeding an entire data set to an agent would make the analysis any better. Really, what you need is to provide LLM with tools to interrogate and analyze data. And currently, as I mentioned, they're using code to do that. But another option you could imagine is a spreadsheet API where an LLM can now create graphs, can create pivot tables, can write formulas, um you know build an analysis and probe data with spreadsheet functions and spreadsheet formulas and that's where we are headed. Um so even in the age of AI, I think spreadsheets will continue to exist. At row zero, we've built a spreadsheet that runs in the cloud. Uh so it connects directly to data bricks. It can open billion row data sets and we provide enterprisegrade security. You can limit data export. You can limit sharing. Uh you have full traceability and versioning. Data teams love us for the built-in connectors. Uh like there's no easier way to connect your business partners to your cloud data warehouse. uh you have shared data sources where you can share a query with folks who don't know SQL and allow them to run it using their own credentials in data bricks. This will enforce RLS and arbback uh and so you don't have to worry about security issues and people exporting uh data to their local machines. Business teams love it because it works just like Excel. There's no learning curve. Uh every BI tool talks about self-s serve analytics but for a spreadsheet there is truly no learning curve. All the formulas are exactly the same. It's a thousand times more powerful. So, give us your biggest data sets. Uh, and it's connected and dynamic. No more building one-off analyses that you need to refresh every week or every month. When you build an analysis off of a connected data set, every time that data updates in data bricks, your spreadsheet refreshes, so your analysis is live uh and current. Uh, AI is on the road map. Uh, so you can expect some exciting things from us over the next few months. If anybody would like to talk about spreadsheets, we're at booth 308 and I'll be available uh after this talk.

Original Description

Despite the proliferation of cloud data warehousing, BI tools, and AI, spreadsheets are still the most ubiquitous data tool. Business teams in finance, operations, sales, and marketing often need to analyze data in the cloud data warehouse but don't know SQL and don't want to learn BI tools. AI tools offer a new paradigm but still haven't broadly replaced the spreadsheet. With new AI tools and legacy BI tools providing business teams access to data inside Databricks, security and governance are put at risk. In this session, Row Zero CEO, Breck Fresen, will share examples and strategies data teams are using to support secure spreadsheet analysis at Fortune 500 companies and the future of spreadsheets in the world of AI. Breck is a former Principal Engineer from AWS S3 and was part of the team that wrote the S3 file system. He is an expert in storage, data infrastructure, cloud computing, and spreadsheets. Talk By: Breck Fresen, CEO, Row Zero (Sponsor Speaker) Intelligent analytics for everyone: https://www.databricks.com/product/business-intelligence BI meets AI: https://www.databricks.com/resources/ebook/business-intelligence-meets-ai Explore the only unified platform for agent systems: https://www.databricks.com/product/artificial-intelligence See all the product announcements from Data + AI Summit: https://www.databricks.com/events/dataaisummit-2025-announcements Connect with us: Website: https://databricks.com Twitter: https://twitter.com/databricks LinkedIn: https://www.linkedin.com/company/databricks Instagram: https://www.instagram.com/databricksinc Facebook: https://www.facebook.com/databricksinc
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from Databricks · Databricks · 0 of 60

← Previous Next →
1 Building AI Agent Systems with Databricks
Building AI Agent Systems with Databricks
Databricks
2 Databricks Workflows
Databricks Workflows
Databricks
3 Automate Unity Catalog Upgrade with UCX Part 1: Overview
Automate Unity Catalog Upgrade with UCX Part 1: Overview
Databricks
4 Automate Unity Catalog Upgrade with UCX Part 2: Installation
Automate Unity Catalog Upgrade with UCX Part 2: Installation
Databricks
5 Automate Unity Catalog Upgrade with UCX Part 3 - Assessment
Automate Unity Catalog Upgrade with UCX Part 3 - Assessment
Databricks
6 Automate Unity Catalog Upgrade with UCX  Part 4 - Group Migration
Automate Unity Catalog Upgrade with UCX Part 4 - Group Migration
Databricks
7 Table Migration and Catalog Design with UCX | Part 5
Table Migration and Catalog Design with UCX | Part 5
Databricks
8 Setting Up Azure Access for UCX Table Migration | Part 6
Setting Up Azure Access for UCX Table Migration | Part 6
Databricks
9 UCX Table Migration: Creating Catalogs and Schemas | Part 7
UCX Table Migration: Creating Catalogs and Schemas | Part 7
Databricks
10 Automate Unity Catalog Upgrade with UCX  Part 8: Code Migration
Automate Unity Catalog Upgrade with UCX Part 8: Code Migration
Databricks
11 Streaming to Kafka Just Got Easier with DLT Pipelines
Streaming to Kafka Just Got Easier with DLT Pipelines
Databricks
12 Data Engineering From Data to Dashboards with DABs: Crunching the Cookies Dataset
Data Engineering From Data to Dashboards with DABs: Crunching the Cookies Dataset
Databricks
13 Epsilon helps businesses connect with their consumers using Databricks Data Intelligence Platform
Epsilon helps businesses connect with their consumers using Databricks Data Intelligence Platform
Databricks
14 Unilever transforms operations with GenAI using the Databricks Data Intelligence Platform
Unilever transforms operations with GenAI using the Databricks Data Intelligence Platform
Databricks
15 ActionIQ enables businesses to unlock customer data with the Databricks Data Intelligence Platform
ActionIQ enables businesses to unlock customer data with the Databricks Data Intelligence Platform
Databricks
16 Mixed Attention & LLM Context | Data Brew | Episode 35
Mixed Attention & LLM Context | Data Brew | Episode 35
Databricks
17 Inside Databricks SQL: Engineering innovation with Hans
Inside Databricks SQL: Engineering innovation with Hans
Databricks
18 Inside Databricks: Engineering innovation with Michael Armbrust
Inside Databricks: Engineering innovation with Michael Armbrust
Databricks
19 The Money Team at Databricks: driving revenue and customer growth
The Money Team at Databricks: driving revenue and customer growth
Databricks
20 Unity Catalog unveiled: engineering data governance at scale
Unity Catalog unveiled: engineering data governance at scale
Databricks
21 Create a view in Databricks and share it with Power BI using Delta Sharing
Create a view in Databricks and share it with Power BI using Delta Sharing
Databricks
22 NDUS leverages Databricks Data Intelligence Platform to revolutionize higher education management
NDUS leverages Databricks Data Intelligence Platform to revolutionize higher education management
Databricks
23 Démo Databricks de AI/BI
Démo Databricks de AI/BI
Databricks
24 EMEA Data + AI World Tour 2024
EMEA Data + AI World Tour 2024
Databricks
25 GenAI: The Shift to Data Intelligence - Customer Panel on Industry Use Cases
GenAI: The Shift to Data Intelligence - Customer Panel on Industry Use Cases
Databricks
26 GenAI: The Shift to Data Intelligence - Ft. Ash Jhaveri, VP of Reality Labs Partnerships at Meta
GenAI: The Shift to Data Intelligence - Ft. Ash Jhaveri, VP of Reality Labs Partnerships at Meta
Databricks
27 Virtue Foundation leverages the Databricks Data Intelligence Platform to advance global health
Virtue Foundation leverages the Databricks Data Intelligence Platform to advance global health
Databricks
28 Announcing Synthetic Data Generation in Mosaic AI Agent Evaluation
Announcing Synthetic Data Generation in Mosaic AI Agent Evaluation
Databricks
29 AI/BI Dashboards Embedding - A tutorial
AI/BI Dashboards Embedding - A tutorial
Databricks
30 Bayer transforms global data management with the Databricks Data Intelligence Platform
Bayer transforms global data management with the Databricks Data Intelligence Platform
Databricks
31 Databricks at AWS re:Invent 2024
Databricks at AWS re:Invent 2024
Databricks
32 Hive Metastore and AWS Glue Federation in Unity Catalog
Hive Metastore and AWS Glue Federation in Unity Catalog
Databricks
33 Data + AI World Tour Paris 2024
Data + AI World Tour Paris 2024
Databricks
34 Retail reimagined: Currys data-first strategy to driving growth and improving operations
Retail reimagined: Currys data-first strategy to driving growth and improving operations
Databricks
35 Mixture of Memory Experts (MoME) | Data Brew | Episode 36
Mixture of Memory Experts (MoME) | Data Brew | Episode 36
Databricks
36 Verana Health Data Curation and Innovation with Databricks and AWS
Verana Health Data Curation and Innovation with Databricks and AWS
Databricks
37 Securing SaaS Applications: Obsidian Security on Their Journey with Databricks and AWS
Securing SaaS Applications: Obsidian Security on Their Journey with Databricks and AWS
Databricks
38 Twilio Eng VP on Data Intelligence & AI at AWS re:Invent 2024
Twilio Eng VP on Data Intelligence & AI at AWS re:Invent 2024
Databricks
39 Chegg Eng SVP on Data-Driven Approach to Student Success with Databricks and AWS
Chegg Eng SVP on Data-Driven Approach to Student Success with Databricks and AWS
Databricks
40 Ibotta Personalized Rewards Innovation with Databricks and AWS
Ibotta Personalized Rewards Innovation with Databricks and AWS
Databricks
41 Simplify AI governance with #databricks AI Gateway
Simplify AI governance with #databricks AI Gateway
Databricks
42 Databricks SQL and Power BI Integration
Databricks SQL and Power BI Integration
Databricks
43 Databricks Serverless SQL Warehouses
Databricks Serverless SQL Warehouses
Databricks
44 7 West powers audience growth with the Databricks Data Intelligence Platform
7 West powers audience growth with the Databricks Data Intelligence Platform
Databricks
45 Secret to Production AI: Tools & Infrastructure | Data Brew | Episode 37
Secret to Production AI: Tools & Infrastructure | Data Brew | Episode 37
Databricks
46 Skyflow CEO on Data Privacy with Databricks at AWS re:Invent
Skyflow CEO on Data Privacy with Databricks at AWS re:Invent
Databricks
47 Databricks Clean Rooms Product Demo
Databricks Clean Rooms Product Demo
Databricks
48 Dun & Bradstreet Enrichment & Monitoring, powered by Delta Sharing & Databricks Marketplace
Dun & Bradstreet Enrichment & Monitoring, powered by Delta Sharing & Databricks Marketplace
Databricks
49 Unpacking Libraries in Databricks
Unpacking Libraries in Databricks
Databricks
50 Providence uses an AI agent system from Databricks to help doctors improve their communication
Providence uses an AI agent system from Databricks to help doctors improve their communication
Databricks
51 How State Street Uses AI to Transform Millions of Trades Daily
How State Street Uses AI to Transform Millions of Trades Daily
Databricks
52 Vevo Therapeutics CEO on Curing Disease with Data at AWS re:Invent
Vevo Therapeutics CEO on Curing Disease with Data at AWS re:Invent
Databricks
53 Over Architected with Nick & Holly: Databricks updates for Feb 2025
Over Architected with Nick & Holly: Databricks updates for Feb 2025
Databricks
54 The Power of Synthetic Data | Data Brew | Episode 38
The Power of Synthetic Data | Data Brew | Episode 38
Databricks
55 Use Databricks Lakehouse Federation to break down data silos
Use Databricks Lakehouse Federation to break down data silos
Databricks
56 AI's rugby score: National Rugby League rallies fans with analytics and unified data
AI's rugby score: National Rugby League rallies fans with analytics and unified data
Databricks
57 Open Variant Data Type in Delta Lake and Apache Spark
Open Variant Data Type in Delta Lake and Apache Spark
Databricks
58 How would you sort Ætheldred in the alphabet using Databricks?
How would you sort Ætheldred in the alphabet using Databricks?
Databricks
59 A guide on how to operationalize the Databricks AI Security Framework (DASF)
A guide on how to operationalize the Databricks AI Security Framework (DASF)
Databricks
60 Future-Proof Your Asset Performance Management with Generative AI - Field Assistant Live Demo
Future-Proof Your Asset Performance Management with Generative AI - Field Assistant Live Demo
Databricks

This video discusses the importance of spreadsheets in modern data analytics and how to support secure spreadsheet analysis with AI and BI tools. It covers strategies for data teams to provide access to data while maintaining security and governance. The video also explores the future of spreadsheets in the world of AI.

Key Takeaways
  1. Understand the role of spreadsheets in modern data analytics
  2. Learn about security and governance in spreadsheet analysis
  3. Explore AI and BI tools for data analysis
  4. Integrate SQL with spreadsheets
  5. Apply AI to data analysis
💡 Spreadsheets remain a crucial tool for data analysis, and securing and governing their use is essential for data teams.

Related Reads

📰
Exploratory Data Analysis (EDA) — New York city Yellow taxi — Part 1: Data Preparation
Learn to prepare data for exploratory data analysis using the New York City Yellow taxi dataset, a crucial step in understanding and visualizing data insights.
Medium · Data Science
📰
Segmentando Clientes com Análise Fatorial e Clustering
Learn to segment customers using factor analysis and clustering, reducing 14 variables to 4 personas
Medium · Data Science
📰
From Four Platforms to One: How Tongcheng Travel Built a Unified Data Integration Platform with…
Learn how Tongcheng Travel unified four data integration platforms into one using Apache technologies and a batch-stream architecture
Medium · Data Science
📰
Longitudinal Data Infrastructure
Learn how longitudinal data infrastructure can become AI's next foundation for continuity
Medium · AI
Up next
This could be the most perfect data frontend
Matt Williams
Watch →