Enterprise Apps Unpacked: Using a citizen developer program to boost AI deployments

Key Takeaways

[music] >> From Informa TechTarget, [music] I'm David Essex, and this is Enterprise Apps Unpacked. Most companies are still figuring out how to use AI. It's especially challenging when in-house development and data science is required to get AI applications that go beyond publicly available generic capabilities and meet specific needs of the business. A citizen developer program could be a solution. It involves giving employees tools to build their own apps instead of expecting on-staff programmers and IT departments to do it. I've seen efforts to democratize development with supposedly user-friendly tools before, and they usually fail to live up to their promise. For decades, there have been visual design and programming tools to turn the reimagined workflows of business process re-engineering and business process management strategies into software. More recently, low-code, no-code development platforms are said to provide modular drag-and-drop simplicity to both professional developers and non-technical business users. In theory, it's a good idea since non-programmers usually know how work actually gets done better than programmers do. But, how much are these tools really being used by us regular folks? Today's guest is Fabian Cross, chief data and AI officer at Decker Carle, a management consulting firm that established a citizen developer program late last year. Nearly half the company's 200 employees across several departments are participating, and the resulting AI automation has already cut operating costs by 3%. In the podcast, we talked about why they did it, how they did it, the results, and some of the challenges. Fabian also shares his advice on starting your own citizen developer program. Aligning software with the needs and capabilities of a business and its employees has historically been a major challenge for IT. It sometimes felt like a high priesthood would learn about a problem, go away and work its magic in secret, and return with a supposed solution that everyone was expected to use. Those days are gone, but translating business needs into software is always going to be a challenge. We talked about what happens when business users are given their own development tools, who handles which parts of the process, and the steps Stucker Carlisle took to ensure that everyone works together to produce real results. Don't forget to rate, review, and subscribe to Enterprise Apps Unpacked for the real story on the software that runs your business. Thanks very much, Fabian, for coming on the podcast today. Yeah, anytime. Thanks for having me, David. Um, I understand that Decker Carlisle created a citizen developer program to build and evaluate AI agents. Why did you move to that particular model? So, we failed a lot for a very long time until we found this this model of like decentralized AI and like this citizen developer program. One thing we realized was really like we had three type of people inside the company. We had kind of like technology people, you know, mostly coming with me from like Google type of like backgrounds, and they believe that they are the only one able to create technology, and you need, you know, you know, a fancy PhD to create AI agents. They want to centralize everything. They want to control everything. And then, you have two other buckets. You have these kind of like what I call like the bold business users. Those people are every time there is a release, they install ChatGPT, Claude, Gemini, everything, deep seek. So, they are creating a lot of shadow AI. But that's a good thing. I mean, they are trying to to to learn and they are trying to do something. And then you have the shy business users. Those people believe that, you know, tech is not for them. It's too complex, too complicated. So they do nothing. And we have 200 employees at the at the company. And so what we realized that we needed to give those bold business users a frame so they can build in a controlled environment. And we needed to give the shy business users a framework to reassure them so that that they are capable of building tech. And we had to give the tech team more like a different mission about building critical um uh use cases and technology for the company. So the way we kind of organized this full kind of like citizen developer program, it's really everybody in the company can be a citizen developer. They have tools. They have coaching. They have trainings. They have a framework. They have support to create their own agents. And then you have the central team which is really kind of like helping the decentralized citizen developers to build, but they don't spend like all all their time building for them. They are here to help them and and debug them. And then they are responsible to pick one, two, three big use case and develop that from like from A to Z so we can control everything and we build everything ourself. So it's really kind of like how we we develop this one. And again, it was based on lot of frustration about a centralized model. And a lot of requests from everybody in the organization because everybody wants to build things. So we had to do something for them. So what kinds of uh and development tools do you give to the citizen developers to use? So, what they have right to we decided we tried multiple uh options. We tried full open source and it's great when you are at the Google when you have like unlimited budget, but when you are in a way smaller firm, you know, I it was very expensive and very complex to maintain, to version everything. So, we decided to go away from open source. And we went to like a third-party provider, kind of like a a SaaS platform. So, every citizen developer they have access to this platform. So, it's a drag-and-drop platform. It's called stack.ai. We we believe that for us it was the best one for multiple reasons. And then on top of that, they have like we have weekly calls with like a central team helping them. We have monthly calls. We have actually like a program to track all the use cases to create business case on every single use case that the citizen developer are uh creating. And then we have a monitoring aspect tracking kind of like the usage of all those all those uh AI agent that the team is creating. So, it's a mix of it's a mix of technology, governance, support, and training, I would say. Can I give some examples of uh maybe the uh controls or uh rules or guidelines and expectations Yeah. that you had to set up so that this would work well? So, typical guidelines it's what LLMs to use. So, we decided not to to restrict our like platform to one LLM. So, we are we are helping our our developers our citizen developers to pick. So, for example, if there is no confidential information when it's mostly research, we use a lot Perplexity. I think they are very good, very efficient. But the minute that there is some information that are critical, we we are hosting like OpenAI on Azure and we have our own API keys inside the platform. So that's typical guidelines. Then you have all the architecture of your agents. So typically like how to set up like human in the loop, how to compare the different LLMs, uh all the kind of like the the analytics behind the AI agent. We are really kind of like giving that to the citizen developers, so they can really kind of like have the full experience of building a technology from A to Z. So how many employees are doing this and what percentage of your workforce does that represent? So as I said, we have 200 employees. Uh right now we have 60 active developers uh on the platform and we we will never reach 100% but we are targeting like this summer to have at least 50% of our workforce. We do it in batch because I have a small team, so I want to make sure we can kind of like support a big number. So we did a first batch, a second batch, a third batch. And the idea is to keep like a white glove service to those citizen developers, so they have everything they need and you know, when they ask a question, we we can be reactive and it's manageable on on my team as well. So is this pretty much every department in the company that's uh doing this? Uh-huh. We have people from HR to finance to we have an M&A practice to a consulting practice to a pricing practice. So it's really across the entire organization. Can you give some examples of agentic AI apps that have actually made it into production or are close to to that stage and they're finding some good use within the company? >> absolutely. Oh yeah, we have it's literally like all over the place. We have like some agent to create like market surveys. So, truly like what type of market survey, drafting the questions, really guiding the the analyst on like the best way to create those surveys. We have other agents more for like commercial teams, which is really like drafting SOW, like statement of Uh we have same thing in commercial. We have like agents to facilitate kind of roadshows. It's creating your emails, looking for everybody that is based in Dallas, Texas, and then give you recommendation on like who to invite in your next roadshow, for example. Then you have like like personally I have one which is my personal assistant. It's It has access to my calendar, to my email, to like all my kind of like background information. So, when someone is trying to, you know, get in contact with me or like scheduling a a meeting, I I just prompt the agent, and then the agent is finding the right moment, sending the invite, all of this kind of stuff. >> [snorts] >> Then we have um we have agents that are actually This one is really interesting built by one of the citizen developer, which is really kind of like a I don't know what's the right term, if it's a fake client or a cyber client. It's really like you are giving a lot of information about your clients, and then you send your document or the meeting kind of like agenda that you have, and this cyber client is criticizing and like giving you feedback, so you are much more prepared uh when you when you meet this person when when you meet a real person. >> [laughter] >> So, this one is good. We have another one that is We We have a big practice around consulting, and so we have one agent spotting kind of all the small mistakes, like like on on one slide you copy-paste it from another deck, and there is a name of another client, or you know, all these kind of like small details, and like page 54, you know, there you forgot the page numbers. All those kind of small details that nobody wants to do. The machine is taking care of it. Huh. Very useful. Yeah. Um yeah, I can we've all made those kinds of mistakes, I think. Um Absolutely. So, the the IT industry, which I've covered for a lot of years now, has tried all sorts of ways for years to get programming accessible to non-technical users. You could even say that just have the object-oriented programming is, but then there's visual design tools, business process management. And the promise was always to democratize programming, but I often have gotten the sense that maybe the non-technical users didn't really warm up to those tools. And uh lately the the terminology and the tools seem to be so-called uh low code and no code development tools. So, I guess in in light of that, my question is how well have Ducker Carlisle's non-technical business users taken to using these tools? And and you're right, David. I think that's something that I've seen like it it was a big push like I think for the past 10 15 years where low code no code was kind of trendy. But at the time, we didn't have like the generative AI piece of of the puzzle. So, it was really like you were trying to to drag and drop some like like boxes and behind those boxes, you had like manual codes. And it it was good. I mean, like like a wix.com, it was great to build the website, but like the minute that you wanted something complex, the machine was not capable of generating code. And I think that's really like the earthquake. It's really like when now you have an LLM that is capable of creating codes. So, that's like that's ponderable. Now, you can ask anyone that is non-technical to actually de- describe something and then you have an output that that is pretty good. So, in in in what we've done and and at the Curcurai, what we what we do it's I think we are not yet on like building a full like like a full ERP for our company. Right now, we are mostly covering like AI agent. So, like the LLM wrap. So, give information, call an LLM, and then you have an output. You can already do a lot. I think that's really step one. So, we are not trying to rush directly from business users to create kind of like a full enterprise software with SLA and all that kind of stuff and Kubernetes services. So, that's way too advanced. But but already if we can make kind of the business like the basic business users go to I have a cyber workforce. So, I have like an agent doing that, another agent doing that, an agent doing forecasting, an agent doing personal assistant, an agent doing research. That's for us where a P&O company. So, that's where most of the value is coming from. It's this capability of actually creating a cyber workforce that will, you know, support the human workforce. And I think that's really if we if we can really extract all of the value just from doing that, that's a massive massive uh opportunity for our organization. And then I think slowly the technology will can will continue to evolve and and continue to progress super fast. And then I think we will reach a point where we will be able to build a full, you know, like software end-to-end. Mhm. So, what kind of training did you offer to your employees so that they could get comfortable doing this? So, it's that's the beauty of the decentralized model. It's mostly so we we give them kind of the basics. So, it's like I think we have like a 2-hour like video track where we know we explain the concept of an AI agent, the different architecture, what is a human in the loop, you know, what is kind of like the different way of building AI agents. Um this kind of like I would say basic foundational like knowledge, and then we let them build. And I think that's a beauty of it. We We tell them like, "Go explore. Go check." And then in parallel of like their own experience, we do a lot a lot of sharing. So, like every week we ask some of the citizen developer to explain what they've done, what was great, what they are stuck. And then we share all of this information. And and we record everything. So, we are building kind of a massive database of like like bugs management, where they are stuck, kind of a Q&A big Q&A that the new batch they can benefit from. And they keep building. And on top of that, when we when we identify like areas where the the developers or the citizen developers are stuck, we when we debug them or we when we sometimes we have to do like manual code to help them because they want to do some something specific. So, what we are creating we are creating kind of a brick that they can reduce. So, we create like a almost like a microservice that they can, you know, we put that on the shelf and then everybody can go and and do that. So, so that's way the more people are joining, the more microservices or brick or tech brick that we build, the more use cases they can do and we keep kind of like expanding what's feasible. I'm always curious about the interaction and and relationship between IT departments and non-technical developers. So, I'm thinking of of course specifically in your company now. The days of a high priesthood have gone away. They don't go away anymore and come back with a solution and kind of foist it upon everyone. It's always been challenging for the people who do the programming to figure out what the business people need and and then cooperate with them and and get them something that they really can use and helps them in the business. So, first question related to this, I guess, is what is the division of labor? Who does which parts of the development process at your company? And I think that's really the value of this kind of like program because what we're saying is in the past and and and we started with the traditional mistakes like we try to centralize everything. We try to, you know, do workshops and and coming from Google, I was like, yeah, we need to find the moonshots and honestly, yeah, I mean, it looks great when you have again unlimited budget, but when you are on a budget, you need to actually look at the value and this vision of like moonshots and all that stuff, honestly, that's not working when you are a middle market company. What is working though, it's like let the user tell you by showing you. And and I'm again um you know, I'm I'm far from saying that everything is bad, you know, at Google, there is a lot of good stuff and I think this concept of really being user-centric is amazing because that's really what we've done. At some point, we stopped. We say, you know what? We try to do this moonshot. Honestly, we are kind of like, you know, ivory tower trying to do our stuff, but nobody is really kind of liking us and nobody is using what we are building because it's too complex, too new, too kind of like too soon. What we said, we said, okay, let the user show us. So, we really let the user like build their own stuff, even if it's very simple. We let them use it and then we we use our own company as a marketplace. And when we see a lot of users starting to use the AI agent that are created by their peers, that's when we know that, okay, we went from one user to 15, 20, 30 users. This is something that is very helpful. And we know So, now we pick this and we say this is very good. We know with data that this use case is critical and it's used by a lot of users. We will take it out of this like platform third-party platform and we'll build it directly using our own tech. Uh we're using Agno, so it's like really like coding it from from the A to Z, so we can really control everything and we you know, we manage the the back end, the front end, the infrastructure. So we completely own this kind of use case and we are creating an asset for the company. Now it's just not like it's not just a workflow or an AI agent in a third-party software that anyone can do. Now it's something that we extracted, rebuilt it, that we manage as a product. So we have like product roadmap, we have a bunch of features, we have a full monitoring, a full control, we have an SLA. So now we can really tell like now we own this use case and we are creating something very unique for the organization. Did you have to take any particular steps to encourage the business and IT side to work better together and get real results? Not really, honestly. I think where we where we got lucky or visionary, you never know. I would say lucky. Where we got lucky it's AI is so trendy now that we again, what we said we let the user do their stuff and then we use the power users as uh like shiny cases. So we we we had like roughly four or five people that were really excited about the program and they built a bunch of things and then we just kind of used them to really kind of explain, "Look, these guys are non-technical. Here is what they built. You are using those agents now every day. You could do the same thing." And then it's I mean it's it's like going to market and launching your startup. You have the early adopters that are, you know, like the the raving fans about your startup. And then, you know, it's very difficult at the beginning and then you reach a point where now it's mass market. Like everybody wants to be part of the program. They understand the value. They want to shine, etc., etc. So, that's really how you convince the business users to to work with your tech team. Before you came to Docker Carlyle, you've mentioned Google a few times. You worked at Google and freelance in uh data and AI roles, I think, that are sounded maybe similar to what you're doing now. I think um which experiences at Google helped you the most, do you think, in coming up with this citizen developer process? Yeah, it's I I think really what was very helpful and I think it's just the story that not many people understand. It's I think really understand generative AI was really the key. Because I believe that I don't have the right number, but I would say close to 90% of the people in the marketplace, they have no clue about generative AI. They they use like OpenAI and ChatGPT and they believe they are an AI expert. But really understand, like going back to 2017, realizing that actually the Google research team created the transformers. That's, you know, that is the birthplace of generative AI. And then, you know, the Titan infrastructure. When you understand those kind of things, that is really helpful to understand the impact that you have on on an organization. And I see and I see a lot of companies like like appointing like a guy who did like 25 years of analytics and like, "Okay, you are an AI guy now." Nothing wrong there. They're very good in analytics, but they're not AI. Or like, "You are a data science. Oh, you are very good in machine learning. Now you are generative AI and you need to put AI everywhere." But But they're they are machine learning experts. I think really understanding clearly what is behind generative AI was really key for us. And you know, the funny thing everybody's like, "Yeah, but like it's open AI who created generative AI." Absolutely not. So, that is for me, that was really kind of the the critical point. And I think again, this this decentralized model is is based on like everything that I've I've learned at Google or before when I was part of of another company that we sold, but that was really like the this this idea of the impact of the generative AI inside the organization, meaning I think there is no disconnection anymore between tech and IT and business. So, that's a fact now that you have access to almost like unlimited resources to build codes at scale super fast, means everybody is an IT person. For the good things and for the bad things. For the good thing, you can create like microservices, you can create softwares, you can create solutions, you can you can create website. Everything that you want to create is very easy. But on the bad thing, it's you have to manage all that stuff. Now, you cannot blame the IT anymore. You need to be responsible of, you know, what you do, how you do it, the architecture. When it's done, you have to solve that. So, I think this decentralized model really kind of like merge both worlds because now they cannot really blame they cannot really blame IT like, "Oh, it's an IT problem." No, I mean you build something that is bad by design. Like if it's not running, that's on you. Like don't blame another team. But now you can take benefit of something that is running super well. And you know, you can share that and you can be more efficient and it's working super well. So, that's kind of like how we tried to merge those two, you know, based on our our learnings at Google. So, how successful do you think the citizen developer program has been? Do you have actual quantifiable results that you can point to? But I would imagine there's also been anecdotal benefits and things that are more qualitative and and not and hard to quantify. So, what what are some of them? No, it's actually very easy to to quantify. Right now, we are on track to deliver a million dollar savings. And the way to track it is actually pretty simple and it's a good question, David. I I have this this question all the time. So, if you look at it, it's a pyramid. On the top, you have what we call the value on paper. So, every single citizen developer, when they build a use case, they need to to build a business case. We say like, no business case, no use case. That's step one. So, you need to start thinking about what the value you will deliver, how you calculate this value. And that's a value on paper. That's something that we track, that we that we share with the board. That's the value we want to track. And then, you have the realized value, which is really kind of all the KPI that you can get from technology. You have the number of users, you have the number of token, you have the number of runs. So, it's very easy to say, okay, you think you will save a 100k because instead of wasting 3 hours to do research, now you will do 5 minutes in with with AI. That's your assumption. You believe that you will run, you know, I don't know, a team of 50 people, they will do 50 runs per month or whatever, and that's you know, that's your equation and that's 200k. Now, we can really backtrack it and say, okay, this month you did five runs, you have only one person using it, you are completely off from your value on paper. You need to do something. Either you change your value on paper or you need to do more training, you need to onboard, you need to share more, you need to explain why your entire team needs to work around it. So, that way it's very like, we go in front of the board, we can really tell them we will we will we will overachieve a million because the value on paper we are like way above a million, we are close to 2 million now. And the realized value is very easy to say, well, based on the current usage of those tools, yes, we are like 800k, 950k. So, it's very easy for them to quantify the impact. And for us, indirectly. That's a good thing because I think a lot of IT efforts, you can't say that, you know, it's generally more um anecdotal and you know, it's hard maybe to show real hard savings, but sounds like you can really track them really well. >> but I think it's because again, it's we are asking technical people to do the work that a business person should do. Like, if you ask me like like the agent for the road show, like you ask me, I have no clue how to quantify the value. But I mean, it's it's not really my job and and I don't understand why he's doing that. But if you ask the business person, he's like, yeah, I can save like 1 hour here and I can get like much more leads and I can do that and I can and he can really quantify that because that's his job. So, now he's sharing that with me and and so, we know the the value on paper and and now I can tell him based on like the way we build things like, okay, like your assumption are pretty good. I I I I love it, but now I will tell you like how far are you from the value on paper? And that way we can work together not to argue on the value, but to say, you know, you don't have enough users or the way they are using your tool is not the way you design it or those kind of things. So, it's kind of like they are becoming the IT kind of like where I'm I'm like more like coaching them, but they they are responsible for the usage, for the cost, for like the, you know, the the the the fin ops aspect like you are burning way too much token because you didn't really design your AI agent properly and now you have a lot of usage, but you have a lot of cost that you didn't anticipate. Um how do you see this evolving in the next year or two? Is there some, I guess, risk that this could turn into just a short-term fix for the problems that you were trying to deal with in AI development, or do you see it, um, you know, continuing for a long time? And how might the the balance, uh, change, perhaps, between the developers, uh, the citizen developers and the IT folks going forward? I I hope it will continue long-term. That's my plan. And I think if if I did a good job, the business user will see the benefits. They will continue to come. They will the beauty of decentralized it's every time you have a new LLM or you have new capabilities like a cloud, like cloud PPT skills is a perfect example. Like for a very long time everybody was asking like, we need a tool doing PPT. Okay, like but they like gamma.ai that it was not working well and we tried a bunch of things was not working. So we said, you know what, there is not much we can do. And so we said, you know, decentralized AI do your own things. There is nothing we can do centrally. And then they release, you know, cloud skills and then all of a sudden now those citizens developer they are like, oh boy, like yeah, we can actually do PPT on our own and it's working. It's not it's far from being perfect, but it's it's okay, actually. So now it's like, okay, interesting. And we've seen a huge amount of people using like, uh, cloud skills PPT PPTX. So we said, okay, we will, you know, take this capability and build that inside our like central platform. And I think we will continue to do that again and again. Like we will have new capabilities, new features that will be released. Our our decentralized model will help us test those things to see what is I'm picking this one because this one is is probably the outliers, but 99% of the time they are all excited about a feature that they've seen that they've seen on LinkedIn. They try it and they realize is completely useless because it's sold by, you know, like a young kid with no experience and just doing a prototype to sell her a fancy uh um course of something. But that's that's really cool because we don't waste time. We scan of the noise. We really let the the the the community like absorb those news, like use it, try it, and when it's really working, like they share that with us and we we scale it. So, we stop wasting IT IT time to chase kind of like unicorns. Now we're chasing scale. Chase the unicorn on a on your own. Have fun. When you found something that is working, you come to us and we will help you really scale that inside the organization. So, what's your advice for other organizations who are considering maybe a citizen developer approach to managing their IT strategy? Are there some things they should watch out for and be aware of or are there some maybe uh secrets to success that you have in mind that you think are key in how you've done it? You know, I think the mistake that most of the companies that I discussed with are doing it, they think it's a tooling thing. They think like we just need to pick a platform and magically we will scale AI. And and I'm I'm sure, David, you've seen that in IT because that was a common pitfalls like "Hey, we have a problem with sales. Oh, just install a CRM. Oh, we have a problem with production. Just change your ERP." Yeah, come on. Like it's never a tooling problem. Almost never. So, I think for me it's more like if you want to scale AI, one you need to to have at least someone with the right expertise to kind of drive the programs. You know, hire a chief AI officer or find someone that is really good. Again, not someone who read like 15 posts on like Medium and believe he's an AI expert, but someone who's who has spent some time really building um like AI solutions. And then then ask this person what tool should we use and how should we frame our AI strategy. I think for me it start with the people because at the end of the day the people are using the solutions. It's not just a tool that it will magically being used and scale inside your organization. Well, that sounds like a good advice. Fabian I thank you again for coming on the podcast. Yeah, well thanks again for having me. Thank you for listening [music] to Enterprise Apps Unpacked where we give the real story on the software that [music] runs your business. To learn more about business applications, explore our content on [music] searcherp.com. Find us on YouTube at our channel Ion Tech. [music] Subscribe to our podcast to receive the latest episodes as they drop. [music] And if you liked what you heard today, give us a rating and review on [music] Apple, Spotify, or wherever you're listening. Until next time. Podcast programming and promotion by Kelsey [music] Odell. Music supplied by DP via Getty Images. >> [music] [music]