Luciano Resende - What's new on Elyra - A set of AI centric JupyterLab extensions | JupyterCon 2020

Key Takeaways

hello it's really a pleasure to be here again in another jupiter con talking about some of the projects that i have been contributing around the jupiter ecosystem today i'm gonna be talking about elira and some of the new functionalities that we have been uh adding into that more recently my name again is luciano hazenji i also have carla spoldaro with me we work at ibm at the code which stands for center of open source data and ai technologies uh we pretty much do 100 uh contributions to open source projects directly into their community so what is lyra elera is a set of ai centric extensions for jupiter lab it's an open source projects that we have announced around april 29th uh a lot of people ask me about the name elira name is kind of like a word play uh with one of the jupiter moons we had a y there so instead of alara it's now lyra uh similar to what we have in jupiter let me go and move to the live demo that is gonna help you guys understand a lot more of what's going on what it is a liar and how it can help you build your analytics and ai models you should already be familiar with the jupiter lab interface and you will notice that after you install lram there is going to be a new section in the workspace uh where some of the editors are being available and you're also going to start seeing a few new additions on the left side panel with related capabilities let's take a quick tour around the lyra focusing on some of the new enhancements introduced recently one of the most popular features we have is the pipeline editor the pipeline editor simplifies you the conversion of multiple notebooks into batch jobs or workflows you can start adding multiple notebooks and we recently just announced the ability to also include python scripts you can then start defining the dependencies between them which will influence the execution order and you can add additional properties to each of the nodes where you specify runtime image which is a docker image use it as an execution environment we have predefined docker images but you also can specify your own [Music] you can define file dependencies environment variables and also output files note that the output files are then going to be propagated to subsequent operations on the flow uh if you have been using the library in the past you've seen that we have uh enabled the execution of these pipelines in kubeflow pipelines uh we now added a panel a left side panel here where you can see a list of runtimes that you have available that gives you quick access to kind of like access the pipelines and experiments on that ui and also look into associated files in the object storage we also recently announced the ability to run these pipelines locally and what is happening here is uh we're just calculating kind of like the the right order and executing those pipelines in your local environment if you then uh look here you will see it in the console uh kind of like the jupiter lab blogs uh all of the execution that is happening and then coming back uh once that execution is finalized uh we get a notification and we can kind of like just go ahead and open those notebooks and you will see the executed results all there you can also just go to where those files are and see generated files for example regarding the runtimes when you add that you can see that now we have additional files for username and password that that is to enable you connecting to uh tubeflow pipelines that are secured those are sort of like a common environment when you are on enterprise or when you are execute that kind of like a in production environment [Music] let's take a look at the lyra support for python scripts alarm introduces the ability to create python scripts directly from the workspace launcher but for this demo let me clone the github repository that i already have some python scripts file there okay let's take a look here this is a repository that i just cloned alera allow users to locally edit their scripts and execute them against local or cloud-based resources seamlessly you can see that now i can select a environment that i need for example if i'm doing model training i need tensorflow with gpus or if i'm doing some analytics and i need pi spark but for the case of this demo i just need python a simple python environment i'm just doing some pandas analytics here and calculating kind of like the the sales price the mean sales price for a given city i can then go and execute those files and you will see that the bottom uh on the python console output all the results are going to be available for you here independent if this has been executed locally or for example remotely using cloud resources eliara also recently announced a code snippet extensions that allow users to add custom pieces of code that can be then reused in making programming in jupiter lab more efficient and reducing repetitive work let's take a quick look on how to use this let me create a new notebook i have here some pre-configured code snippets in this case let me just add here the cell that check the version of tensorflow that is being used i see oh this is the right version now let me start putting some uh my graphing things and requirements that i need to start building and using matplotlib so i have all the imports here and i have a template for kind of like a graph i can basically easily import those and start executing you see that out of the box out of the back the box i then have kind of like a working notebook with all that i need and then i can start customizing so that it applies to the use case that i'm trying to resolve here this gives you a quick overview on what's new around the lyra there are a lot more of the functionality that has been uh available for a little while uh we have a several other talks around jupitercon that it's interests of you and we'll demonstrate different areas and different aspects of lyra please let me pass to carla which is going to give you a lot more details on those thank you very much we have put together some helpful links for you to get started with the lyra today go to ibm.biz slash a lyra dash demo to run a lara from binder no need to install anything it's just a nice and easy way to test drive it you can also install a lyra on your machine just by following this link here are all the prerequisites you need all the installation steps you can also run elara from docker and from here you can have access to the complete elira documentation we have tutorials user guide how you can set up your deployments you can read about the developer guide and so on there's also a link to a lyra's github page so either if you are a user a data scientist or a developer wanting to contribute to this project your feedback is always appreciated there are many ways to get involved contribute to the projects and become part of the open source community please give us a start on github fork the project if you want to help out on the development submit bug reports or suggest improvements by creating issues or pull requests on github you can also help on code reviews test them give your feedback and elara also has community meetings where you can stay up to date with the project get to know the people involved feel free to share your comments live with the team uh our meetings happen once a week every thursday 9 00 am pacific time there is a link to the web bags room at the bottom of the read me eliara's github you're more than welcome to join us and we're also on getter if you prefer to chat with us also if you're interested in learning about elira in more depth here are other jupiter cone talks hosted by the ibm coding team including some talks about jupiter lab extensions which is pretty cool so that's it from us we hope you get inspired by eliara today