Developing open-source software: lessons, benefits, and challenges - Nils Reimers

foreign coming to to the final Point developing open source models which is also my Computing slide um as you know I've been developing a lot of Open Source Frameworks all of the time and what I can recommend is to focus on the user experience so what does your user want to build with your framework sadly you see a lot of Open Source Frameworks that are not built for the user but maybe just built for the prison itself or built for an experienced user or just build to reproduce some numbers in a paper um and a test I put out is like really okay I want your framework so simple that you can explain me the framework when we're drunk at a bar with just penetrate with a pen and paper on a bmat so so can I just bump into you at a bar and ask you hey how does this framework works and that's like my guiding principle so for example easy nmt it's a framework to do machine translation which I developed I don't know a few years ago to simplify machine translation it's kind of easy so first you run pip install Easy nmt then you do uh imported load the model and you call it translate and say what's the language and that's pretty much all the code you need and now you can start to machine translation and do what the user wants to translate text from one language to another language also you should heavily focus on documentation do you think that's what successful open source project sets apart from unsuccessful open source projects really focus on documentation documentation is way more critical than the code itself and the code architecture software architecture beneath the library so it's a really invest heavily on the documentation and here it's not easy so start with an example like this so here take a sentence translate it to German and then over time you can build up complexity say okay this is also what you can do and this is like different sentence betting and language identification and so on and so on and so on but really start easy finally if you want that other people contribute to your open source Library um aim for like simple code or software architecture um really simple for people outside people to understand the architecture to contribute to your code so don't use all the fancy Technologies from object-oriented programming with a lot of abstractions and decorator and spread your code across 1000 classes cells I don't have like a really fancy complex software architecture but I found it really good if the software architecture is easy you can just look at the code see how does it work even if you're just a beginner in the world of programming yeah that's so much for my search um thanks for listening and looking forward to your questions and also questions from Jay amazing thank you so much Niels I'd love to remind everybody who was either watching this on YouTube or live uh we have the thread in the Discord that's the right place for questions uh and we can you know answer them as they come in and and after the session as well so I'd love to direct you there um yeah thank you so much Niels that was a great sort of run over you know a lot of the different topics um that we uh on this late late last slide I'm really curious to hear more from you about your experience sort of running sentence pert so it's this research project and it's code that you've uh launched but then it sort of it succeeded and it took a life of its own I'm curious to hear from you sort of you know what are the benefits what are the insights that you get as a let's say a maintainer of a popular open source uh package like sentence Transformers but also what are the challenges yeah um so yeah first I would say it's that the sentence from formal library is a bit different than to a lot of libraries you see people put publish with the code so often what you see was a lot of Frameworks is um that they published the code just with the aim to reproduce the experiments in the paper to say okay this is the table we have and here's like the few lines of code you can run to reproduce the numbers but people do not really care about the numbers so they do not really care about scores on this one Benchmark like blue Benchmark what they care about like okay I want to take this and build my own thing and here sometimes Transformer library is different you can't take the sentence from formal library to reproduce the results from the sentence part there are like some models available where you can reproduce the numbers but because it's an actively maintained Library always up to date also up to date with the new side effects Transformer Library there's been like a lot of changes and you cannot really reproduce the results but still it's focused on the user experience so so people are not really interested to take this and reproduce this one number in this one table but they are interested to run it on like topic modeling to see what other topics in the document um so so benefit here is like if you publish the code there's like not really a way how you can betray so so totally with this jelly beans you can just run a lot of tests a lot of ablation hyper parameter tuning and then you find like aesthetic which works well but if you put out the model so that other people should use it it has to be provide value to them so if your model just works for this one setting and will not work for anyone else and provides no value for them so for other people and people will not use it so my goal was always not to publish a paper and get like this build line and claim this is a cool new model if I really put out models that are useful and that are easy to use for for other people's second inside I get is like a lot of sense from what other challenges people have in the field so number one question I got a long time is like multilingual embeddings How can I get embeddings for other languages than English where at that like two years ago I said okay there's like no good way how you can get sentence embeddings for other languages and there are no good models available and methods available so I started research to do this um softness was like a paper presented two years ago at the NLP on sentence level not paragraph level second question was like out of the main generalization like people say hey my system doesn't work well for legal domain or fashion industry um what can I do and so that's where I started the gear benchmarks say Okay first I wanted to have members on it and then investigate ways how you can adapt these models to other domains um challenges is yeah it's time consuming so so if you put it out and you interact with the community and answer questions and pull requests and try to troubleshoot things it takes away time you could spend on like research creating new papers building new things so there you have to find like the right balance for that