Creating and Manipulating PDFs in Python With borb | Real Python Podcast #84

Key Takeaways

welcome to the real python podcast this is episode 84. have you wanted to generate pdfs from your python project many of the current libraries require designing the document down at the pixel level would you be interested in a tool that lets you specify the page layout while it handles the specific details of laying out the text this week on the show we talked with joris kellykins about his library for creating and manipulating pdfs named borb borb is a pure python library that can read write and manipulate pdfs you can use it to build fillable forms invoices with attached data files and multiple column document layouts we discussed the extensive example repository yours has created for the library yora shares his background in working with pdfs he talks about starting the project and the challenges he had to overcome we also talk about licensing and maintaining an open source library this episode is brought to you by cloudsmith cloudsmith is a secure software supply chain management tool for your python packages and dependencies try cloudsmith for free at cloudsmith.com sign up alright let's get started [Music] the real python podcast is a weekly conversation about using python in the real world my name is christopher bailey your host each week we feature interviews with experts in the community and discussions about the topics articles and courses found at realpython.com after the podcast join us and learn real-world python skills with a community of experts at realpython.com hi yours welcome to the show thank you for having me it's so funny because we mentioned the library back in episode 76 david and i were talking about projects and orb came up and was really having fun kind of playing around in the library and it's kind of one of these things i keep talking about pdf libraries pretty commonly because i was doing a lot of work for small businesses in creating forums and creating other kind of tools for them and so it's always been kind of one of these like hybrid things a tool that i use very often in python so i'm very excited to have you on the show thank you yeah i i was completely surprised last time when borb came up because i only saw it because somebody linked it to me and i was like wow it's happening somebody actually published content that wasn't me yeah yeah it's cool and it got featured in uh pie coders a couple times too which is great i'm guessing those are based on the articles that you've been kind of writing on about the library yes so i've been trying to get one article out every week but i see that it's quite a journey to not just get the content out that's actually the easy part but remembering to link it on all my social media accounts and every newsletter that i come across etc i i really hope as soon as we turn into an actual full-fledged company i'm hiring somebody to do that for me that's yeah so maybe we could talk about a little bit about the history of the project certainly a few years ago i i don't recall how long ago actually i was working at another software company that builds pdf software pretty big name in the pdf landscape so they are called itext and they mostly build software for java and net okay and i'm still close friends to the founder he lives two streets away from me we talk quite often uh we have a similar interest in movies so we see each other quite often in the local supermarket or the local movie theater but then the company changed as companies do the company was brought over by a korean holding and a lot of the developers there including myself started looking for other pathways in their career and i actually didn't think about pdf that much anymore until a few years ago when i was working in python which was a first for me and i had to write some code to generate a pdf document as the output of a test framework that i had built okay so i just wanted to do some basic pdf manipulation create a table automatically switch to a next page if needed etc and to the best of my abilities i couldn't find a pdf library that actually was as good in python as i text is in java most libraries will force you to go down to the nitty gritty level of telling you like you need to specify exactly at what pixels you would like this piece of text to be report lab would be an example of that and i've had somebody on talking about it and it's very very very intricate and it almost feels like you're doing yes i don't know a form of pixel perfect layout which is really painful okay it's it's great that you can do that and i can understand that there is a need for that that there are people who would like it to be exactly pixel perfect down to the very last dot and that's great but i don't have that amount of time sure it's funny that that there's a this development of pdf software in in belgium that's pretty cool yeah so i developed a library as one does when one is confronted with a problem so i i decided to hell with it i i know how pdf works at least that's what i thought and i'll write a piece of software that actually does what i wanted to do and that's that's how this library started so from there on out of course a lot of the ideas came forth not just from my time with itecs but also having talked about pdf with other international companies like adobe sort of having a finger on the pulse with with regards to where is pdf going in the future what features may be unimportant now but might become very important later on you know things like that being able to design upfront what the library needs to be able to do and that's i think one of the great advantages because most companies like itext very good software but if a company exists for 15 years you start seeing it in the code base there are things that you can no longer refactor because a lot of people depend on it or because the software has grown organically in a certain way so it's it's certainly an advantage to be able to start afresh but with the ideas and experience of all those years that's interesting to think of these larger projects sort of calcifying sections of the code that you can't then restructure it sounds like you have a pretty thorough background in postscript then in order to make a library like that is is that the case um not really okay so mostly my development my knowledge of pdfs comes from having worked at itext and afterwards having worked independently for small other projects with pdf okay but i do know what a pdf looks like on a binary level if there is one thing i have to credit itecs for it's the ability to actually invest in people's education so by the time i was there for a few months i was able to write a pdf file in a text editor and have it actually be a valid pdf okay cool like i noticed you go into that a little bit in your documentation you're talking about the or actually i guess it's one of the tutorials i'm trying to remember which one where you're showing the binary well the text output of it with the eof and at the end of the file and and then that it was sort of built backwards i thought that was fascinating yeah i think a lot of things in pdf are fascinating from a viewer point of view and from a developer point of view are just an absolute nightmare building a file like that upside down is is difficult then that's not even the most difficult part for instance having to insert a signature in a pdf is its own special layer of hell because you have to be able to tell the signature up front like you're going to calculate the hash value of these bytes in the file but then the signature also needs to be part of the file and you don't know up front how big the signature is going to be so that requires a lot of insanity to be able to get that working huh so something i had experience with actually i was creating these forms as far as an end user and you would you know it's like these daily reports that you know there's an environmental science company so they're out in the field and getting dirty and they were having a lot of forms that were like sort of half filled out as far as like paper you know with a clipboard and so forth and then they would come home and they'd scan them and you know get the scanner all filthy and so forth um so they decided uh maybe we should go with like ipads and so we had to decide on a type of software that could could read them and there's a universe of that of course and so i wonder if that's something that compatibility wise you go oh it's just a pdf and it's like well there's lots of ways of handling these things and signatures was a nice thing that like on a phone or a ipad that you typically could do then with your finger you know maybe the style of like a checkout kiosk or whatever paying for a credit card so that that workflow of like signing with your fingertip has become somewhat noticed as far as done but then like taking that actual signature i guess bitmap or whatever and then like putting it into the pdf that sounds like that's a bit of a challenge then as far as storing it yeah i think one of the main disadvantages of pdf or one of the main challenges of pdf is that it's actually not just one type of document it's an umbrella term for a whole collection of standards each of which derives from the same base standard iso 32 000 but each of them also tries to improve upon what the other ones got wrong so you have a standard that just defines what the base pdf looks like and then afterwards another standard that says well you know we got this part wrong actually you should do it this way i think we're now at version two of the standard which finally allows for other levels of encryption allows for algorithms that are no longer deprecated so good things are still happening in pdf land but it's always a bit of a nightmare are there ways to uh your the library which we should talk about a lot more as far as like the features of it but one of the things it does is is read yes pdfs and extract information from that is there as far as i know globally this might be difficult to determine but in your experience maybe through support tickets or what have you that you're seeing come in is there some standard level that seems to be standard like people are using or are you seeing pdfs like from the beginning of the dawn of pdfs uh coming in and those versions being difficult to read it's it's actually a lot of different software producers like for instance microsoft word allows you to export to pdf but then there are low level linux commands that also produce pdf documents which like you said are more closer to the dawn of time okay in general actually it's not the most it's more likely to be the most recent producers of pdf documents that screw things over because only more recently have features been added that make it much more difficult to extract text from a pdf document and i think it's similar to for instance having a standard like ascii where you have only that many byte patterns that represent the full alphabet that you needed at one point in time and then afterwards people realized oh wait there are tons of other languages that we need to support and we need to somehow hack this into a system that still works and pdf suffers from the same diseases so you have a very old kind of base standard and if you only ever want to extract western text and western alphabets that works perfectly and most libraries get that right to a very high degree but then you start looking into things like oh well i have the letter a with this insane accent on it right how is that going to be represented how is the font that has to be embedded in the file going to represent that and suddenly it's a lot more challenging to be able to do that yeah i can imagine the specifications of unicode uh sort of developing alongside these changes in the pdf architecture and then this i don't know even what to describe it as but this explosion of emoji use maybe being a problematic too like just as far as like i'm guessing most ocr is just like to say well i don't know what that is it's just a blob exactly now the fun part is and i've seen that being used and i was this is devilish but creative was my first response i've seen somebody create pdfs where the actual text that is being embedded in the document is just a repeat of their company name over and over again and that's one of the things that pdf allows you to do it allows you to create your own characters so that you can say okay it's going to be drawn as if it is this but when you extract it as unicode you should do this and i've seen people use that as a sort of drm protection which is beautiful and sad at the same time yeah definitely well let's dive into the library a little bit sure i mentioned briefly that it can be used for reading of pdfs but where it seems to be most powerful in the tutorials that you've been doing is the the creation of pdfs so maybe we could dive into some of the certainly just like a survey of the features because there's tons of them yeah so my main goal for bob is to have a library that makes it as easy to generate documents as you would with microsoft word and that's a very very tall order because people are very used to working with that particular program you're used to being able to add text and that it automatically travels to the next line or that it even automatically hyphenates text and all of that is non-trivial to do when you're talking about pdf so bob like the libraries that we've mentioned before like report lab it allows you to precisely say i would like content at this particular location because sometimes you have perfectionists but then it also has a sort of layout engine built into it where you can just say i would like this page to be laid out in two columns or in this way or whatever and then you can just say add this text please in this font size in this style or whatever and it will automatically take into account leading and margin and padding and whatever and the document will just look as you would expect a word document to look in the case of that the kind of thing makes me think of two questions or actually one comment and a question the first is the the idea that it's a little bit like in a gui per se where you are defining sort of column type structure and then the window can be adjusted in size and and kind of go around it so that you don't have to constantly worry about the pixels i guess that's almost sort of a css type of thing too as far as web design you know kind of this sort of general layout kind of stuff which i think is really cool and and needed i used a program from apple called pages and i was very familiar with the kind of general idea of it had a page layout mode where you could create these you know basically squares you know or rectangles on the screen and say okay this you know is where this information in and then the feature that i enjoyed about it was for like a magazine or what have you you could say the text should flow from this container into this next one you actually had like little arrows to draw where that would happen is that something that's implemented in orb like as far as like going across pages with this sort of column type layout well it's not implemented to that generic of a degree so you as the user cannot say here is a collection of rectangles that span a page and i would like them to be respected in that order what you can do is use one of the existing layouts for instance multi-column layout which divides the page evenly into columns there's also a single column layout and some other ones that i'm probably forgetting but it's a great idea and i'll see about adding it as a feature it's something that i was really impressed with in like you're saying like that kind of talking about the idea of somewhat abstracting some of the the work involved in the layout so that's really cool the other question i have that's kind of related to that is you're in laying these things out you are creating you're kind of doing this extra work behind the scenes and i'm guessing that this is you know kind of the python stuff behind the scenes of it deciding how to break words like you said leading and other things how much kind of work went into doing that that is sometimes you come across a challenge where you think oh this is going to be trivial and then it turns out that it's actually an entire field of research and and that we as humanity don't know yet how to best tackle this and then sometimes for instance with hyphenation i went into this thinking this is going to be one of those days because how the hell do you teach a computer to hyphenate a word correctly in any language that you want to support right and then it turns out that there's a beautiful algorithm that most software most commercial software at least has been using since whenever the algorithm came out and that it's it boils down to a simple lookup of certain patterns within a word and that it marks hyphenation locations based on that so i was very lucky to be able to find that i was lucky that the license matched with the license of bob so that's the algorithm that's currently being used and it's the same algorithm you will find in other libraries nice yes i'm guessing that was one of those things like you maybe started down the rabbit hole in digging and then sort of paused for half a second and said wait maybe i should look around well my first instinct is always to look around okay good i love learning about new stuff and i'm not naive enough to think that i'm the first person to have to tackle a certain problem so there's a good balance between doing it yourself and making use of somebody else's work and i think with bob i'm being on the side of cautious i think there's only like four or five dependencies so i'm not trying to import the entire python ecosystem but i'm also not insane enough to write font handling if somebody else has written a beautiful library that handles font files yeah how are fonts handled inside of bourbon fonts are handled to the level of reading the files and determining the attributes that a pdf needs from a font is done using the font tools library okay and then for certain built-in fonts well to clarify pdf defines 16 or something fonts as being built in so any reader or writer should support them and for those fonts i simply looked up the font files and extracted its attributes manually and put them in there as a json file so that it would be easier and faster to look up [Music] security within software supply chains has become the major focus for developer and engineering teams cloudsmith is a software supply chain management tool that provides public and private python repository hosting for ultra fast and secure delivery of your python packages cloudsmith is a fully compatible pi i like repository with cloudsmith you have the ability to develop your python packages internally and privately share them with other teams across your organization to get started with your own private python repository visit cloudsmith.com sign up for more information [Music] yeah you're using json pretty extensively across the library what was your decision for doing that i would like borp to be as close to open source and free and accessible for everyone and the idea of using a format that only i can read is doesn't appeal to me me as a developer and as a debugger if i'm outputting a file somewhere i would like it to be legible for me so that i know okay something went wrong or i need to change this or this is what the system is currently doing so it's convenient for me as a developer and and it really ties into my ideology as having made this project i want it to be open source cool one of the things that i found interesting and i commented on in the podcast episode earlier was that you have a feature of embedding files within the pdf and in the in that case it was like a an example where the data that was inside the tables of i think it was an invoice yes could be included as a an attachment of json with it and so that if you were sending this file you would have the pdf to look at and to print and what have you but instead of having to go through like an extraction process or some other kind of thing sort of attached to the file was the json ready to go i haven't seen that often that in use and so i was wondering is is that something that you experienced in use and or how do you see people using it i guess there are kind of two questions there well i think for most of the things related to pdf spec and standard there will be very large differences in terms of what certain countries use yeah so for instance embedding files is something that i haven't seen much in the us market yeah but in germany for instance there is an invoicing standard called sucfat which actually tells you that you have to do this you have to embed an xml file that complies to a certain schema and that specifies what invoice items there are in this invoice what the quantities are the price etc so depending on where you look you will find either one or the other okay yeah i have to get the name from you and um because i'd like to learn a little more about that because i think that i mean that's really powerful i mean yes we're passing around these fairly universal reading and printable documents but you know having the data with it is like or in our case in the u.s centric kind of thing not having the data with it it seems kind of like a waste you know yeah i mean the pdf is more than likely to have been generated by some automated system anyway so we know the data and we have the data in a database telling you exactly what the item is how much it cost etc so it makes sense because it's not an extra effort it makes sense to embed it machine readable anyway yeah definitely we mentioned a couple of them already but what were other crucial features that you felt you wanted to have included inside of borb well every time i include some kind of crucial feature it's mostly because somebody i know happens to be using it and then nags me for it so okay like i said it just started out with me being able to produce a report and i was quite happy once that report rolled out and then somebody said yeah but i'd like to embed images because you know our company logo and so on oh god no why would you why would you do that to me and then afterwards borb started supporting more than just the basic fonts which was also a big one and then another big one but not in terms of features is actually having set up a testing strategy that checks all of the documents that are produced during tests for visual imperfections that was that was a big game changer for me because before then every release i would run the test suite and i would have to go over each file and check is everything still okay and now i'm able to do that and just have it checked automatically what would be examples of that like things overlapping things hitting the edges of like where it can be printed yeah things things overlapping or things suddenly switching to the next page because the layout algorithm thinks there isn't enough room anymore or in the case of hyphenation you know getting the hyphenation wrong and skipping to the next line already or with fonts getting the width of a certain character wrong and suddenly the line shifts a few pixels ultimately i was surprised by how much test coverage i can do just by checking whether it is still visually equal to the golden sample and what's involved in doing this sort of compute i'm guessing it's not like you know like computer vision or what have you but like how is it doing uh this this comparison well first i have the golden samples which define what i want exactly okay and then i convert both the output that was produced on the latest test run using ghost script it has some insane command line flags that allow you to output a png image i do that both for golden sample and my output and then i compare those pixel wise to check whether they are pixel equal okay i do have some special markers in there because for every test i also print the date of the test on the test itself so i've set it to ignore one specific color uh so that i can print the date in that color and be assured that it's not going to count that as wrong wow so it's like kind of a mix of computer testing and then some actual output testing yeah some of the documents some of the test output at least is just produced because i think it's a fun document i mean at the office i'm mostly known as that guy who produces pdf documents so yeah but it's it's fun like during kovitz we've all been at the office part of the time and and not part the other time so it's fun to leave a sudoku puzzle in the break room so that somebody has something to do when they are sitting there eating their sandwiches alone right so yeah some of my tests produce sudoku puzzles or other stuff that you can just have fun with and they are tests but they're also just documents that i like yeah so along with the the test suite and and the primary github repository you decided to kind of break things apart and create an examples uh repository yes why did you choose to do that i think the examples repository was getting too big okay one of the comments that i received early on already was that the github repository for board was getting out of hand in terms of size and i noticed that a lot of my code was actually just producing examples and trying to change one parameter about how to add text and then this is how you add text in bold or in italic or in this particular color so rather than having very extensive examples that also double as tests my tests are now much more compacted and testing various features at the same time and the examples go in detail about you know if you want to influence this particular aspect you should change this parameter yeah the and more than that it's like like a very detailed set of documentation i was uh very impressed by the kind of granular detail have you had experience in writing documentation for library before no not yet but i was originally planning to release a book a technical manual on borb and then i've talked to various publishers and the original content which is now the examples repository was refused because it was too technical and did not tell a cohesive story so most publishing houses would like me to publish something like you know we're going to focus on i don't know creating an invoice in order to do that you need to be able to create a document with an image because you want to have a company letterhead etc so everything needs to flow from this example yeah whereas the repository that i currently have like i said it's much more of a listing of features and things you can do with it so i asked for permission uh with the publisher that i'm currently working with and they said well go ahead and publish it so that became the technical manual and along with creating all of that you've been writing a set of articles which is again how we kind of discovered the library uh you know with pi coders how many articles have you done now i think there's about 10 articles on stack abuse and some of them are still waiting to be published okay i try to get a head start and write articles well in advance so that they can still be reviewed and that i can have a release schedule of you know one article every week coming out yeah i guess this might come out in time for some of those other ones i've seen six of them i think up there on stack abuse currently yeah i'd have to check yeah that's the wonderful thing about writing them up front after a while i get complacent and i no longer check whether there are still any articles yeah do you have experience in writing articles before this yes sort of so for companies that i've worked for before i have extensive experience as an international speaker and oftentimes there is a certain please write what you are going to talk about stage yeah so it's it's the same kind of skills you have to write a compelling little story that you can bring to an audience in just a few minutes and i found that that skill translates really well to being able to write examples yeah to take a step back to you said that you're i guess i'm got a little confused in the back and forth of you are creating a book for board yes okay so there's currently currently i'm working with manning to write a book on boarb but it's a whole process and even though i have a lot of the content already in my head and written down it's going to take at least until next year according to their schedule before we are able to release so that's a very long term kind of plan yeah this is kind of just a side note i love the pictures of the birds um are those ones that you've done or who's done that artwork uh no those are those are artwork that isn't done by me um i actually have to check that i think the license is compatible with being able to publish them in educational material but i just i just googled the words boob and i found these wonderful drawings yeah like you said they they were very captivating and i was like oh i need to i need to have them and i need i need to make sure that there are enough for each chapter you know yeah they're like these chunky little birds everything exactly a couple kind of general questions um that i was wondering about uh with the release of python 310 have you been going through getting bored ready for it i've not yet uh had the opportunity to do that i should really look into that okay i think one of the major ones is pattern matching right so yeah yeah i i think it might offer some opportunities in the low-level parsing of pdf documents but you know that part has been optimized so heavily already i doubt that it would gain very much but we'll see [Music] this week i wanna shine a spotlight on another real python video course the course is based on a topic we briefly touch on during our discussion this week and it asks the question what does it mean to write python code in the way that it was intended or as many call it pythonic code it's titled writing idiomatic python the course is based on an older video course by modi youssef and instructor martin royce has updated the course for modern python and he takes you through how to access and interpret the zen of python how to initially set up a python script how to test truth values how to take advantage of built-in functions and methods how to swap variables in place what is the drive principle and how to create pythonic for loops it's a short course and i think you'll find it's a worthy investment of your time it's a great overview for people coming from another language as well as an introduction for beginners to the idiomatic practices within python and like all the video courses on real python the course is broken into easily consumable sections plus you get additional resources and code examples for techniques shown all of our course lessons have transcripts and include closed captions check out the video course you can find a link in the show notes or you can find it using the enhanced search tool on realpython.com [Music] i kind of wanted to talk about the license a little bit sure it sounded a little unique to me when i read it initially and it kind of gets into that whole open source world of like okay how do you choose what license you want and then in your case you're you're looking at building orb it sounds like into a business um am i getting those things right yes so actually first step is already there so i am according to belgian law at least a business okay that was a very scary step for me okay my my mentor in that regard bruno who is also the founder of itext has always said you know as long as you have no money there are no worries as soon as you start making money you have to find a way to you know pay taxes pay people and whatever so it was all fun and games as long as i wasn't a company but now you know it's a company in terms of licensing it's currently licensed as a dual licensed project so either you abide by the agpl version 3 or you buy a commercial license and okay the idea behind that is that you have to support the open source community that is actually what i am forcing you to do at least that's the reason why i picked those licenses so either you support the open source community by abiding by the agpl and you make your project open source that's great more code in the public domain people not having to solve the same problem over and over again that's one option the other option if you can't do that if you are working in some kind of industry where it's frowned upon to share code then by all means buy a license and you are at least supporting my open source company right in the further development of the library there yeah so either way you are contributing to some open source company can you explain a little bit about the agpl what are some of the distinctions there just to kind of clarify it for people a little bit so as a sort of disclaimer i'm not a lawyer sure yeah anything i say is my my opinion of it rather than my legal expertise yeah so to me the main so you have sort of a collection of permissive licenses where the most famous one is perhaps the mit license which essentially says you can have this code but i'm not taking any of the responsibility for it if it burns your computer or your company down that's on you then you come into sort of more professional licenses and and the whole debate versus copyright and copy left where you have the gpl family of licenses okay these are typically copy left licenses which force you to be open source or else okay and that's also the one that i used for bob where this or else is the buy a commercial license clause and then the main difference between agpl and gpl is actually whether or not you can use the product as a service or not whether or not that counts as using the product now the reason behind that and actually the reason behind the existence of agpl in the first place is oddly enough also tied to pdf and to google so at one point when google was expanding they built google docs yeah great success uh suddenly we could all use google docs online and whatever and they also had an export to pdf functionality and they used in their earlier versions a version of itext which was gpl licensed and google claimed well you know our users are using it as a service they have no access to the code they are not actually purchasing a product they are using it as a service and this is not covered by this license so we are in the clear and it is as a response to these big internet giants using people's code to build their software so quickly and so pervasively that the open source movement started with including this whole as a service thing into the licenses and that's where the agpl comes into play yeah it totally makes sense to me that i it's been an ongoing conversation on the podcast about just generally like okay you want to do an open source project and what does that mean and the licensing of it but then also just everything from getting contributors which we might talk a little bit more about too but also the the funding of it and i had um some people from tide lift on to talk about what they're doing he's also the president of uh open source initiative and so that was very interesting kind of like you know this sort of this sort of struggle to keep things open but also to continue the development of it and yeah i always find that it's one of the things that irks me whenever i see my library being mentioned on reddit and somebody says well you should watch out because it's a gpl license then you can't use it in commercial products then i wonder like okay so you expect me not to have food or shelter or whatever right that's basically what you're saying what a bad bad person for wanting payment for his work i guess yeah i i wonder like do you think that there would be an effect on the library's growth and acceptance be because of that choice um in some cases i think it's certainly possible to grow a strong product even if you are licensed in this way which as much as it is me when people comment on it i understand that it irks other people as well so i understand that it's not the most permissive of licenses and that some people will just opt out because of that but i think it's certainly possible for instance uh itext itself uses the same licensing structure and they are a multi-million dollar company so it's possible yeah yeah and so as far as individuals using it and experimenting with it and building like tools for their own purposes that's not the issue it's when it becomes like a commercial kind of gulk issue that where you're like okay this is where you're kind of crossing over to like certainly like i said i'm not a lawyer and i'm certainly not going to sue anybody who's using it for their own small little hobby projects or whatever right but i think like you said there is a certain degree of fairness that we need to respect if you are using it in a company that has a very nice profit then surely you know spread the wealth around you are making your profit by using somebody else's work to me it makes sense that you would pay them right and continue you know instead of them having to go back and find some other library or potentially exactly you know do the additional development on top of it yeah it's there's all these interesting intricate models of how open source can kind of move forward you know i was just talking to wukash longa about the sea python developer and residence program and the idea that there's some funding there you know python itself is this large open source project too you know and just trying to see companies kind of coming back and saying hey you know you can't just grab everything and like assume since i found it on the internet it's free right exactly i i had the same conversation with somebody who also used the code and he was like well it's open source so i can just use it right so then i had to explain that you know open source is not the same as which is another term yeah this is what it means these are the differences but i'm generally i think i'm perhaps a bit too soft to do the sales and marketing of this company because whenever somebody asks me for a quote they're like well you don't have any prices listed on your website no that's true i i want to have a discussion with you if you are a small company of two people obviously i'm not going to charge you the same as if you are a company of 500 making several millions right that seems unfair to me so please tell me why you need it what you're going to do and we can find something that works for both of us so i know i just find it fascinating it's i think it's really cool also you had mentioned in a recent post looking to get other people kind of involved in i think the initial like borb site you uh had a another contributor yep that's right yeah and he was helping you mostly with like graphical stuff and web stuff or was that other things yeah he he sort of motivates me whenever i get down in a sort of oh god no this is marketing he's the person he's the person who says you know you have to do this otherwise nobody is going to know the library exists you have to and i'm fortunate enough that he's also a great web designer and web developer so he was able to offer his services to have a website up and running cool do you have experience in contributing to other open source projects is that something that you have a background in doing i've done that a few times here and there i'm certainly not the most active person on github but i have contributed in the past to open source projects um like well they are mostly going to be pdf related so for instance i text for instance vera pdf which is a pdf validation engine okay just casually logging tickets and and adding my knowledge of you know it's doing this and i think this is still a valid pdf according to the rules please check or i see that it's doing this in this piece of code the one thing that i'm most active on which is sort of related i'm quite active on stack overflow and quite well versed in pdf there i think pdf is my highest ranking tag there okay like answering questions and yeah answering questions helping people understand supports yeah okay yeah mostly telling people no this is not possible yeah it's kind of it's funky you know like i i think i've run into that a couple times myself where i'm like okay well i'd like to do this or i want to do that and and then sort of ran into the brick wall of of working with it but i had mentioned i think in passing thought i was interested in a library that could be able to create something that had forms in the sets that were actually fillable forms because i find that process to be excruciatingly painful with adobe's tools to just you know okay you've got you know maybe you designed it using word or some other kind of thing and saved it as a pdf and now you want to create the the fillable boxes and it was a lot of work i mean i was paid money to do this for other companies and stuff as you should be when you're doing that right it's work and so i was like man there's got to be a way to do this and so i had kind of mentioned it and i haven't followed up on on diving into it but that is something that borb can do am i right yeah that's something that bulb can do and it does it in the same way that it does all the other stuff so form elements like a text box or a radio button or whatever i just lay out elements that you can specify at which position you want or you can leave that up to the layout engine yeah and then you can actually create forms by saying okay i want to have a table this many rows this many columns and you know this column happens to be a text box or something do you have favorite tools for working with pdfs say outside of python like you're either reading them or in my case like filling them in or what have you i dabble and i have an adobe license for other creative tools like photoshop and so forth so i kind of get acrobat in it but it's kind of my last go-to and other things won't work um i don't know if you have a similar experience there i usually use adobe whenever i'm trying to validate a pdf because okay honestly that's most people are going to open the pdf document in any way so right for me that's always a first check does adobe accept this pdf document that would be the free reader version yeah yeah okay the free to be precise the free reader of i think two years ago because i use i use linux and at some point they stopped supporting linux so the best i have is adobe two years ago okay then i also use vera pdf which is a java program that actually scans a pdf for its adherence to the pdf specification okay and it outputs these beautiful error codes where it just says you know you you've invalidated this particular rule this particular subclause so that's that's very helpful because you know it's on track if if everything goes through and then for linux uh just reading a pdf casually i use ocular which is an open source pdf viewer okay yeah i'm always kind of like wondering other tools and you know like i said i found a few on ios and i'm mostly on the mac platform but you know i bounce around between different things and when i was supporting these small businesses i was having to find you know tools that they were going to be comfortable using day in and day out that that i could sort of guarantee i wouldn't need to do tons of support on the sort of consumption side of it you know and the use side of it so i'm always intrigued to hear about more tools yeah and then like i mentioned ghost script as well that's one of the beautiful small little gems in linux that you can just convert a pdf to an image or split a pdf into pages oh cool what do you feel are like the different audiences for this library or users you could say i think definitely python developers who you know like average day business development write me a piece of code that writes an invoice or something that's yeah that's certainly one of the major use cases but i think pdf has so much more features and can do so much more that you know it will find a use even if you don't find it yet for instance pdf allows you to embed interactive 3d models in a pdf document that you can sort of configure and work with using javascript that's that's beautiful and to the best of my knowledge and not a lot of people do that but i can imagine that if you are a company that builds i don't know little gadgets as business gifts or whatever it would be cool to give somebody an offer with this pdf where they can actually see you know this is the 3d model of what we're going to deliver to you in in that case is it interactive inside of the reader yeah okay that makes me think about something that i was exploring in this whole process of creating you know fillable forms and stuff like that the idea of sort of code blocks uh javascript uh inside of the the pdf to you know say if then you know basic logic you know like if somebody fills out these things then automatically fill out these other things for them is that something that i don't know if you've looked at or worked with before inside of pdfs like i said pdf supports a basic level of javascript it blocks certain constructs because of course you don't want to pdf to randomly start downloading data or start uploading your private data yeah but you can certainly do that and the fun thing is that even though it allows you to do a lot these viruses that you could write in pdf don't happen that often but when they do they are beautiful so you can create a pdf document that forces itself to print hundreds of copies of itself on the first printer it finds and that's that's evil but also beautiful i can imagine uploading that document to stack overflow and saying something is wrong with my pdf please check and like just how much paper getting used yeah the chaos that that would consume wow are the tools that you know that can scan for that kind of thing like i said most people don't expect viruses in pdf so like adobe has some protections where it will say you know this javascript looks a little dicey for my taste i'm going to turn it off and you can turn it on if you want to yeah but other than that no okay and for instance the print issue is solved in most commercial readers they will simply force a dialog box on the user saying you know this document wants to do this do you agree right yeah do you want this insanity to happen or not that's too funny you mentioned a little bit about the future of pdf i don't know if you can speak to to that like um how you said that you know where adobe sees it going or the company you were working with previously i text you know what do you see is the future of pdfs i think there are challenges within the pdf standard itself so when the pdf standard was written a lot of options were left open to be future proof and we are slowly getting to the time where we start thinking about these so for instance if you are encrypting a pdf document you have to allow the latest encryption algorithms to be able to do that otherwise you are at some point going to get stuck or you're going to be stuck using some kind of old hashing algorithm sha1 whatever which is deprecated and yet you have to because the standard says you have to so in terms of encryption definitely making progress okay in terms of image formats of course you know jpeg 2000 and so on leaving room to do that i think one of the cool new features is also that pdf is now so so free in a certain sense to allow new content but it also has a sort of compatibility mode where you can say if by some chance you can't render this content please render this image instead so the user might get an image containing an error message saying it turns out your reader doesn't support this encryption algorithm try again or please use this reader so within the standard there are there is certainly room for improvement i think if you go up one level and if you look at the way pdf works there is also opportunity for improvement signing a pdf for instance is currently a sequential process so that means every time you sign a document with let's say two people the first person signs a hash value that belongs to the document and the second person signs a hash value of the document plus the hash value of that first person signature okay now in a corporate setting that goes wrong quite quickly because you have to sign something with a board of directors of let's say 10 people and you email it to bob who is on vacation and he doesn't sign the document and in the meantime somebody else has sent the document around to another part of the board and now you have two documents that contain signatures that can never be merged right because either one does not have the correct hash of the other so it's undoable and then you have beautiful new technologies like blockchain which is no longer that new at the moment but allows you to build a chain of signatures of hashes so these technologies seem to match quite well it seems like you would be able to do something with that yeah so perhaps need to look into that i think another one is is effectively bridging the gap between a structured and an unstructured format because pdf like i said is an umbrella term for a lot of competing standards one of them the most basic form of pdf is unstructured where you just have rendering instructions that may or may not be in order and you also have structured pdf where each piece of content is labeled as being for instance a paragraph or a cell in a table or whatever so is there a way to go from one to the other is there a way to add structure back because that would solve a lot of issues suddenly pdfs can become accessible to people with an impairment sudden