Full Dash App - Data Visualization GUIs with Dash and Python p.12

Key Takeaways

what's going on everybody and welcome to another data visualization with Python and - tutorial in this tutorial what we're gonna be doing is kind of bringing everything up to at least the point where I'm at right now with the app there's a lot of little things that have been thrown in here and changed and added and all that and I'll just kind of go over them pretty quickly there's really no super new concepts here except for maybe the way that though that we're doing the database which I may get into the but that's a totally separate tutorial it's it's mostly just for text search specifically with SQLite and getting that to be very fast so I probably won't do that necessarily for this one but I'll at least tell you what we've done so anyways this is the app live Twitter sentiment you type in a search and let's say you wanted to search for car you wait a minute or two there we go there's car now part of the reason why we wait a while so for example there's car but if I'd highlight car and just delete car really quickly it's much much faster and the problem with with the search as it is right now is if we type car each letter gets searched in the database so if I typed it slower you would see oh it searches for C and then C a and then R it's just if I type it quickly enough it's not going to you're just not going to see it all update you're just gonna see the full term but anyways so that's one thing I'd like to improve otherwise what we have is just live updated sentiment over here just basically it's every second I think and then over here this is just longer term sentiment its updated we can look at that later but I think maybe 60 seconds here are the tweets these are updated instantly so as they come in it gets updated here and then also I show basically the date the tweet and then the sentiment and I also color code the sentiment with a slight kind of background tint either green or red and then we also track positive and negative sentiments take notes a lot of sentiment is neutral I thought about showing both and then I realized I you can click on these and like make it appear disappear it's kind of annoying though because each time the graph will it goes back to wherever the original state was and I felt like this was a little more useful than also showing the neutral but all the code is open sourced which I have the the repo right here sir cent X slash social sentiment so github.com cent X slash social sentiment I will put a link in the description though so if you need it you can head there rather than typing that inks that would be so horrible everything is pretty much explained here so I guess I don't really think there's anything else I really want to go over oh well I guess we have like related terms and stuff like that but that's all so so like depending on what you search into here like if we search car we can see the related terms to car and that's just any basically it's any nouns that are contained in tweets with car and then we count and then the top ones get shown so for especially for a lot of things like for example with car career it gets chosen well car is a it's a sub word inside of career and so that's probably what's going on same thing with car whereas if we do maybe nothing we get maybe a little more useful information and then if we were to search maybe maybe Trump hopefully we'd get some more related terms to Trump thing just the fact that because Trump's probably not a sub word to so many things those sessions obviously Trump president not sure what what Connor would be in reference to Rex Tillerson just stepped down Russia because of course the GOP not sure about knew and would and I'm not sure ku local Oh anyway and then obviously just recent trending messy that's just real really recent I actually just truncated the database Stephen Hawking unfortunately passed today so that's trending and yeah the other things so let's run through the code at this point so what we have here is this the the contents in here basically the main application is is - mess if you wanted to run it locally you can use the dev server just so you could run it on you know on localhost port 80 50 the Twitter stream this is what you're going to use to actually stream tweets from Twitter you've got the config which right now is just stop words so it has to do with these these related terms for the most part so there's a lot of words that we you know they're just totally used useless so we actually want to just ignore them if they did happen to come up so that's all that's in the config but we'll look at it in a moment you get cash this is just for caching especially in on the web app itself just to hopefully get things to run quicker if so we're not just performing the same searches and stuff and then the DB truncate hi this is just so how we can just truncate an infinitely growing database so I just cleared it I actually made a mistake when I was building the truncate script and I was either gonna have to leave probably 10 million records that I'd never be able to delete or just start over fresh I chose to start over fresh but hopefully now the the the truncate script works so anyway we lost I think we had like 35 million records or something and I just I didn't want to leave a bunch of records that could never get rid of so I just started over but it should be about 3.5 million records a day if you're faster processing you should be able to get more than four and if you're slower at processing you might get less but that's a lot of records every day and that database grows very quick it's just a simple SQLite database but we're actually using ft/s with the SQLite which is great for text search but more on that in a moment so I think probably the first thing we'll open up here is just - messed up pie to kind of run through the code here I'm trying to think if there's anything super major so a lot of stuff here it's just really pretty pretty badly organized so so you if anybody wants to contribute that's probably one really easy way to do it is just a pet bait this code it's just all over the place but as you can see we've got you know basically the layout of the the application I'm using the row you know Road divs basically from materialized CSS so that's kind of how we're maintaining the interact ability I guess of the of the application so it for example if I squish this up everything's kind of interactive and eventually it'll pop down and then we just have one graph and one thing per and the tables actually interactive as well so using materialized CSS we're able to actually have that interactive interact ability I guess and then I'm also importing Google Analytics yeah so other than that let's see let's go back up ok so that's basically how the layouts happening I don't really think that should be too too complex it's just by giving it proper class names especially unlike the row and then each element class name this is just how big that classes and stuff like that so if you want to learn more about that just go to materialise CSS I think if there's anything like so probably you know the line graphs you should have already seen I'm pretty confident I can't remember it's been a while since I filled that filmed the last tutorial I think we covered the sentiment sorem the volume part - the only graph that probably we haven't covered is a pie chart but again all these things are just plotly graphs so with the pie I honestly can't remember if it's a donut in know it's a pike yeah it's called pie but again to figure out for me the way that I did that I just went to to the plotly documentation looked for a pie chart what was required how to how to make it and like that's it and then as usual the return is just the data and the layout so any graph you can find in the documentation I mean that's all you really have to do and then you know everything's pretty much the same unfortunately yet one thing I really wish you could do is like have multiple outputs per callback like if you could I mean I don't even know how you would do it to be honest but that would be nice if you could do that so you can definitely have one input go to multiple callbacks and then output things but especially like one thing that we're having trouble with was the doing some various calculations so you for example we're using this caching system so that we only pull like four examples is trying to look for one that updates our cache it's at least one of the SQL queries that's like the 10,000 pole for example hmm so I'm gonna be that one I could just do a search I suppose for cache cache get I don't want to do that here's a cache touch set here at least yeah so like for example in the historical graph we we need to at some point have made this this query right but while we since we've made this query we could also perform an operation but then we need to be able to save those terms and then share them across and stuff like that so anyways reusing the cache for that the other thing while we're here while I'm looking at a query just note that these the select statements are a little different right we're using the FTS from SQLite and at least to my knowledge the main purpose of FFTs would only be for text search so so you're probably going to get any gains with it using for other means but if you're looking like if you're doing what like what we were doing where you're you're looking for a word inside of one of the columns and you're using that whole like like thing then you will probably be able to see significant gains like huge gains by using FTS fast text search actually I'm not even sure if that's what it stands for but that's probably what it stands for anyway sorry if that's how it stands for anyway so that that is a little different but as long as you keep the database kind of small like you it probably won't get too too absurd but even after like the end of a full day these queries can can take a considerable amount of time so that the FTS stuff helps a ton and thanks to Daniel kook yella who's who's the one that basically wrote all that of trying to thinking there's really anything else I mean the rest of this stuff is pretty simple logic and stuff that you've really already seen everything boils down to these callbacks you take some input you spit it out to output and all that one thing like I would really like to be able to do is in these like the recent trending and the related sentiment so these words right here I'd really like to make them clickable and the issue I had was obviously to make them clickable he would just give them some sort of ID and then have like an on click event the problem I came across is when they are displayed on the page that causes an event which caused them to run and then it just starts this you know never-ending loop of one gets called pops up does the search get some related terms as soon as the first one gets you know thrown in boom that one runs and it just goes on forever so really sure how to how to best do these but yeah it'd be really nice if I could just boom click Hawking and then bang it just Sujit searches for Hawking so anyways that that's probably the biggest thing I'd like to fix other than just the way this this looks I wish it was a little better looking but I'm not good at styling other than that I'm trying to think if there's really anything else that is crazy with this code I mostly just wanted to to put the code out describe a few things to you guys and explain a few things and then and really that's it so I don't think oh the other thing I still was making that mistake here where we were just defining external j/s but then I was iterating over external CSS which was kind of weird so that's been going on for a while I never fixed that so Fischer you don't you're not making that mistake still other than that let me scroll down here and see okay so Twitter stream so yeah if you want to clone this all you really need to do is obviously make sure you install the requirements fill in your information to the Twitter dream dot pine good apps I Twitter if you need to if you want to deploy it use this otherwise just run the dev server instead and then this should be probably run from time to time I think that's really it I don't really know that I need to cover anything else but if you have like questions or whatever on the application itself feel free to ask in the comments below you can also join our discord I'll have a link in the description for the discord if you want you can also file an issue like someone says social sentiment wasn't working but I guess he hasn't responded yet but I wasn't able to replicate that issue it's I've never seen that error when I tried to load this so I'm not really sure otherwise I think that's it if you have ideas on how to do the clicks that'd be great if you have ideas on how to make this look better even if you just want to change the colors go for it most of the colors like I said are who's that there's that good-looking individual oh my goodness it's gonna - mess so the main colors are right here in app colors and then these are sentiment colors for the related sentiment I believe and then really really well placed down here I think this is for the hmm I can't remember this for the pie chart or if this is for I can't remember sorry but the pie charts colors are at least in here any other colors I wanted those to be a little brighter though than you know the colors up here and then I also wanted the table colors to not be absurdly you know brightly colored or not so anyways there's the some colors are in a few other locations but I mostly think it's like these main colors here like the black background and the color for these lines and stuff like that that would probably help in just the way like these you know titles and stuff like that it's just not very well designed so so probably those main colors and maybe some more of this layout stuff like that yeah I think that's it questions comments leaving below ask us in the discord post an issue make a pull request otherwise I'll see you in another tutorial