Infinite Scroll Pagination with Firestore

Key Takeaways

infinite scroll is a feature that can have a huge impact on the user experience by loading data asynchronously based on scroll position in this video I'm going to show you how to implement infinite scroll and angular with the firestore database but we're going to do it in a way that not only allows you to scroll down to load content but also to scroll up and load content in the opposite direction this has been a commonly requested feature and it would be used if you were doing something like Facebook Messenger or slack where you load a timeline of messages in Reverse we're going to accomplish this by creating a generic service in angular and it will not only retrieve the data but also tell us when we've reached the end of a collection and when we are loading new data if you're new to the channel make sure to subscribe and grab the source code for this project from angular firebase comm I'm going to be starting from a brand new angular app and it's only dependency is angular fire so make sure to go to the documentation and follow the setup instructions there then the next step is to create a directive called scrollable this is going to tell us whether or not the user is scrolled to either the top or bottom of the page we're going to do that by monitoring the scroll event on the element itself that'd be that container element and then we're going to omit our own custom event that will tell us whether or not it's the bottom or top you can create a custom event and angular by using the output decorator and event emitter so our custom event here is named scroll position then I'm going to add element ref to the constructor but I want to warn you that using element ref for direct Dom access can cause issues in angular with platform server and web worker we can address this issue by wrapping native elements in a try-catch block then provide a fallback such as a load more button when the Don doesn't actually exist first we'll set up a host listener on the element then listen to the scroll event so keep in mind that your host element needs to be something scrollable and we'll set up a function here and put our try-catch block inside of it then we need three values from the native element itself we need scroll top scroll height and offset height first we need to know if the user has reached the bottom of the element so we can do that by saying top greater than height minus offset minus one extra pixel when that condition is met we our event emitter send out a string value of bottom four top we can simply see if the scroll top is at zero before we go any further let's make sure this works in an actual component so I'm going to switch over to the app component then I'm going to import observable as well as the pagination service which we're going to create the next step then I am setting up an event handler here that will just console.log the event from our custom event in the directive all we need to do is set up a div and inside that div will add the scrollable directive and then for the event we use parentheses with the scroll position event and then we'll run our scroll handler and I'm going to add an extra div in here with some height to it just so we can actually scroll through this parent div if you watch the console to the right you'll see we either get a bottom or top event once we've reached the bottom or top of the element itself and before we get to firestore let's generate a component called loading spinner then head over to spin kit and choose your favorite spinner and just copy and paste the HTML and CSS into that component before we build this big complex pagination service I want to show you how it's going to be used in the component to paginate in fire story you need to send a new query for each new batch of data that you need the service I'm creating will allow you to make a query with some basic configuration options and this will populate an observable on the service that we can subscribe to the first argument to an it is the collection and the second is the field that you want to order by you can order the collection in Reverse and you can also prepend new items to the beginning which is what you would use if you're doing an upward infinite scroll after the initial query we'll listen for events in our scroll handler once we receive the bottom event we can just call the more method from the service and that will populate the next batch of items from firestore so the service we're building will give you infinite scroll with just two lines of code the service will also give us a data variable that's an observable array that we can loop over with the async pipe so we'll display some information about our boats in the database but the service also has two other variables that are important the first one will tell us once we've reached the end of a collection and that's the done variable so we can listen for that and then say I ran out of boats once we reach the end we have a another observable that will tell us whether or not a new query is loading so we can show a loading spinner for each new query and that also helps us prevent sending multiple unnecessary queries if we take a look in fire store quickly we just have a boat's collection that's in the route and each document has a URL and a year field so we'll be sorting by the year when we look in the app we get the initial loading spinner and then once we scroll down the loading spinner will reappear and about a second later we'll get some new data loaded now we're finally ready to start building the service we're going to need angularfire store and we're also going to make pretty heavy usage of rxjs behavior subjects next I'm going to set up an interface that defines the query configuration options so we have a required field of path which is the path to your collection and then we have the field that you want to order by then we'll set default values for limit reverse and prepend prepends the important one that's the one that you would use if you wanted to have the items at the beginning of the array instead of added to the end of the array next I'm going to set up a few behavior subjects the subjects are all private variables and they have underscores so you can keep track of them easily in the code done will tell us once we've reached the end of the collection loading will tell us when a query is in process and data is the last batch of information that has been retrieved from firebase then we'll set the query configuration as a variable so we can keep track of it between queries and that'll be typed to our query config interface then for each of these subjects we're going to set an observable that will be consumed by the component this will keep all of the data changes isolated to the service this part is technically optional but it is considered a good practice all we need to do is call as observable on the source data then the next step is to add angular firestore to the constructor then we're going to go through a lot of code here so I recommend checking out the main article for detailed explanations on each of these methods the first one we'll set up is a knit which is going to make an initial query as well as save our query configuration on the service it requires a path to the collection as well as a field to order by and then it has optional defaults here that you can override by passing them as an object so we'll set a default limit up to and then say reverse false and prepend false now we can use this object to make a reference to the firestore collection we're going to order by the query field if the reverse property is set to true then we'll go ahead and order it descending otherwise it'll be ascending then we'll go ahead and limit it by whatever set in the query configuration then I have a map and update method that I'm going to show you in just a second here but first I'm going to show you how we define the observable data that will actually be used in the front-end component so we'll take our source data and then we'll call the rxjs scan operator on it scan allows us to build a larger array over time if the query options are set to prepend then we want to take the new value and concatenate it to the beginning of the array and if it's not set then we just want to go ahead and concatenate it to the end scan is a pretty weird operator to learn so I recommend going to rxjs marbles and checking it out visually now let's take a look at that map and update method it takes the angularfire store collection and then Maps it down to the snapshot that we need and then sends it to our main data source if the collection is already done or loading we don't want to make the query so we'll just return out of the function but otherwise we'll set the loading state to true and then we'll call collection snapshot changes to paginate a cursor and firestore you have to use the document snapshot so what we're going to do is map down the array to the data that we want to show on the front end as well as the snapshot you'll see in a second here how we use the snapshot to offset all future queries then after we have this data mapped down we're going to check and see if the prepend option is set then we need to reverse each individual batch to maintain a consistent order going in Reverse there may be a better way to do this but I wasn't able to figure one out for the upward scroll when you're pre new items to a list after we have the values mapped down we'll go ahead and update them in our main data source at this point we can set our loading state to false and we can also check to see if that was an empty array at which point we know we have reached the end of the collection and this is going to be a one-off operation for each new query so we're going to say take one and subscribe this will end your real-time connection with firebase but pagination with real-time data introduces a whole other host of issues and trade-offs that you have to think about it's beyond the scope of this video but please reach out on slack if you want to talk about it now we're ready to set up a method to make secondary queries so first we need to know the cursor that we want to paginate from if we're a prepending we want the document snapshot from the first item in the last batch otherwise we want the last item in the last batch if the array is empty we want to return null otherwise we will get an error from firestorm the more method is going to make all subsequent queries first we need to get the cursor for the pagination and then we'll create a firestorm reference it's identical to the initial query except that it uses start after and passing that cursor as an argument this tells firestore where to offset the query from then once we have that set we can call a map and update and our data source will be updated that was a lot of code to get through in 10 minutes but make sure to check out the full article for detailed explanations on all these methods my goal is to give you a generic service that can work with any firestore collection and give you infinite scroll either upwards or downwards that's it for firestore infinite scroll with angular if this video helped you please like and subscribe and if you want twice as much content every week consider becoming a pro member at angular firebase comm you'll get exclusive content a free copy of my book and one on one project consulting thanks for watching and I'll see you next time [Music] [Applause] [Music]