Python Tutorial: Understand the data

Want to learn more? Take the full course at https://learn.datacamp.com/courses/analyzing-iot-data-in-python at your own pace. More than a video, you'll learn hands-on coding & quickly apply skills to your daily work. --- Welcome back. After having aquired the data, we should store the data to disk for future analysis. This is common practice since not all IoT Data sources give us unlimited historical data, but also because we want reproducible results for our analysis. One premise for this is to have the same data available for multiple runs. If we want to train a Machine Learning Model, we want to keep as much historic data as possible to achieve better results, so having access to historic data is key. Storing JSON data in Pandas is as simple as calling dataframe.to_json() - specifying the filename as first argument. We specify orient equals "records", to archive a human readable result. As we can see, the format of the data stored is identical to the downloaded data. There are many other storage formats available like CSV or hdf5, all with different benefits and drawbacks, but the formats are beyond the scope of the course. After having collected the data and stored some history to disk, we will have to load the data. Pandas provides different convenient methods to load the data, depending on the storage format used. Common formats include csv and json. If the data is stored as JSON, we use pd.read_json() to load the data. Similarly, when the data is saved as csv file, we can use pd.read_csv() to load the data. After having loaded the data, we should have a quick look at the data and check if the data was loaded correctly. The simplest way to quickly check a few things is to use df_env.head(), which will print the first 5 rows by default. We can get an overview of the loaded data by using dataframe.info(). dataframe.info provides a quick summary of the dataframe. We can see the number of columns, 5 in this case, the column names, the number of non-n