Python Tutorial : Data transforms, features, and targets

Key Takeaways

now that we're familiar with our data we need to get it prepared for machine learning to prepare our data will need features and targets our features our inputs we predict future price changes with the 10-day price change and volume our targets are the future price changes generally we can use pandas dataframes and series in our machine learning algorithms it's useful to incorporate historical data as features for example the price changes in the last 200 days instead of including all previous two hundred days price changes we can concentrate past data into a single point using technical indicators like the moving average shown here a moving average is the average of a value in the past n days classic moving average periods are 1450 and 200 days for stocks here's a plot showing the AMD price and a 200-day simple moving average or SMA you can see moving averages smooth the data the other indicator will use is relative strength index or RSI this oscillates between 0 and 100 when it's close to zero this may mean the price is due to rebound from recent lows when RSI is close to 100 this may mean the price of the stock is due to decline the equation for RSI is 100 minus 100 over 1 plus relative strength relative strength is the average gain of price increases divided by the average loss of price decreases during the time period n both RSI and moving averages can be calculated with the ta Lib package this Python library is a wrapper for C code meaning we can run C code using Python to use ta libs functions for RSI and moving averages we provide an umpire array of prices and the argument time period this is the value of n mentioned in the previous slides we're using 200 for time period and adding the new feature to our data frame as ma 200 for the 200-day moving average and RSI 200 for the 200-day RSI 40 a Lib functions we must provide numpy arrays not pandas objects the values property of pandas series and data frames yields numpy arrays we can now make our features and targets we choose the ten-day close percent change and the 200-day moving average and RSI from our data frame these feature names go into a lists which selects the columns from the data frame then we select the ten-day future close percent change as our target finally we create a data frame with both the features and targets so we can check for correlations before we do any machine learning it's good to check features and targets for correlations we use the pandas function core to calculate Pearson correlations and the Seabourn library has a handy heat map function for plotting the correlations the a notes option shows the numeric values for each correlation in the plot the plot looks like this with colors ranging from black for negative correlations to white for positive correlations the numeric values are also shown in each square to examine a correlation between two variables we look for the intersection of the two variables in the plot for example the RSI and tend a future price percent change intersect in the bottom left corner these variables have little to no correlation since the value is close to zero usually a value of 0.2 or greater means there is some linear correlation present the diagonal line with all ones shows the correlation of each variable with itself which is 1 this means each variable is perfectly linearly correlated with itself as we expect ok I think you're ready to create your own features and targets