Big Data, Hadoop and Machine Learning Explained using Dams

Key Takeaways

i was recently asked the difference between big data hadoop and machine learning let's say there's a dam to store water and there are lots of pipes connected which deliver water to homes in the summer months there's less rain and there's less water in the dam water flows without any issues this is small data in the rainy season the torrential rains and now dam has exceeded its capacity there's so much water and it's overflowing everywhere your pipes are under tremendous stress they can no longer hold foot and they start leaking this is big data but without infrastructure now you try to mitigate this problem you distribute the way you store water using something called hdfs use a technique called mapreduce to process water and this is called hadoop with this you're able to control the flow of water you now want to understand though what's flowing is it just water or is there something else you take a sample of the water apply something called machine learning on it your research helps you understand there's some industrial discharge into the water now you approach the authorities and ask them to fix it and now you're happy because big data hadoop in ml has helped you organize everything