Friction Between Data Scientists and Software Engineers

Key Takeaways

so my question is and by the way I'm also coming from electronics so hackle okay so my question is what are the main main fiction points between data scientist in software engineer because as operations as a operations engineer with with background I see this fight again between software engineers and operations yeah what's new yeah well nothing's new it's the same old same old story totally agree so so just to paint the picture you know ten years ago maybe not even ten years maybe less than that there was a big divide between operations staff and developers developers would develop software I mean you've all had this before right it's the whole reason why DevOps became a thing was to try and attempted to smash down that wall between the developers and the operations teams and that has worked somewhat I've seen larger companies there are still very big Operations teams so they still exist in they're still there primarily because there's also a lot of old software still running so they don't need to keep running that software so they see it and so went out when I sort of started doing data science ml more full time which that kind of about 2010 the same thing was emerging with ml I would write algorithms I would develop you know did science solutions and then I would hand it off to a software engineer and expect the software engineer to implement that properly because data scientist can't write proper software and then that would then go off to do the operations and so you had these you know multiple jumps and it was a daily a daily exercise of complaining about the other person so the software engineer would complain to the data scientist saying you know I don't understand this what's going on here is crazy it's too complicated and the data scientist is going to the software engineer so it's not complicated it's dead easy how can you not understand it so I fully agree with the push of devops up into data science as well and I guess that's that kind of falls under the banner of ml ops or data DevOps or whatever you want to call it but basically trying to bring the the data scientists into the software team into the operations team so all together as a team they can actually they can they can be responsible for and they can deliver good ml solutions the the downside is is that in many in many sort of roles that scope falls towards a single person it's becoming more and more expected that a single person would be skilled at doing all of these things and so that makes the the stack the full stack is becoming fuller like the word full stack is is pretty funny because it implies that there's nothing else to add but in fact this stuff being added all the time so it's we're going you know uber full now if you're adding data science on the top of that and there's there's obviously a limit to that there's only so much that someone can learn and yeah so that's an organizational challenge yeah the the full stack is now overflowing stack [Laughter]