Over the last few months, I have seen the number of AI projects taken up significantly and most of the folks working on AI projects in their firms are planning to increase their AI initiatives even further over the next 12 months. Many of these initiatives come with high expectations but AI projects are far from fool-proof. In fact, there are predictions that more than half of all AI projects will fail to deliver against their expectations. Failure can happen for many reasons, however, there are a few glaring dangers that will cause any AI project to crash and burn. Based on my experience, discussing with fellow AI practitioners in various organizations and going through many surveys done in the last few months, I know these mistakes are all too frequent. One thing they have in common is they are all caused by a lack of adequate strategy & planning. So, in no particular order, here are some of the most common challenges faced by business in AI projects which cause them to fail that I’ve come across. Cultural challenges Most of the challenges that an organization faces while working on AI initiatives are cultural ones. Company politics is one of the primary reason as working on AI initiatives is a team effort having diversified skill-set, from functional to technical ones. Different departments & leaders have different motivations to keep AI initiatives under their umbrella. Data literacy is also a major challenge as different people have a different understanding of the data field. Due to this, the business having unrealistic expectations with AI initiatives is not uncommon. As AI resources are the costliest ones and with uncertainty over the outcomes, getting stakeholders’ buy-in also becomes a challenge. Organizational maturity in running data-intensive projects also plays an important part. The organizations & departments who have worked on data-related projects are quick to handle the obstacles proactively. The major stakeholders of the AI initiatives are the non-technical ones. Hence storytelling becomes an important skill to make these stakeholders realize the true potential of these AI initiatives. These challenges can be overcome or can be addressed if we can inculcate data culture in the organization proactively. Operational challenges Another area where AI initiatives face most of the challenges is operational ones. The talent gap is one of the known challenges, getting access to relevant data for the project can be another big challenge sometimes. Many times having SMEs for the functional area can be a roadblock, sometimes functional SMEs are so scarce that their availability to you project can be a huge challenge. Because AI is an interdisciplinary field, the success of its initiatives depends on coordination between many different teams including IT teams. Even when you have a solution ready, deploying it at client-site or the way customer wants to consume the results can be really challenging. When you have many opportunities available, choosing the right one with high ROI/CBA becomes important. Opportunity assessment should also be included in AI operational framework. Otherwise, no one would like to be in a situation where we find that we were solving the wrong problem after putting a large chunk of efforts. Data security is another area which should be included in the operational framework to avoid problems later in the project. We can analyse that there are plenty of challenges which can be avoided if we can address them by having an operational framework in place for AI initiatives. Data challenges Not until recently. business realized the value of data. So except for the businesses where data quality has been a regulatory requirement, data quality is the frequent issue with AI initiatives. In fact, it is the most challenging part of any AI initiatives according to various surveys. Most of the time, we don’t get the data in the shape that we want and we need to consolidate, transform and aggregate the data in a way that is useful to our use case. So data consolidation is also one of the challenges AI staff need to deal with. There are many use cases where due to data privacy, you can’t use the data to make that AI initiative useful. Due to many recent or upcoming laws & regulations around the use of data (like GDPR), data privacy has also become one of the main challenges to be addressed. Technology challenges In an enterprise tech stack can be huge and disparate, especially where there is no enterprise IT governance in place. Since AI projects need to integrate with typical IT projects at various levels, having different tech-stack can also be a huge challenge. Another challenge is algorithm limitations, there is no silver bullet in AI, every approach has its pros and cons. We need to manage the trade-off between variance & bias. Model explainability is another challenging area to address, due to recent and upcoming regulations, AI explainability is becoming more and more important but we also need to manage the trade-off between accuracy and explainability. Conclusion Being a new and evolving field, executing AI projects successfully is full of known and unknown challenges. But if you are going to work on an AI initiative soon, I would suggest you keep above-mentioned challenges in mind and if you find any of it applicable, you need to proactively work towards resolving them. I will elaborate on these challenges and possible solutions in the upcoming blog posts, stay tuned. Ankit Rathi is an AI architect, published author & well-known speaker. His interest lies primarily in building end-to-end AI applications/products following best practices of Data Engineering and Architecture. Why don’t you connect with Ankit on Twitter , LinkedIn or Instagram ?
