Project Management of AI projects with Essy Dahlin and Krzysztof Kotowski

Our experts, Essy Dahlin and Krzysztof Kotowski, explain the differences between an IT and AI project, and talk about the differences between AI, ML and Deep Learning. Then they discuss the importance of maintaining clean and ordered data in projects as well as the process of moving on from the MVP into a fully-fledged ML product. Essy shares some successful cases studies where AI was successfully implemented.

Our Guests: 

Essy Dahlin: With almost 20 years of experience from different industries in both private and public sector, Essy Dahlin brings insights from a broad range of roles and areas of expertise. Since 2016, she’s been working as a digital strategy advisor at Sogeti, part of Capgemini. With a holistic view of business, market, and organisation, she advises business leaders on how to address challenges facing their industry, accelerated by digitalisation and innovation. In close collaboration with architects, designers and developers, she also leads innovation projects where emerging tech proves its value.

Krzysztof Kotowski is Lead developer and machine learning specialist in Future Processing Healthcare. Co-author of two deep learning-based products: the CE-certified Sens.AI tool for gliomas detection and Adaptive Vision Deep Learning Add-on for industrial machine vision. Speaker at multiple scientific and industrial conferences. Graduated from Cranfield University – MSc in Digital Signal and Image Processing. PhD candidate in Computer Science at the Silesian University of Technology, specialized in brain signals analysis.

Michał Grela is Future Processing’s Relationship Manager, working within the marketing department to establish and nurture relationships with prospective customers and expand the company’s network of contacts. He strongly believes that business is about people and that, at the end of the day, it’s all about Human-to-Human rather than Business-to-Business.

Michał Grela (MG): Hello, and welcome to another episode of IT Leadership Insights by Future Processing. My guests today are Essy Dahlin, Digital Strategy Advisor at Sogeti and Krzysztof Kotowski, a Software Developer at Future Processing, Future Healthcare division, both with vast experience on running AI projects. And today the topic of our discussion is gonna be project management, in particular in AI projects. But before we start, before we answer this question, what was the distinction between the regular IT projects and projects that involve modern technologies, such as AI, could you guys please tell us more about yourself? Essy?

Essy Dahlin (ED): Thank you. Well, in my work, I help business leaders overcome the challenges facing their industries with the help of technology and innovation. And I also lead different kinds of innovation projects, often related to AI.

MG: Thanks. Krzysztof?

Krzysztof Kotowski (KK): As well, as you mentioned I’m a software developer but also a researcher in Future Healthcare division. But also I’m finishing my PhD at Silesian University of Technology here. And I think it’s in AI domain, the business and academia are inevitably connected with themselves. So I’m trying to connect the academia and business.

MG: Okay, guys, so just very briefly before we dive deep into the topic, with so many buzzwords, such as AI, ML, deep learning, neural language processing, and all the others, what are the distinctions from your perspective? What sets them apart? What are differences? Essy.

ED: Well, first of all, I would like to say that there’s two concepts, overall concepts of AI, artificial intelligence, is the one that is more based from the imagination, but it’s actually working on from organisations like Open AI, where they try to build a safe general artificial intelligence. We’re talking about singularity and so on, but that’s more for the future. And then we have the overarching concept, artificial intelligence that covers different technologies, more hands on to the regular people like you and me. And then we have machine learning and deep learning, natural language processing, text to speech, speech to text and so on.

MG: Kris, would you would you agree or?

KK: Yes, of course, I totally agree here. In my developer practice, usually we call it narrow AI, the solutions that are based on specific data for specific solution and in practice, it’s like 99% of projects do use narrow AI. We are trying to develop something beyond this narrow AI, like artificial general intelligence, but it’s still just the research field and we are talking about this but we are not implementing this for now.

MG: And in another episode with Essy, we discussed that the topic of innovation and we’ve agreed that innovation must go both top down, bottom up, and when we were talking about the management layer, how do IT management people explore the opportunities behind behind AI? Essy, you work with business management on daily basis, how would you say they can explore that one?

ED: Well, both by first of all, it does not need to cost very much.

MG: So, that’s the argument.

ED: That’s overcoming some obstacles anyway. You can explore quite a lot with the open platforms that you pay by use. And by discovering and learning from both IBM, Microsoft platforms and many more, you can see both advantages and in which different areas, different platforms excel. So having a large variety of different platforms and tools, I think it’s good when you’re in exploring mode. But then also you need to have your mindset that this is not an IT project, this is a business project actually. Because even though you have engineers, you have software developers involved, deeply involved of course, in writing algorithms and so on, it’s often the business that needs to be as much involved to validate data, to validate processes, and actually being very much in part of in the maintenance of the product in the end as well. So, that would be a start.

MG: Krzysztof, speaking more about the development part, I’m thinking about the software development life cycle. And when it comes to AI IT projects, is the life cycle much different than the usual software development projects not involving AI?

KK: I would say, of course, over a few years here in Future Processing, I’ve been involved in multiple AI-based or machine learning-based projects. And from the beginning, we were wondering how to start, what’s the process? Is it Agile, or is it some kind of Machine Learning Agile? So we developed our own life cycle of the product. For the start, the start of every machine learning project is the same, it’s the data. So, you need to get familiarised with data, you need to know what you have in your data. And then you need to create some creative space for your data scientists, engineers to have some, maybe not fun, but some brainstorm about the ideas how to use this software and this process in this particular point, we call it ‘Sparta mode’, because project managers were very worried about this part of the project because we are avoiding tasks like very heavily agile and heavily documented process and life cycle. We tried to have a small brainstorm and to get some idea for the proof of concept or a Minimum Viable Product, and then we can go to the Agile development, like standard and we are proceeding this way.

MG: So, there are differences?

KK: There are differences.

MG: Foundation is more or less similar.

KK: Yes, but we found them by ourself, in our company.

MG: You’ve mentioned data, and let’s focus on that bit for a while. Because, as far as I understand, and correct me if I’m wrong, data is the the absolute foundation of ML, AI, whatever this sort of technology products. And although companies nowadays possess a lot of data, I guess they won’t admit that they’re having a bit of a mess there. How do you start with that one?

ED: Well, one of our key experiences are the importance of quality of data. Yes, when you’re supposed to build a model, from data you already have, you need to know your data, exactly as you said, and also to know the quality of the data. But also to actually know where was the data coming from, from the beginning, from which age maybe. Because even though we’re talking a lot about being data-driven, we need to understand how and in which context was this data collected. Because if it was collected in a time where biases – gender biases, for example, was very high, the data will reflect that. And if you are going to use that data when recruiting new talents, that could be skewed very much in pointing out on maybe middle aged men, when you’re not really, that’s not our values today and that’s not what we want to do. So you need to be very aware of the biased data itself.

MG: So you are saying the context is very important.

ED: Yes.

MG: The context in which the data was collected.

KK: I would add that I don’t worry about the mess in data. I worry about the bias in the data, and I have practical example also, because there are gender biases like social and moral biases. But in practice, we got some data, two sets of images. One was images without defect and images with defect and they were taken in different conditions. So it was a bias of condition when these images were taken, so for example, environment, camera angle, light. So our classifiers were 100% right, but it was just a bias of environment. So practical example is also…

MG: And did it affect the project?

KK: Of course, because client had to redo all the data, all the images under our supervision.

MG: And how do we avoid these biases?

KK: You need to make some understanding in your clients that you need to take care of your data from the beginning. If you are not using now AI-based solutions, you probably will use them in the future. So you need to take care of your data, of the organisation of your data and of biases in your data to be the representative data.

KK: Well, I’ve heard the saying that the AI product will be only as good as the data behind it. So that’s a good tip, I guess. So let’s agree that once we have this data ordered, or clean or without these biases you’ve mentioned, what does the the next stage look like? You’ve mentioned about prioritising MVPs, Essy, what would you say the next step would be?

ED: Well, either you go from innovating from the data, which is ‘what can we do with this data set?’. Open questions, or you go from the top and see ‘we have a problem here’ or ‘we have a need’, or ‘our customers are demanding something from us that we can’t deliver today’. And, how can we find data? Which data do we need and how can we work with that data to deliver that output? So, you can go from both directions really.

MG: From your perspective Krzysztof?

KK: I would say from a developer’s perspective in this second stage after your data are here and cleaned, you need to start research. And basically, I was trying to do all the parts like data preparation, research and going from MVP to product. So, this part with research is quite tricky because you cannot estimate the task for research, you cannot estimate if the results will be good when you try for one hour, for two hour or full two weeks. So you cannot estimate this part. And in project management there is like a gap in data. But you need to, for example, you need to approach to not push your developers, but also state a deadline for this research part. So don’t get too loosely…

MG: Give them space.

KK: Give them creative space, but with strict…

MG: But expect something.

KK: Expect not all the ideas, but the best idea.

MG: So since we have the ideas, you’ve mentioned going from MVP to a fully-fledged product, was the process like in here? Do you have any experience in that case?

ED: Well, for example, lean startup methodology is for good when you are in a very ideation mode, and also trying and testing your ideas all the time to see if it actually delivers the value you were thinking of. But also in a design thinking, from a design thinking perspective, that you get to design your product, your finished product first and then go from there, backwards, so to speak. But there are a lot of different methodologies and how to approach and how to move forward in an AI project. Just as same as any software development project, but the important thing is I think is to be agile and to iterate and to have different milestones where you decide ‘should we pivot or not’? ‘Should we stay on course? Should we end the project?’ And exactly as you said, you need to have some kind of deadline, otherwise you can go on years and years and years. So, you have a certain timeframe to play with, so to speak, but be very agile during the time.

MG: Would you add anything, Kris?

KK: I can say that we develop the fully-fledged product, like standard Agile deployment in Future Processing, the same life cycle, but we need to take care of quality. So, right after we have proof of concept of Minimum Viable Product, we engage quality engineers but not normal quality engineers but machine learning quality engineers. They need to know the basics, the knowledge about the AI to practically test our solutions, the quality of our solutions if it doesn’t change and for example, if it doesn’t have biases and there are more integration tests than unit test, because usually we cannot unit test our machine learning models unit integration test, or end-to-end test to try different possibilities like in our standard project. So there is an Agile approach, normal tasks and to take care of data infrastructure, user experience. So these are normal parts of normal product. And it’s not an easy process, because if it was the universities would be very rich. In Future Processing we have for example, human resources and project management that can lead to the product, not only research idea.

ED: I’ve also been in different projects and some more like yours where you can actually build a lot on the IT side, so to speak, but also projects where half of the work is done by the business side. Maybe you don’t have any initial data to work with at all, or maybe you just scan a lot of documents, for example, and then you put it into an administrative process, then it needs to be validated and built on and continually trained by the administrators. So there are not just one type of project or AI project, but there are different types that need to be adapted, according to the situation and according to the output and what kind of technology and data you’re gonna work with.

KK: The structures of projects are similar, but each project is different.

ED: Yes.

MG: A vital part of every project is a team that actually delivers it. And you’ve already mentioned that there are some differences between team members, the team management, the team structure and what are the other differences between the standard IT team and the AI IT team?

KK: I would say we tried two approaches, that we have different subteams of data scientists, data engineers and infrastructure engineers – this is the one approach. And the second approach is, like I was trying to convince to this approach, that there are full stack ML developers, so they can not only program, but also supervise all the steps in there, from the MVP or from even data, through MVP to product.

MG: What’s the advantage of that approach?

KK: The main advantage and the biggest advantage is to avoid some misconceptions between the subteams. If you have one team of guys that know machine learning concepts and programming, they won’t take some arguments as subteams between them. So the communication layer is the first biggest draw for us of this approach. And the second is, for example, I lost my…

MG: Don’t worry, I guess there are many, Essy, what would you add? For you, I guess you’ve had experience of working with many different teams, do you use both AI and not AI? Can you tell the difference, apart from the members or modes of management?

ED: As I talked about earlier, of course, slightly different competencies required, of course. But also AI projects, from a business perspective, are often more controversial than normal IT projects. So from a cultural perspective, and then from a change management perspective, it’s important to also have that into consideration. And taking into account people’s apprehension for this kind of change and what it can mean for the workers in your company. So, depending on the project, I would say, you need to take into account different aspects, but I think the buy-in from the organisations that will actually receive the product in the end is crucial otherwise they won’t even, they will never use it, unless they are on board.

KK: I found my point in the meantime, about skills and skills of the development team, for example diversity – you can really benefit from diversity in your team. If you have this full stack developers from different backgrounds: from mathematics, psychology, neuroscience and physics. There are many physicians in machine learning domain, so it can benefit from diversity and also you need project manager with something we call technological quotient. So this project manager is open to changes and not afraid about changing this mindset of the team, of your own and the team. So that was the second point of the full stack ML developers and teams.

MG: Essy, you’ve mentioned that ML, AI project, these sort of projects are to some extent controversial. And from my perspective, this controversy comes from people, from the fact that people are maybe not feeling the tangible benefit of it. Maybe it’s not there yet, here with us on daily basis. We know that companies are working on AI projects, ML projects, but we don’t feel them yet like living on daily basis in the society. For us, for most of us, it still might be in the area of of maybe a terminator or, I don’t know, a computer that will take over the humanity someday in the future. But we all know it’s not true. The examples of benefits of AI are there yet. Can you perhaps share some successful and positive case studies where AI were successfully implemented?

ED: Well, I must disagree somewhat with what you’re saying.

MG: You are free.

ED: I would say a couple of years ago, maybe three to five years ago back, those conversations were very common. ‘AI is something out there, we don’t know what it is. It’s Hollywood stuff and so on.’ But during this last few years, the conversation has matured and business leaders and IT managers, at least in Sweden, I can’t speak for Poland, are very much involved with the development and are very much aware of the benefits. So we see in many organisations implementations of AI, but also implementations of AI-related software. For example, chat bots, chat bots are very popular, but not every chat bot is truly AI or machine learning, for example. So, there are a lot of processes being changed with AI. A lot of, for example in the manufacturing industry and maintenance industry, working with predictive analysis to prevent downtime. We have fraud for example in bank and finance.

MG: Insurance.

ED: Yes, exactly. Many implementations in fraud predictions.

MG: Kris, what about healthcare?

KK: Of course, it has changed a lot since I started my work here in Future Processing and Future Healthcare now. A few years ago, it would be impossible to certify the medical products using machine learning or some neural networks, it was completely magical for certificate committees. And in a few days now, we are approaching to certify our medical product, Sense AI, for brain tumor segmentation.

MG: Fingers crossed.

KK: Yes, fingers crossed, so it is possible now. And even the the FDA – an American certificate can be used to provide, can be used in AI-based solutions. So it is changing still, and I think it will be much better in one or two years.

MG: I agree with you both. I’m on the same page, definitely. My goal was just to say that it might be a bit tricky to explain it to non technical people. And bearing that in mind, are there any challenges for example, when communicating with the…, with the communication between the tech side of it and the business side of it? You’ve mentioned that the business already gets the point, but is that common?

ED: Well, in that case, I mostly talk about the leadership. Many leaders, business leaders, get the point and understand the benefits of using AI. And I believe that every area and process will somehow be impacted by AI in the future. But not every single problem needs to be solved with AI technology. So that’s also something, you don’t need AI for everything. And it’s not a magic wand, you can just wave around with and it solves every problem.

KK: And sometimes, expectations are way too high.

ED: Yes.

KK: That’s the problem in the industry, that after all this hype and all, because hype is still on. And the expectations are very high if the things that clients want are very hard to imagine, sometimes for us and even more to develop them.

ED: And in more traditional organisations, the majority of the employees may not understand the more hands-on implementation of AI.

MG: You’ve mentioned AI has huge impact and will have, continues, and it will be ever growing. But does it also affect project management as a discipline?

ED: Sure.

MG: In what ways?

ED: Well, automating process management, of course, finding signals when workload is too much on one individual, in real-time analysis of all the different tasks and what data is available or not. You can put in anything actually and create a new process based on different AI analysis technologies.

KK: I think project management is like complex process that you can model with some machine learning methods. So, you can still benefit from machine learning in the project management for example, like chat bots for JIRA, there are some existing projects like this, but also I think that project managers are in high demand now, because the AI-based solutions are in demand, so they are. And they don’t need to worry about work in the future, because they say that AI will take millions of workplaces, but I think project managers are quite safe here.

MG: There is this threat in society definitely that it will take over some work, but at the same time will it not create any further opportunities?

ED: Of course, yes. And many roles will change, it’s not like they will disappear altogether, but the roles will change. What you do in your daily work will change. As you say, project manager, I mean it’s still a leader, we will not be able to change, exchange our leaders for a machine, and leaders and human beings are still in high demand, especially good leaders.

KK: Until we manage people, not machines.

MG: We can have this conversation for a good few hours, but I think it was very interesting and it’s high time to summarise it. Any takeaways for team project management in AI?

KK: There are plenty of them from this conversation, but I would add one more, that you need to talk in your team because I said that the best development, for example at the beginning of the project, is that everyone talks with everyone.

MG: Communication.

KK: Communications, business and developers and quality assurance and project management, they need to talk.

MG: That’s a great point, I love it. Thanks, guys. It was very interesting. And thank you our viewers for watching this episode of IT Leadership Insights by Future Processing. If you found it useful and enjoyed it at least as much as I did, please don’t hesitate to like it and share it and feel free to drop us line and if you want to have another topic covered in further episodes, thank you.