Interview with David Cournapeau, Head of the MLE Team: Engineering Products from Research Results
This is the third installment in a series of interview articles with David Cournapeau, head of the Machine Learning Engineering Team at Cogent Labs. This time, David discusses the product development journey, from the original research to the finalized product, and the challenges along the way. This is a topic he is intimately familiar with, having worked for many years at the interface between research and engineering.
- Topic 2: Engineering Products from Research Results
- Q: Shortly after joining Cogent Labs you launched the Machine Learning Engineering Team to work at the interface between research and software engineering. Could you explain the rationale for doing so?
- Q: Could you explain some of the different strengths and weaknesses, as well as different ways of working between researchers and engineers?
- Q: From your experience, what are some of the challenges this poses?
- Q: How have you tried to overcome these challenges and make sure that research is being fed into product development and vice versa?
- Q: Could you share some concrete examples of how you have developed products at Cogent Labs?
- Q: From your experience, are there any ideal processes for developing new products, or any general qualities that make AI or machine learning products more usable or useful?
Topic 2: Engineering Products from Research Results
Q: Shortly after joining Cogent Labs you launched the Machine Learning Engineering Team to work at the interface between research and software engineering. Could you explain the rationale for doing so?
DC: As I mentioned previously, often at big companies you have separate teams working on research and engineering. This means that it can take several years to turn a piece of research into a product. As a small company, we cannot afford to do that. Ideally, we should be working together, not just to speed up the process but also to enhance the quality of communication. Furthermore, this allows us not only to develop products based on the research but also start driving some of the research from the product as well. This would not be possible if we worked in completely isolated environments.
Q: Could you explain some of the different strengths and weaknesses, as well as different ways of working between researchers and engineers?
DC: I have found that for researchers or scientists, when they develop a piece of software, they are not interested in the software itself. It is merely a means to an end. For them, the value lies in the output of the software. They want to be able to change everything all the time to get the output they desire, and when they are done with the software they may simply throw it away. In a similar vein, they are sometimes not very interested in working on the product itself.
For engineers, a piece of software is a living thing. They see it as something that needs to be maintained for a long time because customers are using it. They therefore want to be able to control and constrain it. Researchers want the exact opposite.
As an example, imagine using a regular car to commute to work every day versus a Formula 1 car. They are both cars but their uses are completely different. You can mass-produce a Prius and regular people can use them easily. On the other hand, a Formula 1 car may be very fast but needs around 50 people to maintain it and a world-class pilot to drive it. It does not make sense to go to work in a Formula 1 car.
Q: From your experience, what are some of the challenges this poses?
DC: It is not as if one approach is right and one approach is wrong but fundamentally, the researchers and engineers have different goals. A lot of people who do not have a technical background do not realize this. Sometimes even the researchers themselves do not realize this and wonder why the engineers they are working with are making their lives difficult. This difference in goals is something that needs to be managed.
Of course, it is important to have the input of researchers for developing the product. You also want some input from the products to drive research. However, research and product development operate on different timescales. We are working on products on a day-to-day basis and, especially since we are a small company, the timeline for new features is often a matter of months or weeks. Research, meanwhile, operates on much longer time horizons. Managing this difference in timescales is another major challenge.
Q: How have you tried to overcome these challenges and make sure that research is being fed into product development and vice versa?
DC: We have tried to get the researchers and engineers to work closely together. We make sure the Machine Learning Engineering Team is not closed off physically to the researchers and we create some forums where people can exchange ideas. For example, some researchers organize some paper reading sessions that machine learning engineers can join. By having people who understand enough about deep learning on the Machine Learning Engineering Team we can ensure that the necessary serendipity is occurring.
We also sometimes ask some machine learning engineers to work on something that is less defined and not directly linked to a product. They would do that together with researchers. For example, we have one research project with two machine learning engineers working directly with a researcher. They meet a few times a week, working on various research or papers, and engaging in closer dialogue.
Another effort we are making, but which I think we can still improve, is sometimes embedding some researchers into the products, if there is a specific need. For example, if we need to enhance the AI / deep learning side of a product, we may embed a researcher who can really think more deeply about it than an engineer.
DC: Our three main products are TSF, Tegaki and Kaidoku. I mainly worked on the first two. TSF started out as a proof of concept for a client. This was before I joined but we basically had one research engineer working with the client to use market data to predict the volume of trading on the Tokyo Stock Exchange. We received some data from the client and were asked to develop something that could do a better job of forecasting the volume than what the client was already using. We were successful and were then asked to develop a product or web service that could be used every day.
That is when I got involved. We would no longer be working with offline data on our own machines. We would instead be connected to the Tokyo Stock Exchange and getting data delivered in real time. I worked on this for three or four months with another engineer and the research engineer who created part of the model. At this stage of the project, the challenge was much more about the engineering and less about the modeling side.
As for Tegaki, this was a much bigger project that had just launched when I joined the company. At that time, there was a small team of researchers and engineers working together on it but the arrangement was not working very well. It was a pattern that I have seen many times before. There were no machine learning engineers yet and that is partly what motivated me to create a different team of software engineers who understand enough about the deep learning to be able to hold in-depth discussions with the researchers.
Another challenge, which is quite a typical one, was that a lot of the code and AI was originally written by researchers, but only the researchers understood it. To make sure that others could understand the model I had some of the machine learning engineers basically restructure the code to make it easier to understand, and to be able to modify and improve it.
Q: From your experience, are there any ideal processes for developing new products, or any general qualities that make AI or machine learning products more usable or useful?
DC: First, there is no single process. The processes for developing Tegaki, TSF and Kaidoku were all quite different. TSF was very business-driven. Kaidoku was the exact opposite and was very much research-driven. Tegaki, meanwhile, was a bit of both.
In general, if you try to have multiple products, especially as a small company, I think you need to be opportunistic. You cannot have just one process. As for product types, I think it is dangerous to build your product just around AI. It is risky to start from the technical side and then try to work out how to make something that has some business value. I think that is a mistake that a lot of people make, especially people who recently moved from academia to business, since that is their natural inclination.
It is much more interesting to look at some concrete business problems that are very difficult to fix traditionally, and think about how they can be solved through deep learning or machine learning. If you figure that out, I think you have a business. At the end of the day, the question is less about the AI and more about what the value proposition is.
A lot of people have this fantastic image of AI but it is not yet powerful enough to do many of the things they imagine. There are many more opportunities in concrete applications. This is a clichéd example, but a good one: how can you analyze satellite images to have a better understanding of weather patterns or automatically discover where people are cultivating drugs, etc.? Those things have all been done before but not very efficiently. With AI, you can do them systematically. The systematic aspect is very important and is where I think we will see a lot of companies successfully using AI. I think too many people have this idea that most of the money is in self-driving cars or something cool like that. I am not convinced. I think most of the opportunity is in things that are, for lack of a better word, less “sexy” but where AI can really make a difference.
Software engineering is a fairly old practice now but everything about making AI-based products is still pretty new. Even the best companies at building products based on AI are still figuring it out. There is no book about how to build AI products or at least, if there are, I doubt they are very good. I think we have not found the recipe yet. On the one hand that means we have more freedom, but on the other it also makes the process more difficult. That is what makes working at a company like Cogent Labs so interesting.