David against Goliath - Synerise against the giants of artificial intelligence

Seven minutes was enough for Synerise algorithms to do what it took for the technology created by the leading Google research team - DeepMind - as much as 12 hours to do. Does that mean you are the best in the world? 

Jacek Dąbrowski: We can definitely measure up to the best.  Winning these competitions proves it. The competition you are talking about is the KDD Cup, hosted by Stanford University and Facebook. The task was to predict which of the 153 categories the scientific publications in the collection, which contained 122 million such publications, belonged to. This collection had a graph structure, i.e. at the beginning we had data from the quoted articles and from other publications by the same authors and their affiliations. In fact, we only needed seven minutes to complete this task. Our technology also performed very well in the Twitter RecSys Challenge, where we had to predict user reactions to a specific tweet - we came in second there. In the SIGIR Rakuten Data Challenge, where we took first place, the idea was for the artificial intelligence algorithm to find the right photo of the product - only on the basis of a text description. On the other hand, the challenge of Booking.com AI Challenge was to predict what the next city would be visited by a specific tourist - incl. Based on his travel history, country of origin, and even whether he was using a smartphone or laptop. So the tasks were very varied. 

Michał Daniluk: We don't have a huge financial or hardware base like Nvidia, Google, or Baidu. We cannot, like them, hire hundreds or even thousands of specialists. Google DeepMind used 4 Google Cloud TPUv4 servers in the KDD Cup, which are not currently available to the public. The cost of renting a farm of TPUv3 servers for a year, much weaker than TPUv4, amounts to tens of millions of zloty. The cost of a single GPU card that we used is about PLN 40,000. The interference on 4 such cards took DeepMind 12 hours, us on a single one - 7 minutes. So we were 400 times faster. 

Basia Rychalska: The key is that our models are conceptually different from those presented by the competition. The global players we faced in competitions used algorithms created by a narrow group of institutions, including Carnegie Mellon, Stanford, Facebook AI, DeepMind. These centers are, of course, the elite of the industry. But they notoriously trade employees among themselves, and this makes their perspectives and mindsets broadly similar. In the most important machine learning competitions this year, our competitors did not conceptually present new technologies, creating large versions of already known models. Unlike us. 

What exactly is your technology based on? What does it offer? 

Jacek Dąbrowski: We have a proprietary behavioral modeling engine that supports any data, from any source, with any modalities. Our solution consists of several elements. One of them is Cleora, an open-source vector tool for creating graphs and hypergraphs. Another is EMDE - a model used to estimate density in differential manifolds, i.e., to determine the probability of different events. Simply put, they are tools for describing and modeling behavior, both the behavior of people and any entities, for example, products, stores, or devices. Thanks to these tools, we can model the trajectory of any interaction which is described by an image, sound, or text. 

Michał Daniluk: Basically, Synerise is an operating system for data. Its foundation is a super-fast database called Terrarium. It allows you to collect huge amounts of data, such as data on all transactions from a very large supermarket in the last 10 years. But more importantly, it allows you to perform instant analyses based on this data, in milliseconds. Additionally, it is an execution base. For example, it allows you to measure how many Coca-Cola bottles are sold per hour among customers who spend more than PLN 200 a month in a specific store. This, in turn, allows you to react to a rapidly changing situation on an ongoing basis. Make a quick decision to change the price of a product to balance the sale. This type of activity is possible thanks to Synerise. All in one ecosystem. 

Jacek Dąbrowski: By default, the situation is that someone has a huge database. It stores very large amounts of historical data. Analytical inquiries take several dozen minutes, sometimes several hours. They run once a week or even less frequently. In contrast, execution and any possible business rules to be run in real time must be created on a completely different system. Virtually every such rule must be predicted, designed, and programmed in advance. We do not have it. There is full fluidity and freedom of action. Thanks to our proprietary hybrid transactional and analytical database - the user can apply the rule immediately. In this way, we support, among other things, the Żabka loyalty program. The second pillar of our activity is artificial intelligence and the models that we create thanks to it. They are very universal, so they allow you to predict many events. We are able to predict consumer preferences and their possible behavior. For example, how much a person will spend on shopping in a given category, or which products they may like. We offer all this in the form of a convenient platform with a user interface. In addition, each element is wrapped as an API, which means we become the equivalent of the operating system. And in this system, our user may, among other things, build their own applications. 

You went instantly from Stanford's academic peaks to a very measurable business practice ... 

Basia Rychalska: Because what we do is not strictly academic. We create solutions at the intersection of science and business. At present, most research centers, their conferences and the media deal with algorithms that are impractical. For example, they can only run on very small datasets in a reasonable amount of time. And in companies that generate millions of transactions per month, these datasets are huge. Moreover, scientists often prefer research directions in which they feel best, which creates a fashion for some directions of development and rejection of others. At this point, a favorite topic like this is certain types of artificial neural networks that are getting more and more complex. And this has a negative impact on their applicability in practice. I have the impression that in many areas of machine learning, the divergence between academic and business thinking is growing. Of course, in the scientific world you often have to wait years for a breakthrough, which means that so far purely academic solutions are quickly adapted by business. Perhaps this will also be the case with the currently dominant models. We, however, have discovered an alternative path and are actively exploring it. Our models, although also based on neural networks, contain many unprecedented assumptions and techniques, most of which we created ourselves. Many of them contradict generally accepted truths in our field. And yet it is precisely these assumptions that allow for extraordinary performance and applicability to huge data volumes. 

How long did it take you to reach this level of competence? Did you have any development strategy? 

Jacek Dąbrowski: From the beginning, our ambition was to create a solution that would allow the processing of any heterogeneous data. Data coming from various sources and belonging to various domains, such as textual and visual data or data on user behavior on the Internet, without a predetermined pattern. We wanted to be ready for a priori unknown, any new sources of any data. Very few companies would choose to take on such a challenge. Because it is difficult to achieve - both technically and scientifically, especially with a relatively small team. Today there are 30 people, but we started with 7. If we wanted to create dedicated machine learning models for each industry we serve, we would need to employ about 250 people. Most big companies would. But we had to find another way. And after a deep analysis of the problem, we found that all this data, collected by companies, has one common denominator: describing behavior. This common denominator, which we call behavioral modeling, underlies all of our algorithms. Appropriate generalization of the problem allowed us to unify many sub-disciplines of machine learning and attack them simultaneously - a small but very ambitious team. This strategy paid off fairly quickly. We believe that unsupervised learning and behavioral modeling are one of the key components of the so-called strong artificial intelligence. 

So you chose innovation. From a business point of view, there is probably a certain risk involved?

Basia Rychalska: This is a risk you can't really get away from. At the top of the technological hierarchy are truly innovative organizations that make breakthroughs and shape the optics of entire industries. Their innovations drive adapters and followers. Adapters adapt new technologies to specific applications, thanks to which we have direct contact with technologies in many devices or services. Followers try to copy. There are, of course, many more adapters and followers than innovators. Each technology company has to choose which of these paths it wants to take. Adopting someone else's technology can of course be a viable strategy. It has brought success to many companies. Resources: human, hardware, and even marketing play an important role here. When everyone can take advantage of the same publicly available technology, those who can do it on a larger scale have a better chance. So it is a good path for the organizations of the rich. Those less affluent have to fight on the market, lowering margins and prices, and often fall into the middle-development trap, from which it is extremely difficult to get out. Unfortunately, this is the path that is most often chosen in our region of the world. This is somewhat understandable as it provides a much easier start. There is no big capital investment and no long waits in the uncertainty whether the research will result in something applicable in practice. 

We decided to take a risk. As a company, we do not have the resources of large corporations, so we cannot threaten them with economies of scale. However, competing mainly on cost and price is out of the question for us. So the only option is to create an innovative technology that will make us permanently different. Something that cannot be faked. It is not without reason that innovation is associated with high costs, but paradoxically, the poorer ones (both companies and entire countries) cannot afford the lack of innovation. 

So stand out or die? Does winning competitions against the biggest help? Is it important to you? 

Michał Daniluk: Of course, you always want to win. But these competitions are not a key part of our business. When we entered them, we did not expect to win immediately. We took part in them to prove ourselves. We wanted to verify that what we did was working. And it turned out that it was. 

Basia Rychalska: Such a win adds to our credibility. We present solutions to specific problems that are objectively assessed through automated tests and public rankings. There is no better way to show the market that what we are doing is right. The results of these competitions are an objective measure of what is possible. It is also worth understanding their special importance on a global scale. These are kind of clashes between giants. Innovation leaders such as Google and Facebook set the pace of progress for everyone. Contests are a tool for them to show their strength and confirm that they are at the forefront of the world all the time. Just to win, they employ, among others, multiple world champions in the field of data science. These people and their specially selected teams devote all their time to it. We do everything at once. We run a company, implement projects for clients and at the same time take part in these competitions. All with much less money. The differences in infrastructure between us and them are gigantic, they can be counted in tens of millions of dollars. And yet we burst among the great ones and proved the value of our technology. It is unheard of on a European scale, as the competitions have long been dominated by leading American and Chinese centers. One of our main goals is to contribute to the reversal of this trend. We want to show that Stanford or Carnegie Mellon do not have a mandate to create innovation. There is still a lot to discover in the field of artificial intelligence, and our region is home to many highly qualified and ambitious specialists. If our success gives them courage, it will make the effort worthwhile. 

Jacek Dąbrowski: Success in the international arena translates into greater trust in Synerise as a company, as a technology creator. Potential customers already recognize us. We also get very positive feedback from our current clients. They knew what these competitions were and what the scale of these achievements was. We got letters of congratulations. The issue of future contacts that we have established thanks to these competitions also looks very good because our thinking - our technology - fits in a very important trend. It's about creating solutions that reduce the amount of tedious and repetitive work. That all the things that need to be done happen automatically. Because people's time should be devoted to creative work. It is known that there will be something special in every business that the model will not guess by itself. But this solid foundation can be common to all industries. And this gives us great business opportunities. We have one proven model that can handle a variety of cases. And it does not require enormous effort from us every time. Actually, without modification, we can apply our concept anywhere. And a relatively small team to support a very large number of companies. This is our business advantage. This is what makes us different. 

You have done something else that makes you stand out. You have released one of your tools for free as open source. Has it paid off? 

Basia Rychalska: That's right, we've released Cleora, our graph-embedding library, to the public. We see a lot of potential benefits in this and only a little risk. We have quite a lot of tools that give us an advantage. Cleora is just one of them. Our philosophy is that apart from optimizing the company's development goals, we also want to support progress. 

Michał Daniluk: Besides, it has already given us a lot of publicity. People ask: Why are you sharing this, what is the benefit of it? If the tool is useful, the programmers are eager to join the project. At this point, our tool is used by companies and individual programmers from around the world. They add new functionalities and improve the tool. Some open source projects have just such a business model. Somewhere above everything there is a company that makes money by integrating its solution in the companies of its clients. But it also gets a lot from the community that exists around it. I think that such a gesture of goodwill was worth making. If our tool is used and something cool develops thanks to it, then this in itself is already a value for us. 

The original text was published in Business Insider Polska