ML is an invaluable asset for modern businesses across the board. However, when it comes to ML models, both B2C and B2B companies are faced with a dilemma of a period of time to market. According to Algorithmia, a vast majority of companies take at least a month or longer to first develop and then deploy their ML model.
This is a complex and often very costly two-stage process. It can be a lengthy and potentially costly process in and of itself. However, many businesses may not realize that the initial stage must be followed by another, potentially more challenging phase deployment. This second stage involves moving the ready model to production, testing, and tweaking it, and then scaling up accordingly.
Only around 10% of all firms are expected to have adequate training, financial resources, and technical expertise to deliver a fresh ML model to production within a week after its completion. Many businesses struggle for up to a year, with at least 30% of all companies taking at least three months post-deployment. How long does exactly matter based on which of the three popular model types the company chooses.
Custom and off-the-shelf adaptive models
Of the ML models currently available on the market, there are three types of models: generic models, custom models, and custom adaptive models.
Transformation in 2022
Join us in the live chat from July 19 to July 20-28 at the leading event on applied AI for enterprise business and technology decision makers.
Generic and custom models are basically polar opposites. Generic models are low in both cost and accuracy, while custom models are high in both cost and accuracy. This is because generic models are designed to suit virtually every company within the organization. These are normally based on ResNet, BERT, andGPT, but these are also far from being a perfect fit.
Custom models are usually tailored to the task at hand, thus they can be significantly reduced due to their high development and maintenance costs. However, those who start with a generic solution and then attempt to improve their ML model venture beyond the models'' basic architecture. What they eventually end up with is a custom adaptive model.
A adaptive model is basically a custom model with several benefits that generic models offer. The adaptive models are created with specific business requirements in mind. In this way, they don''t require that the company execute the MLops during the initial deployment and post-deployment stages. This way, they in some ways operate like generic models, with relatively low maintenance costs and improved time to market.
Choosing an ML model
It depends on your business''s style, whether paying extra is worthwhile the stretch. It may be helpful to send online orders to different warehouses depending on their location. In this case, a generic ML model might just do the trick, especially if your small business is.
A customized model will, on the other hand, work if something specific, such as content moderation for an online community of doctors discussing medical equipment. For example, mentions of genitalia are not only appropriate but necessary in the context of medical discussion. This tailor-made model can be either adaptive or not.
We should look at the pros and cons of each model:
Custom adaptive models
Due to the often unforeseen pre- and post-deployment costs, some businesses prefer to stick away from the tailor-made approach rather than using the less precise but also less costly generic track. The cost of a training model actually gets depends on a variety of factors, including the chosen data-labeling methodology.
The following example illustrates a crowdsourcing-based custom-adaptive model in action, c. a.e., a adaptive model that is based on human-in-the-loop labeling:
One well-known business that offers a technical editing environment wanted to boost its softwares accuracy and decrease the model training costs. The engineering team had to come up with a more effective method for correcting sentences in English. Any solution had to be in line with a fully manual labeling pipeline that was already in place.
The final solution consisted of using a pre-existing custom model for text classification within the target sentences. Upon completion, phrase verification accuracy increased by 6% from 76% to 82%. This reduced the model training costs by 3%. Additionally, the client did not need additional investment into the models infrastructure, as is normally the case with most custom models.
Key points to keep in mind
It may seem difficult to select the right ML model for your organization. Here''s a list of what you should take into account to make an informed decision:
- Always consider scalability if thats something you know youll need, consider paying extra for something tailor-made just for you.
- If you dont require high accuracy but need fast deployment, consider opting for the generic route.
- If accuracy is important to you, consider how much time to market you can spare.
- If youre short on time and require high accuracy, consider taking the custom adaptive route; otherwise, any custom solution can potentially fulfill your needs just as well.
- In terms of the overall cost, the generic route is the cheapest of all followed by the custom adaptive route that bypasses most MLops expenses and finally by all other custom solutions whose costs may rise substantially post-deployment (the exact figures differ greatly on a case-by-case basis).
- Consider whether you have in-house data scientists and MLEs at your disposal if yes, going for the traditional custom option developed internally may be feasible; if not consider the other two (generic or custom adaptive).
- When choosing between custom vs. custom adaptive options, consider how accurate and specific to the needs of your customer the ML model ultimately has to be. The higher the accuracy and adaptability, the higher the cost and longer the waiting period to prepare and maintain the model.
At Toloka AI, Fedor Zhdanov is the head of all ML services.