2.1 C
London
Tuesday, February 7, 2023

When it comes to large language models, should you build or buy? • londonbusinessblog.com

Must read

RCap Resolution: NCLAT issues notices to Torrent Investments and others at the lender’s plea

The appeals court has ordered the petition to be listed on February 9, 2023 and says the appeal will be heard and decided by...

A single ticket in Washington State wins the $754.6 million jackpot

The lucky owner of a single Powerball ticket in Washington state has won a jackpot worth $754.6 million after matching all six numbers drawn...

WhatsApp allows users to post voice notes as status updates • londonbusinessblog.com

WhatsApp Status, the feature introduced in 2017 as the company's take on Snapchat Stories, has received a number of updates to make it more...

SoftBank loses nearly $6 billion in a quarter as recession continues • londonbusinessblog.com

SoftBank Group's investment vehicles posted a loss of nearly $6 billion in the quarter ended December as the Japanese tech investor continues to bleed...
Shreya Christinahttps://londonbusinessblog.com
Shreya has been with londonbusinessblog.com for 3 years, writing copy for client websites, blog posts, EDMs and other mediums to engage readers and encourage action. By collaborating with clients, our SEO manager and the wider londonbusinessblog.com team, Shreya seeks to understand an audience before creating memorable, persuasive copy.

Last summer could can only be described as an “AI summer”, especially with large language models making an explosive entrance. We saw massive neural networks trained on a massive collection of data capable of performing extraordinarily impressive tasks, none more famous than OpenAI’s GPT-3 and its newer, hyped offspring, ChatGPT.

Companies of all shapes and sizes across industries are scrambling to figure out how to integrate this new technology and extract value from it. But OpenAI’s business model has been no less transformative than its contributions to natural language processing. Unlike almost every previous flagship model release, this one doesn’t come with open-source pre-trained weights — that is, machine learning teams can’t simply download and fine-tune the models for their own use cases.

Instead, they either have to pay to use them as is or pay to fine-tune the models and then pay four times the usage rate to use them. Of course, companies can still opt for other peer open source models.

This has led to an age-old business – but entirely new to ML – question: Would it be better to buy or build this technology?

It is important to note that there is no one-size-fits-all answer to this question; I’m not trying to give a comprehensive answer. I want to highlight the pros and cons of both routes and provide a framework that can help companies evaluate what works for them, while also providing some middle paths that try to incorporate components from both worlds.

Buying: Fast, but with obvious pitfalls

While building looks attractive in the long run, it requires leadership with a strong risk appetite, as well as a large treasury to support that appetite.

Let’s start buying. There is a slew of model-as-a-service providers that offer custom models as APIs, charging per request. This approach is fast, reliable and requires little to no upfront capital outlay. In fact, this approach reduces machine learning projects, especially for companies entering the domain, and requires limited in-house expertise beyond software engineers.

Projects can be started without the need for experienced machine learning personnel, and the model results can be fairly predictable as the ML component is purchased with a set of guarantees around the output.

Unfortunately, this approach has very obvious pitfalls, including limited product defensibility. If you buy a model that anyone can buy and integrate into your systems, it’s not too far-fetched to assume that your competitors can achieve product parity just as quickly and reliably. That will be true unless you can create an upstream moat through non-replicable data collection techniques or a downstream moat through integrations.

In addition, for large-scale high-throughput solutions, this approach can prove to be extremely expensive. For context, OpenAI’s DaVinci costs $0.02 per thousand tokens. If you conservatively assume 250 tokens per request and responses of similar size, you will pay $0.01 per request. For a product with 100,000 requests per day, you pay more than $300,000 per year. Obviously, text-heavy applications (trying to generate an article or chatting) would lead to even higher costs.

You should also consider the limited flexibility associated with this approach: you use models as they are or pay significantly more to fine-tune them. It’s worth remembering that the latter approach would involve an unspoken “lock-in” period with the provider, as refined models are held in their digital custody, not yours.

Build: Flexible and defensible, but expensive and risky

On the other hand, building your own technology allows you to get around some of these challenges.

More articles

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Latest article

RCap Resolution: NCLAT issues notices to Torrent Investments and others at the lender’s plea

The appeals court has ordered the petition to be listed on February 9, 2023 and says the appeal will be heard and decided by...

A single ticket in Washington State wins the $754.6 million jackpot

The lucky owner of a single Powerball ticket in Washington state has won a jackpot worth $754.6 million after matching all six numbers drawn...

WhatsApp allows users to post voice notes as status updates • londonbusinessblog.com

WhatsApp Status, the feature introduced in 2017 as the company's take on Snapchat Stories, has received a number of updates to make it more...

SoftBank loses nearly $6 billion in a quarter as recession continues • londonbusinessblog.com

SoftBank Group's investment vehicles posted a loss of nearly $6 billion in the quarter ended December as the Japanese tech investor continues to bleed...

Angel Iris Murphy Brown: all about Eddie Murphy and Mel B’s daughter

Angel Iris Murphy Brown is the daughter of the American writer, comedian, producer and singer Eddie Murphy (aka Edward Regan Murphy). Her father...