AI ERA: what LLM model to use?

Test on a list of models

Shawn Cao
2 min readMay 15, 2024
Photo by Gerard Siderius on Unsplash

Fina Money uses LLM to power up its answer to users’ financial questions.

We have started using OpenAI’s API, but quickly we observed the slow response, especially with GPT-4 model, it makes me think, are there any alternatives that we may consider to balance the workload?

Faster but the same quality

However, not all LLM models have the same quality to achieve the accuracy we want, this lead me to test out a list of models available, and to gain a sense of what the landscape looks like regarding “Accuracy + Speed

To make it short, I would stay with GPT-4-Turbo for now. Though the speed is still a concern, but literally there is no another one that could replace, here is my test report to share with everyone, if you are looking at the same problem for your APP integration, it maybe useful, the list of models in use:

  • gpt-4-turbo
  • gpt-3.5-turbo
  • llama3–8b-8192
  • llama3–70b-8192
  • gemma-7b-it
  • mixtral-8x7b-32768

The whole report is reproducible in below Fina doc, if you want to try it out by yourself, you just need sign up free account, and import the page to your account to play.

LLM Models Test Report https://app.fina.money/doc/jM8LYvPkm07xxg

Let me know if you have any questions, cheers!

--

--

Shawn Cao
Shawn Cao

Written by Shawn Cao

Share learnings to empower others. I write about startups, entrepreneurship, technology, management, finance, data, politics, people…

No responses yet