AI ERA: what LLM model to use?
Fina Money uses LLM to power up its answer to users’ financial questions.
We have started using OpenAI’s API, but quickly we observed the slow response, especially with GPT-4 model, it makes me think, are there any alternatives that we may consider to balance the workload?
Faster but the same quality
However, not all LLM models have the same quality to achieve the accuracy we want, this lead me to test out a list of models available, and to gain a sense of what the landscape looks like regarding “Accuracy + Speed”
To make it short, I would stay with GPT-4-Turbo for now. Though the speed is still a concern, but literally there is no another one that could replace, here is my test report to share with everyone, if you are looking at the same problem for your APP integration, it maybe useful, the list of models in use:
- gpt-4-turbo
- gpt-3.5-turbo
- llama3–8b-8192
- llama3–70b-8192
- gemma-7b-it
- mixtral-8x7b-32768
The whole report is reproducible in below Fina doc, if you want to try it out by yourself, you just need sign up free account, and import the page to your account to play.
LLM Models Test Report https://app.fina.money/doc/jM8LYvPkm07xxg
Let me know if you have any questions, cheers!
