BestViewsReviews

Shorthills AI built BestViewsReviews a GenAI platform that analyses unstructured e-commerce product information like customer reviews, blogs, manufacturer written product description, etc. and generates insights and product recommendations. The idea is to scale GenAI to analyse information for millions of products spread across 1000s of categories.

Problem

Scale the analysis of unstructured/semi-structured language information (reviews) on the internet.

Our journey

We started with small NLP models training to solve specific tasks – initially GloVe, and U.S.E for vector embedding and semantic search; BERT for classification; T5 for text2text; and then GPT2 for generation.

Old generation smaller models had performance and accuracy issues. Deploying and maintaining multiple models was also a challenge.

Dawn of Large Language Models (GPT 3.x)

When we got early access to GPT 3 and 3.5, it showed improvements. We built a successful Proof of Concept (PoC) where a single model could handle multiple tasks with fair accuracy. However, we found that it was difficult to scale, slow to respond, and expensive. The structure, data, and training methodology were also not open.

Migrating and Scaling with Llama

What excited our team the most was that Llama models were:

Performing at par with the larger model

Open source Models

Commercially viable

Scalable and Versatile

We migrated our production workload to Llama 2 (and now Llama 3). It was not easy to build, scale, and deploy the pipelines, but the support from Meta and the community is awesome. A great off-the-shelf model, fine-tuning scripts are available and can be customized easily, deployment support from open libraries like vLLM, Nvidia RT, choice of cloud provider, makes it the best choice to scale your workload.

With the right expertise to fine-tune the model, and efficient deployment Llama 3 gives you a good shot at driving ROI for GenAI projects. We are proud to be powered by Llama 3!

High Level Architecture Diagram