Auto data lookup is designed to take car sticker data and transform it into a form usable by LLMs. We will also be able to query the data for information about the car data derived from the stickers.
Note that we could use a vision model, such as GPT-4V, to query about the image directly, but it is more useful and less costly to not have to keep querying the vision model. By storing the car data in a traditional RDBMS, we have the flexibility to take advantage of non-LLM queries, which are far more performant and cheaper than their LLM counterparts.
Below is an example car sticker
Not all stickers are exactly the same, but seem to include similar information.
Using a Python script and the GPT-4V API, we can transform this sticker data into something usable.
This will create a CSV with the car data we care about.
Here is some sample car data (with minor alterations for the sake of privacy) derived from the above process
Then we want to be able to talk to our data. For this task, I have chosen to try out PandasAI upon the recommendation of a co-worker. PandasAI essentially looks at the user’s request and the data and tries to create a rational query from the request. PandasAI itself is nice because you can work with data in many forms: Pandas dataframes, Polars dataframes, CSVs, Excel files, Google Sheets, or one of their many SQL connectors.
A quick Python script will give us a Flask backed API to query.
Testing can take place locally. If connecting to an external web frontend, ngrok is a lightweight way to test quickly without the overhead of other services.
We can query PandasAI via the API and start asking questions about the data.