Token Usage and Databases - Local vs. API

r/LocalLLaMA
Generative AI NLP

Throwing something out to the community for a bit of an insight. I got thinking about the consumption of tokens when working with various databases and here is my understanding: When I ask as question that is essentially converted to tokens. The LLM then "reads" that and generates the response which in this cases involves a database query The LLM then tokenizes the query results and "reads" them and provides me the results and any insights or answers Rinse and repeat until you have gotten what you want. i.e continue to build token usage.