Is your feature request related to a problem?
The database currently retains llm call rows, which can lead to unnecessary data accumulation.
Describe the solution you'd like
- Implement a cron job in Kaapi to clean up user queries and llm responses older than a configurable value (1 hr, 1 day, 1 week, etc.).
- Retain llm call metadata for debugging purposes but limit the retention of complete user queries and llm responses to a maximum of 2-3 days.
- Ensure voicenotes remain in S3 for at least the TTL (1 hour) when using pre-signed URLs for TTS responses.
Original issue
-
Kaapi will keep only the llm call metadata and run a cron job to clean up the user query and llm response which are older than a configurable value (1 hr, 1 day, 1 week etc)
-
Reasons why we cannot immediately drop the messages:
- for tts response, if we send back a pre-signed url of the voicenote, we'll need to keep the voicenote in s3 for atleast the time period of the TTL (1 hour currently)
- debugging: on rare occassions, the complete user query and llm response has helped us debug certain issues with glific; but we only need that data for 2-3 days at max beyond which we can clean up the user query and llm response
Is your feature request related to a problem?
The database currently retains llm call rows, which can lead to unnecessary data accumulation.
Describe the solution you'd like
Original issue
Kaapi will keep only the llm call metadata and run a cron job to clean up the user query and llm response which are older than a configurable value (1 hr, 1 day, 1 week etc)
Reasons why we cannot immediately drop the messages:
- for tts response, if we send back a pre-signed url of the voicenote, we'll need to keep the voicenote in s3 for atleast the time period of the TTL (1 hour currently)
- debugging: on rare occassions, the complete user query and llm response has helped us debug certain issues with glific; but we only need that data for 2-3 days at max beyond which we can clean up the user query and llm response