The profiles table previously lacked an index on the user_id column, causing slow query performance when joining with the users table. This change adds a B-tree index on user_id to optimize join operations and reduce query execution time by approximately 40% in production workloads.
The core data ingestion module has been rewritten to replace synchronous file reads and database writes with asynchronous coroutines using asyncio. This change eliminates thread pool overhead for I/O-bound operations, improving throughput under high concurrency. The `process_batch` function now awaits `aiofiles.open` and `asyncpg.execute`, with error handling updated to propagate exceptions through the event loop.
