Introduce a comprehensive unit test framework covering the core modules, including new test cases for data validation, error handling, and edge scenarios. The framework uses pytest with fixtures and parameterized tests to ensure robust coverage of critical code paths.
The core data ingestion module has been rewritten to replace synchronous file reads and database writes with asynchronous coroutines using asyncio. This change eliminates thread pool overhead for I/O-bound operations, improving throughput under high concurrency. The `process_batch` function now awaits `aiofiles.open` and `asyncpg.execute`, with error handling updated to propagate exceptions through the event loop.
Implement the `include` directive that allows embedding external template files into the current template at render time. This enables code reuse and separation of concerns by letting templates import shared components like headers, footers, or partials. The feature parses include statements, resolves file paths relative to the template root, and recursively renders the included content within the parent template's variable scope.
Refactor the core data processing module to use asyncio for concurrent task execution, replacing the previous synchronous loop. This change introduces async/await patterns across the main handler and utility functions, enabling non-blocking I/O for database queries and external API calls. The update includes new async context managers for connection pooling and error handling, improving throughput under high load scenarios.
