Complex Data Pipeline with Adaptive Schema Evolution

Design a flexible data pipeline architecture that can: 1) Handle schema evolution across heterogeneous data sources, 2) Implement real-time data transformation and enrichment, 3) Support both batch and streaming processing paradigms, 4) Automatically manage data quality and consistency, and 5) Provide comprehensive lineage and auditing capabilities. Include strategies for handling large-scale data migrations and schema conflicts.