Use this as an engineering review aid rather than a product-selection checklist.
Workload model
- Data volume and growth are measured
- Arrival rate and burst behavior are known
- Required latency is defined per workload
- Batch and streaming requirements are separated
- Interactive and scheduled query patterns are documented
Ingestion
- Source ownership is known
- Schema contracts exist
- Replay / backfill procedure exists
- Kafka topic and partition strategy is documented where applicable
- CDC failure and recovery behavior is understood
Storage and tables
- Durable system of record is identified
- File and table formats are standardized
- Partitioning avoids unnecessary scans and tiny files
- Schema and partition evolution are governed
- Retention and compaction policies exist
Processing
- Spark/Flink workloads have explicit resource models
- Expensive shuffles and stateful operations are understood
- Orchestration is separated from compute
- Idempotency / retry behavior is designed
- Data-quality checks are part of publication
Serving and SQL
- Trino or other query engines have workload isolation
- Connector behavior and source limits are understood
- Query concurrency is capacity-tested
- BI / application SLAs are explicit
Security and governance
- Authentication and authorization are defined end to end
- Sensitive datasets are classified
- Encryption and secret management are documented
- Lineage and ownership are discoverable
- Audit requirements are supported
Operations
- Platform metrics, logs and alerts exist
- Pipeline freshness is monitored
- Cost / capacity signals are visible
- Recovery procedures are rehearsed
- Architecture has named operational owners