Infographics & Architecture Gallery
Visual reference guides, architecture diagrams, and educational infographics created by Alex Merced for Data & AI engineers.
The Open Lakehouse Architecture
High-level architectural stack of the Open Lakehouse featuring Ossie, Polaris Catalog, Apache Iceberg, Parquet, Apache Arrow, and object storage.
End-to-End: Operational Data to AI Insights
Continuous flow of trusted data from operational databases via CDC into an Apache Iceberg lakehouse powering analytics and AI applications.
The Architecture of Apache Iceberg
Detailed internal breakdown of Apache Iceberg: Catalogs, Table Metadata Layer (JSON, Snapshots, Manifest Lists), Data File Layer, and Compute Engine integrations.
Apache Spark: Architecture & Functionality Overview
Unified analytics engine: Driver Program, Cluster Manager, Executors, Catalyst Optimizer, Tungsten Execution Engine, Spark SQL/Streaming/MLlib/GraphX, and data abstractions.
Apache Flink: Architecture & Functionality Overview
Stateful stream and batch processing engine: JobManager, TaskManager, RocksDB state backends, event time watermarking, PyFlink, Table API & SQL.
Apache DataFusion: Open, Modular Query Engine
High-performance Rust-based query engine leveraging Apache Arrow: query execution lifecycle, logical/physical planning, pushdown optimizations, and integrations.
Data Modeling: Structure Data. Align Meaning. Drive Value.
Comprehensive visual guide to data modeling: Conceptual, Logical, Physical, and Implementation levels, Dimensional vs 3NF vs Data Vault modeling, and 6-step workflow.
AI Terminology: Clear Definitions & Mental Models
Visual guide to essential AI terms: Models, Tokens, Context Windows, AI Harness, Memory (Short/Long-term), Training, Fine-Tuning, RAG, and Semantic Layers.
Inside an Apache Iceberg metadata.json (V3)
Internal JSON schema of Iceberg V3 table metadata: table UUID, location, schemas, partition specs, sort orders, snapshot logs, refs, and delete vector settings.
Inside an Apache Iceberg V3 Manifest List
Avro-encoded index pointing to manifest files per snapshot: sequence numbers, added/existing/deleted file counts, partition summaries, and aggregate column stats.
Inside an Apache Iceberg V3 Manifest File
File-level metadata listing data files: column sizes, value counts, null/nan counts, lower/upper bounds, and delete vector references for fast query planning.
Apache Iceberg V3: Equality Deletes vs Deletion Vectors
In-depth comparison of Iceberg V3 row-level delete mechanisms: Equality Deletes (value-level) vs Deletion Vectors (bitsets), query performance, and sweet spots.
Apache Iceberg Snapshot Management
Managing snapshots for time travel, data protection, performance, and cost control: time-based, count-based, hybrid, and reference-aware retention.
Apache Iceberg Table Compaction
Solving the small file problem: Full compaction, partition compaction, incremental/rolling, size-tiered, and sort-assisted compaction techniques.
Apache Iceberg Table Maintenance Operations
Core maintenance operations: Optimize files, Vacuum, Expire Snapshots, Rewrite Data Files, Recalculate Stats, Clean Orphan Files, and Rebuild Manifests.
Streaming to an Apache Iceberg Lakehouse
Approaches, patterns, and best practices for streaming into Iceberg: Engine-native (Flink/Spark), Landing zone microbatch, CDC replication, and Append-only.
The Architecture of Apache Polaris
An architectural overview of Apache Polaris Catalog: API layer, Metadata service layer, pluggable persistence, extension framework, and engine ecosystem.
The Architecture of Apache Arrow
Columnar in-memory data layout, validity bitmaps, offset buffers, data buffers, and the Apache Arrow ecosystem across languages and engines.
The Architecture of an Apache Parquet File
Deep dive into Apache Parquet file layout: File headers, horizontal Row Groups, vertical Column Chunks, statistics, and File Footers.
The Architecture of Apache Ossie (Incubating)
The open standard for semantic models in the Open Lakehouse: model structure, JSON serialization, transport, discovery, and projections.
The Architecture of Model Context Protocol (MCP)
2026 specification breakdown of Model Context Protocol (MCP): hosts, stateless core, extensions framework, MCP servers, and enterprise integrations.
Be More Productive with AI and Agents
Overview of Alex Merced's 6 practical AI guides covering OpenAI Codex, Claude Code, Open Models, Google AI, Microsoft Copilot, and Grok & Cursor.