System Architecture: Top-Down View
graph TD
subgraph Host ["Level 0: Host Machine (Your Laptop)"]
direction TB
subgraph Runtimes ["Level 1: Runtimes & Infrastructure"]
direction LR
Docker["Docker Engine
(External DB Container)"]
K3sSvc["K3s System Service
(Cluster Brain)"]
end
subgraph Cluster ["Level 2: K3s Virtual Cluster"]
direction TB
Node["K3s Single Node"]
subgraph Namespaces ["Level 3: Resource Isolation"]
subgraph etl_poc ["Namespace: etl-poc"]
direction TB
MSSQL_Comp["MSSQL Pod"]
ETL_Comp["ETL Puller (CronJob)"]
Storage[("MSSQL Data (PVC)")]
MSSQL_Comp --- Storage
end
subgraph kube_system ["Namespace: kube-system"]
Logging["Fluent Bit (Logging)"]
Monitoring["Monitoring Layer
(P&L&G Stack)"]
Metrics["Kube-State-Metrics"]
Networking["CoreDNS / Traefik"]
end
end
end
ExternalDB[("Postgres DB
(External Source)")]
Docker --- ExternalDB
Browser["User Browser
(Grafana UI)"]
end
ETL_Comp -.->|Pull| ExternalDB
ETL_Comp -.->|Push| MSSQL_Comp
Logging ===>|Push logs| Monitoring
Monitoring -.->|Scrape| Metrics
Browser -.->|View Live| Monitoring
Resource Breakdown
| Component |
Namespace |
Statefulness |
Role |
| ETL App |
etl-poc |
Stateless |
Processes and filters data. |
| MSSQL |
etl-poc |
Stateful |
Internal destination database. |
| Prometheus |
kube-system |
Stateful |
Metric aggregation & storage. |
| Loki |
kube-system |
Stateful |
Log aggregation & storage. |
| Grafana |
kube-system |
Stateless |
Live Visualization & Dashboards. |
| Fluent Bit |
kube-system |
Stateless |
Collects container logs. |
| KSM |
kube-system |
Stateless |
K8s cluster state metrics. |
| External DB |
Docker |
STATEFUL |
Source data simulation. |