mmrajput.github.io/project-portfolio

GHGA · Research Data Infrastructure

Proposed platform architecture · Research Data Infrastructure Engineer · DKFZ / GHGA Heidelberg

German Human Genome-Phenome Archive NFDI · EGA · GDI · genomDE GA4GH standards · FAIR data
Covered — homelab or professional exp.
Gap — ramp-up required (short timeline)
External network / partner
Domain-specific (GHGA / GA4GH)
External genomic networks & data submitters EGA · GDI · genomDE · GHGA data hubs · researchers
EGA (EMBL-EBI)
European Genome-Phenome Archive · federated node
GA4GH DRS
GDI network
1 Million Genomes initiative · EU member nodes
GDI
genomDE
German national sequencing programme · BfArM
national
GHGA data hubs
Partner institutions · genome data centres
federated
Researchers
Data access requests · analysis submissions
DAR / DAC
Data submitters
Clinical centres · sequencing facilities
upload API
Edge & authentication / authorisation infrastructure (AAI) Federated identity · LS Login · ELIXIR AAI · OAuth2/OIDC · GA4GH Passport
gap
LS Login / ELIXIR AAI
Life-sciences federated IdP · GA4GH Passport tokens · institutional identity
life-sciences AAI
Keycloak 26.x
Internal OIDC broker · ELIXIR AAI upstream IdP · group-based RBAC · MFA
OIDC broker
nginx-ingress
Hostname-based routing · TLS termination · OAuth2 proxy integration
IngressClass
cert-manager
Let's Encrypt · DNS-01 · mTLS between services · webhook TLS
TLS · PKI
Cloudflare / WAF
Edge filtering · DDoS protection · Zero Trust perimeter for admin UIs
edge
GitOps & infrastructure as code All cluster state in Git · ArgoCD App-of-Apps · Pulumi · Helm · no manual kubectl apply
ArgoCD
App-of-Apps pattern · multi-cluster · self-managed · GitOps engine
GitOps
Helm charts
Pinned versions · multi-source apps · templated values per environment
package mgmt
gap
Pulumi (Python)
Cloud resource IaC in Python · replaces Terraform for cloud provisioning
IaC · Python SDK
Terraform / OpenTofu
Existing IaC experience · AWS provider · Boto3 · Hetzner / Cloudflare DNS
IaC
GitHub Actions
CI/CD pipelines · ARC self-hosted runner in K8s · image build & promote
CI/CD
gap
GitLab CI
Transferable from GH Actions · pipeline authoring · runner configuration
CI/CD alt
GA4GH standards · FastAPI microservices layer DRS · Beacon · Crypt4GH · metadata · data access control · Python / domain-driven design
GA4GH standard services
DRS service
Data Repository Service · GA4GH standard · object URL resolution
GA4GH DRS
Beacon API
Variant discovery queries · federated beacon network · access-controlled responses
GA4GH Beacon v2
Crypt4GH service
File-level encryption for genomic data · GA4GH file encryption standard
GA4GH Crypt4GH
Access control svc
DAC / DAR workflow · GA4GH Passport visa evaluation · policy enforcement
DAC · Passport
Platform microservices (FastAPI + Python)
Upload service
Async upload pipeline · Crypt4GH encryption in transit · checksum validation
FastAPI · async
Download service
Authorised data retrieval · GA4GH Passport check · presigned URL generation
FastAPI · async
Metadata service
GHGA metadata schema · FAIR principles · JSON-LD / OpenAPI spec
FAIR · OpenAPI
gap
Domain-driven design
Bounded contexts · aggregate roots · event-driven patterns — learnable
DDD · patterns
Python backend
Professional FastAPI / async experience · REST API design · Pydantic models
Python · REST
Container pipeline
crane image promote · Trivy CVE scan · OCI image signing · ghcr.io
crane · Trivy
Kubernetes platform layer On-prem DKFZ cluster · commercial cloud burst · kubeadm · CNI · RBAC · PSS
kubeadm cluster
On-prem DKFZ · v1.31 · 3-node production pattern · control-plane HA option
K8s v1.31
Talos Linux
Immutable, API-driven OS · hardened nodes · no SSH · next cluster target
immutable OS
gap
OpenStack / de:hub
Academic cloud infrastructure · DKFZ internal cloud · GPU node access
academic cloud
AWS EKS / Azure AKS
4 yrs professional cloud experience · managed K8s · EKS burst workloads
managed K8s
Calico CNI
Default-deny NetworkPolicy · pre-DNAT egress rules · production-hardened
CNI · NetworkPolicy
Pod Security Standards
restricted enforced per namespace · baseline for operators · audit logging
PSS · restricted
RBAC hardening
Least-privilege service accounts · per-workload ClusterRole · no wildcards
RBAC
External Secrets Op.
Vault KV v2 → K8s secrets sync · ESO v2.1.0 · ClusterSecretStore
ESO v2.1.0
Storage & data layer Encrypted genomic data at rest · sovereign storage within DE/EU · PITR · object storage
gap
Ceph / large-scale S3
Petabyte-scale object storage for genomic files · RADOS · erasure coding
object storage
MinIO
S3-compatible · backups · Velero backend · Loki long-term storage
S3-compat
CloudNativePG
Metadata PostgreSQL · WAL archiving · PITR · automated failover · TLS
WAL · PITR
Longhorn
Platform PVs · RF=2 · default StorageClass · Velero integration
RF=2 · PV
Velero
Cluster backup & restore · MinIO S3 backend · schedule CRDs
backup
Crypt4GH at rest
File-level encryption for all genomic objects · GA4GH standard · key mgmt
GA4GH enc.
Secrets & configuration management Zero static secrets in Git · rotation · audit trail · Vault KV v2
HashiCorp Vault
KV v2 · dynamic secrets · Longhorn PVC · standalone · ESO ClusterSecretStore
Vault KV v2
External Secrets Op.
Vault → K8s secrets sync · argocd label pattern · per-namespace stores
ESO v2.1.0
gap
Pulumi ESC
Pulumi Environments, Secrets & Config · IaC-native secret management
Pulumi secrets
SOPS / Sealed Secrets
Git-safe encrypted references · age/GPG · no plain secrets in repo
GitOps-safe
Observability layer Metrics · logs · traces · SLA dashboards · alerting · AI cluster analysis
Prometheus
kube-prometheus-stack · ServiceMonitor CRDs · long-term via Mimir
metrics
Grafana
Dashboards · OIDC SSO via Keycloak · Kubernetes panels · custom dashboards
OIDC SSO
Loki + Mimir
Log aggregation · Promtail DaemonSet · long-term metrics storage
logs · long-term
Alertmanager
Alert routing · on-call notification · inhibition rules
alerting
K8sGPT + HolmesGPT
AI-powered cluster analysis · Ollama backend · root-cause suggestions
AI ops
Uptime Kuma
SLA / uptime monitoring · public status page · endpoint health checks
SLA
Runtime security & compliance Falco · Trivy · NetworkPolicy · BSI IT-Grundschutz · DSGVO · ISO 27001 controls
Falco
Runtime syscall anomaly detection · DaemonSet · Falco rules → Loki alerts
runtime security
Trivy + Grype
Image CVE scanning · block HIGH/CRITICAL in CI · SBOM generation
CVE scan
NetworkPolicy
Default-deny posture · Calico enforcement · pre-DNAT egress control
default-deny
kube-bench
CIS Kubernetes benchmark · remediation report · hardening validation
CIS benchmark
BSI IT-Grundschutz
German federal security baseline · SYS.1.6 container module · DSGVO Art. 9
BSI · DSGVO
gap
ISO 27001 formal
Audit process · ISMS documentation · control mapping — conceptually familiar
ISMS · audit
CI/CD · image promotion pipeline Staging: automated promotion · Production: manual Git tag gate · Trivy CVE block
Staging — automated
GH / GL CIbuild + test
Trivy scanblock HIGH/CRIT
crane promotepush to registry
Helm values updatetag → main commit
ArgoCD sync→ staging ns
Production — manual gate
Validate staginghuman decision
Git tage.g. v1.2.0
Helm values updateproduction-values.yaml
ArgoCD sync→ production ns
Data sovereignty envelope DSGVO Art. 9 · sensitive genomic data BSI IT-Grundschutz SYS.1.6 Schrems II · DE/EU jurisdiction ISO 27001 controls Crypt4GH file encryption GA4GH Passport verification Audit log → Loki Access control trail