Systems Advisory
Data systems for network intelligence
Ten years of pipelines and warehouses, pointed at graph-shaped problems: public-web ingest, network storage, and the audits that keep the bill honest. Same craft as the public research. Different invoice.
Databricks CertifiedDataCampSQL Associate
Book a CallServices
What the work covers
Pipeline & Warehouse Engineering
Production warehouses and ELT that survive load. Models, jobs, and the failure modes, designed as a system.
- Snowflake, Databricks, AWS
- dbt models and tests
- Airbyte and automated ELT
- Lakehouse and warehouse layout
Telemetry & Ingestion Architecture
Collection paths for high-volume events, public-web firehoses, and graph-shaped research data.
- Custom scrape and ingest paths
- Graph and network storage
- Real-time event processing
- Parquet, DuckDB, Spark
System Audits & Optimization
Read the warehouse, the spend, and the query plan. Cut cost. Fix what is actually slow.
- Query performance tuning
- Data modeling reviews
- Cloud cost reduction
- Incident and correctness audits
Process
How it works
02
Proposal and start
If it fits, a proposal follows and work starts. If it does not, that gets said plainly.
Stack
Tech stack
Cloud & Warehouses
- AWS
- Snowflake
- Databricks
- DuckDB
- BigQuery
Pipelines & Ingest
- dbt
- Airbyte
- Airflow
- Kafka
- Spark
Graphs & Telemetry
- NetworkX
- GraphFrames
- Python
- Parquet
Analytics
- SQL
- Hex
- Metabase
- Power BI
Book a Call
60 minutes. No sales pitch.
Pick a time. Confirmation comes once both sides are clear. If it is not a fit, that gets said.