I designed a pipeline that models cross-chain transactions , reconciling the txn from source chain to destination chain.
Agents & analysts can then know exactly what a txn is.
github.com/Freemandaily/cros…
Access to blockchain data has hit the rock bottom . Most blockchain analyst finds it hard to access data to work with .
Want to hear from analyst . Which data would you want to work with ?
Week 2 of Data Engineering with Analytic Sages: done.
We went deep on extracting raw blockchain data via RPC:
→ JSON-RPC calls by hand with the request method
→ web3.py for fetching blocks & logs
→ Decoding raw EVM hex blobs into human-readable transfers
📚 A quick look into our Blockchain Data Engineering class!
From exploring Docker to getting hands-on with the tools and commands that power modern development, our learning journey continues.
At Analytic Sages, we believe the best way to grow in tech is to go beyond theory and learn by doing.
Every session brings us one step closer to building practical skills for the future of data and blockchain.
Want to be part of the next learning experience? 👀
🔔 Join the next cohort waitlist today: analyticsages.io/programs/bl…
Take your first step towards building your Blockchain Data Engineering skills.
#AnalyticSages #BlockchainDataEngineering #Docker #DataEngineering #BlockchainTechnology #TechEducation #Web3
Replying to @agaperste @Stablecoin
impressive work. would love to understand it more
Just built my first end-to-end data pipeline 🚀
DefiLlama API → pandas → PostgreSQL, fully containerized with Docker. Here’s how it came together:
1️⃣ Extract: pulled @monarq_mgmt's daily TVL history from the DefiLlama API with Python’s requests
2️⃣ Transform: used pandas to keep date and totalLiquidityUSD and convert Unix timestamps to UTC datetimes
3️⃣ Load: wrote the DataFrame to a Postgres table with SQLAlchemy
4️⃣ Containerize: packaged the pipeline in a Dockerfile, with dependencies managed by uv
5️⃣ Verify: queried the table with pgcli to confirm the data landed
Big thanks to @AnalyticSages and @Freemandayly for the structure and guidance.
Repo: github.com/tobeydev/assignme…
Freeman🌼 retweeted
If you are a data analyst, the best thing you can do, as you work with an AI, is make a skill out of Kimball's Dimensional modelling book.
I think AI is actually quite bad at making data models. IMO, there is something very human in making and accessing data models that AI can't capture.
If you read (well, the first few chapters) of the book, Kimball is really, really particular about designing the table to be intuitive to the user and never prioritizing hard data modelling rules.
My personal rule to the AI when making tables is that it has to be so intuitive that the user don't even need docs and just get cracking and exploring the tables
Likewise, in this new weird paradigm, also consider designing tables for the AI for the easiest possible access.
Idk what I've cooked with my Cursor, but it keeps creating new data marts out of my gold tables instead of writing queries to get some data💀
I get what she is saying. Historically, you have
- Human slop cannon
- People who think all code should be Good Code, who are very useful because of talent but requiring management because of the constant desire to boil the ocean / move at the speed of a snail
- People who think important (seams, data modeling, rules engines, stuff like that) code should be Good Code, and its fine if other code is slop
The most valuable persona is the third. The first will destroy a business due to overrunning tech debt, the second will tank a business due to neverending refinement. The third understands those forces need to be kept in balance.
I have a huge amount of respect for Lauren, I know folks who have worked with her and say she is absolute top tier. I know she spent a huge amount of time and effort working through Dune for example. There was probably a non trivial amount of code involved there, and it probably is the kind of thing that needed to be Good Code.
"Slop is a slur" I think is cope. "Most code in the before-times was slop and that was ok" I think is reasonable and defensible. "All code is slop" I think is an overreach, and dismisses a very important part of this conversation.
Replying to @ndii_ekanem
Welcome to data engineering , ping me if you find any topic confusing








