Khalid Hussein

We love stars,... you love stars,... GitHub has stars. Love is starring ;)⭐ github.com/xberg-io/xberg

Introducing kreuzcrawl v0.3.0 Shipping the first stable release of kreuzcrawl today; a Rust web crawling engine (scrape, crawl, sitemap discovery, headless-Chrome fallback, WAF bypass) with native bindings for 14 languages. What changed from v0.2.0 (10 languages → 14): The four new bindings are Dart, Kotlin/Android, Swift, and Zig. All 14 are generated from the Rust core via [alef](github.com/kreuzberg-dev/ale…), our polyglot binding generator; we don't hand-write FFI glue per language. Memory-bounded streaming crawl_stream() used to accumulate every page result before yielding. On large crawls the peak working set was around 2.5 GB. The fix was straightforward but the impact wasn't obvious until we measured it: each page now moves into CrawlEvent::Page and is dropped immediately instead of being held in the result buffer. Peak is now around 20 MB. Same API surface, no behavior change on the consumer side. SSRF defense on by default Every call path; scrape(), crawl(), sitemap fetch, robots.txt fetch, and asset downloads; now runs URLs through an SsrfPolicy before the request. Refuses loopback, RFC 1918, link-local, cloud metadata endpoints (169.254.169.254 etc.), and non-http(s) schemes. DNS-rebinding mitigation is included: every resolved IP must pass the policy, not just the hostname. Redirect chains are re-validated per-hop, bounded by ssrf.max_redirects (default 5). Opt out with KREUZCRAWL_ALLOW_PRIVATE_NETWORK=1. Tiered dispatch + WAF detection The crawl engine now chains HTTP → Bypass → Browser tiers driven by per-attempt signals rather than a static bypass flag. Behind it: 35 WAF fingerprints in a TOML corpus matched via Aho-Corasick, EWMA per-domain block-rate tracking that promotes/demotes the starting tier, and TomlClassifier::watch() for hot-reload (debounced 500ms, atomic ArcSwap swap, Kubernetes ConfigMap-safe). MCP server Tools are 1:1 with the CLI batch_crawl, batch_scrape, download, generate_citations), each with read_onlydestructiveopen_world safety annotations. Served over stdio and Streamable HTTP at /mcp. Substrate parsers are now public kreuzcrawl::robots and kreuzcrawl::sitemap are exported; parse_robots_txt, is_path_allowed, parse_sitemap_xml, parse_sitemap_index; usable standalone without constructing a crawl engine. Docs: docs.kreuzcrawl.kreuzberg.de… GitHub: github.com/kreuzberg-dev/kre… Happy to talk through the FFI architecture, the alef generation pipeline, or the dispatch engine design if anyone's curious. Cheers!!!

We've been building something,... yes we've been building @kreuzberg_dev but the amount of joy seeing devs creating products using this tool is really impressive!!! lilbee helps you to find, run, and manage your local AI models, then talk to your files, your code, and the sites you crawl. Every answer cites the source, it all runs on your own machine, and it's one program with nothing else to set up. check on this amazing project here lilbee.sh! also if you're building something with @kreuzberg_dev please tag us; tag me for a cookie :) I'll be posting benchmarks, tutorials, and honest comparisons,... follow along

hey @grok what would it take for L1s to actually start cooperating???

The Balkanization of Web3 @gavofyork just called out the biggest unspoken problem in blockchain: Layer 1s treat each other like rival football clubs in a zero-sum league. Instead of collaborating to grow the entire space, networks fight over the same finite pool of users, liquidity, and devs. Collaboration is seen as weakness. The result? Fragmented ecosystems, crippled cross-chain utility, and a Web3 that’s nowhere near mass adoption. Key points from Gavin: • Zero-sum mindset – Everyone’s trying to steal market share instead of enlarging the pie for all • Immense community pressure – Founders face backlash if they cooperate; it looks like surrender to token holders • Token = sovereignty – Deep economic mergers are almost impossible without one side “losing” value or identity • Technically solvable, humanly hard – The tech exists, but inertia from incentives and ego is massive We started Web3 to build an open, trustless, resilient internet. Right now, tribal warfare is holding us back. Worth watching the full segment on When Shift Happens with @KevinWSHPod: youtu.be/jyMxSIFyXwo?t=2011 What would it take for L1s to actually start cooperating???

The First Principle of Apex SDK Unification Without Compromise. We didn't build a lowest common denominator API. We built a layered system: 1) Unified Primitives for common ops. 2) Native Ecosystem APIs for full power. 3) Escape Hatches for when you need direct access. You get simplicity and sovereignty. crates.io/crates/apex-sdk github.com/eurybits/apex-sdk apexsdk.dev

🏆 Thrilled to announce that Apex SDK won the Community Vote in the @Polkadot Cloud hackathon! Key achievements: ✅ Unified API across chain types ✅ Production-ready architecture ✅ Comprehensive developer experience ✅ Strong community validation Love the community energy🔥

Apex SDK Protocol v0.1.4 released! Write once, deploy everywhere: • Polkadot, Kusama, Westend • Ethereum, Polygon, BSC, Avalanche Get started: kherldhussein.github.io/apex…

Building multi-chain apps is painful. Apex SDK changes that. One Rust SDK → Both Substrate & EVM ✓ Compile-time safety ✓ 6x faster than web3.js ✓ 80% less code Ship cross-chain DeFi in hours, not weeks. github.com/kherldhussein/ape… devpost.com/software/apex-sd…