Pinned Tweet
“We are all driven by an invisible whip. Some run, some have fun, some are hip, some tip, some dip, but we all must answer to the invisible whip.”
— Rahsaan Roland Kirk
i work in incident response. there has been a definite uptick in cybersecurity incidents, especially at the Fortune 500, over the last few months. that, however, does not seem to be what is causing the recent outages. i think it’s twofold:
What happens when a politician tries to somehow deliver a technical outage report.
Ridiculous to say the least. I’m stuck at the “69th installation…”
Duffy: This was the 69th installation of that router. 68 times it worked well. We've been able too recreate what happened. But the root cause we don’t have clear yet.
1
60
David Njuki retweeted
Over the past week, I’ve been overwhelmed by the love and support of friends, industry colleagues, and people I've never met. Thank you to everyone who has shown kindness to my family and me. It means the world to me.
I came to the United States 53 years ago, and becoming an American citizen is one of the greatest gifts and proudest moments of my life. NVIDIA has been my life’s work. Building it into the company it is today alongside my colleagues has been a privilege and an honor. Nowhere but in America could a company like NVIDIA be possible. I love our country, and I love NVIDIA.
I did not live up to the standards expected of everyone at NVIDIA, and I have made serious mistakes in my personal life. I created problems for the Board that I deeply regret. I own my mistakes completely, and I am committed to cooperating with the Board’s independent investigation.
Since my departure, I’ve seen an outpouring of calls from NVIDIA’s employees, investors, business partners, and customers for me to return as CEO. Many continue to reach out to me directly as the company navigates complex technology, market, competitive, and geopolitical dynamics. NVIDIA is a once-in-a-generation company. If the Board of Directors decides that my return is in the best interest of our shareholders, our employees, the company, and America, I would be honored to answer these calls and return as CEO.
Thank you again to everyone who has shared their good wishes. I am profoundly humbled by your support. To my family: I love you, and I will spend the rest of my life being worthy of yours.
Jensen
David Njuki retweeted
We are present in the Africa Tech Festival 2026 where Africa's technology leaders, innovators and Internet community are coming together to explore the future of our digital continent.
We look forward to engaging with industry partners, sharing insights and strengthening partnerships that support a more connected and resilient Africa.
Attending the festival? Let's connect
David Njuki retweeted
More details on the root cause can be found in the Microsoft 365 Admin center, and an abridged version can be found below.
We are deeply sorry for the impact to all our customers and recognize the impact that this has on their normal business & operations.
The incident on Thursday, November 12th, 2026, was caused by three main factors that together caused a global outage for the networking gateway that fronts many Microsoft 365 services.
First, a latent defect caused servers to become overwhelmed and unresponsive under certain load conditions. This was due to lock contention inside an internal process which caused the machine to become unavailable. This defect was the result of a fix that was prepared specifically for Microsoft 365 networking gateway servers; neither the underlying defect nor the fix applies to broader Windows deployments.
The initial change in behavior first started deployment in July of this year. The update went through safe deployment and saturated globally over several weeks, during which time no issues were observed. For more than three months, the change was deployed worldwide without impact.
Our normal load testing did not detect the latent issue because the different traffic and SKU mix used in our test labs. The contention issue is worsened on machines with more cores and newer hardware; whereas our load testing tended to focus on our weaker hardware SKUs with fewer cores.
Second, a surge in load increased the likelihood for the networking gateways to encounter the contention. Within Microsoft 365, we maintain two independent and heterogenous network gateway systems designed for isolation and high availability. This allows many Microsoft 365 services to fail to/from them during outages or disruptions. Approximately fifteen hours before the outage, Teams failed over from the alternative networking gateway to the ‘primary’ gateway that also supports Exchange Online. This increased the load on the networking gateway and changed its traffic patterns. This type of failover is part of normal operations and is integral to standard Microsoft resiliency efforts.
Third, at the same time, nearly 14% of server capacity for the networking gateway was out of service due to an unrelated issue from the previous day. This server capacity was on track to be returned to active service through routine maintenance and health processes. The service had sufficient capacity headroom, so the reduction in server capacity had no impact on its own.
Taken together, the increased load and the reduced server capacity, caused the network gateway to encounter the latent bug from July. This caused requests to build up until servers ultimately stopped responding. This occurred during the peak window of traffic in the United States.
When servers lose connectivity, our traffic management systems automatically route users to other healthy endpoints. This happens often and is critical to avoiding any interruption due to hardware, software, or network defects and is normally what makes the service resilient. However, the redirected traffic increased the likelihood for the same problem to occur on the remaining endpoints. This caused a cascading failure from server to server as the same amount of load was spread across fewer servers. In less than two hours, the previously unknown defect was encountered across our fleet and rendered them unresponsive. At its lowest, more than 80% of servers were unresponsive.
Normally, the network gateway is composed of thousands of ‘capacity units’ which represent shards of capacity, hardware, and networking infrastructure. These ‘capacity units’ are then organized into regions. In this case, due to a separate issue, the normal bulkheads between these different partitions did not stop the cascading failure. This is what allowed the impact to spread across our network gateway infrastructure globally.
Several services like Teams, OneDrive, SharePoint, and Office apps failed out of the faulty network gateway infrastructure. That is why those services were restored earlier, with Teams chats and meetings functioning without issue for most of the outage. But not all services, like Exchange Online, can fail out of this network gateway.
Recovery took longer than usual because as servers returned to service, they were immediately overwhelmed by waiting traffic and became unresponsive again. Our normally automated recovery processes were not designed to restore service from this broad unresponsive state. Additionally, since the main code defect was a latent bug from several months previous, our normal rollback, flighting, and feature flags were unable to quickly revert the change.
After engineers confirmed the defect as the cause, we developed an update that returned the code path to the previously known good state. To effectively apply this fix, and to slowly bring the service online again, we took groups of servers out of service, applied the change, and restarted them before returning them to service. This process was carefully applied to large quantities of infrastructure at a time (e.g. 50%+ of a region).
The final Post-Incident Review (PIR) will provide more details later this week.
Replying to @MSFT365Status
We have released the Preliminary Post-Incident Report of last week’s outage, which is accessible via the Admin center under MO1550821 and EX1550543.
David Njuki retweeted
We are deeply sorry for the disruption this incident has caused our customers and partners. We know the trust they place in our services and take that responsibility extremely seriously. We are seeing broad recovery and stability across our services.
This was a technical issue not related to cyber or AI / agents. The incident was introduced by an issue during an earlier network capacity upgrade that surfaced following a recent network change. We have addressed the underlying issue and continue to restore service to remaining customers and regions.
Given the scale of demand, we restored services gradually, validating stability at each stage. While this took longer, it helped ensure a stable restoration.
We will provide a full post-incident report once our review is complete.
David Njuki retweeted
Microsoft’s Exchange Online is back, and then some. After ~19 hours running ~85% below expected, traffic began recovering at 6:20 AM ET (11:20 UTC). By 8:25 AM ET it was running ~19% above expected as queued mail and mailbox syncs caught up.
What happens when we replace network cables with carrier pigeons? We worked out how many birds you would need.
blog.apnic.net/2026/11/13/co…
so:
- no succession plan.
- 60 reports which means no obvious person with complete understanding of the co operates. or at least it’ll be insanely messy. potential for a lot of infighting.
- a weak board suddenly tasked to manage a crazy crises w/ arguably the most important co of this era.
- possible talent exodus since founder is leaving.
- a ceo search that could take a while with no obvious outsider candidate at the top.
- potentially more revelations.
- jensen will have zero involvement going forward. no chairman role.
- downstream impact on ai ecosystem from funding, support etc.
this is not good. what am i missing?
David Njuki retweeted
sade – “be thankful for what you got” (cover), recorded during the promise tour in tokyo, 1986.
David Njuki retweeted
A luxurious world of its own
pitchfork.com/reviews/albums…
David Njuki retweeted
eMudhra's incident report is out! A Python script truncated their CA private keys to one byte?!
bugzilla.mozilla.org/show_bu…
David Njuki retweeted
The John Coltrane Quartet performed live at New York's Village Vanguard from November 1-5, 1961. The shows ultimately became the material for Coltrane's first live album, and the first record featuring the quartet.
Much of the opposition to DNSSEC appears to come from two concerns:
- The possibility of top-down state control over access to information.
- DNSSEC deployment remains incomplete.
Both concerns are valid, but miss context and the long view.
blog.apnic.net/2026/11/03/re…
David Njuki retweeted
Introducing Zero Wire: a public vulnerability intelligence feed on @CloudflareRadar, bringing the clarity of a news wire service to critical security developments. cfl.re/4iCvSbj
The game is the game 🤯
Our investigators are observing never-seen-before laundering patterns with the stolen funds from the Binance exploit. Thousands of transactions in a short period suggest that the attackers are using automation to obscure the trail of funds. Rather than simply moving funds onward, they repeatedly circulate them among addresses, often sending funds back to addresses involved earlier in the sequence.
Read more: chainalysis.com/blog/binance…
64





























