Professional Googler, President @TheCTOAgency
Cleveland, OH
Joined May 2010
- Tweets2K
- Following559
- Followers291
- Likes3K
Frank DeLuca retweeted
Introducing a new side project called Model Regression. It tests daily Claude, GPT, and Grok on various benchmark statistics to determine how well its performing and to identify model degrades over time.
@edskoudis had an idea for model testing before they conducted offensive testing to ensure the model was performing as expected, and @BlasikRandy pushed me down this road with actually going and doing it.
The main intent here is the frontier models will experience outages, issues, bugs, intentional/unintentional nerfing of the models without notice. You can't typically trust day to day activities in these models for stability, so leveraging this on your daily routine to see how well the model is performing for that day is something I'll be using everyday.
Runs every morning in my DGX sparks environment and automatically updates with how well its performing.
Enjoy!
modelregression.com/
Also open-sourced the project, can run on your own server as well and look at the benchmarks and how they are calculated:
github.com/HackingDave/model…
👀
It's time for Microsoft to have another XPSP2 moment. No more AI, no more features. Just fixes.
When I was working on Windows XP, Blaster hit. It was a big enough deal that we set aside all feature work.
For the next several months, all we did was improve security. We didn't add "security features"; we fixed bugs. Lots of bugs. Until there weren't security bugs to fix anymore.
Then we fixed the ones we didn't know about yet.
Put more simply, we stopped trying to "add value" to the product through features that PMs thought users would like, and instead we focused on the things that had been important for a long time, but overlooked.
Like performance and configurability today.
Rather than trying to improve and add value to the system through new AI features -now-, I argue it's time for Microsoft to stabilize, improve, and make the system more performant. And more usable for power users.
Just for one release. Just till it doesn't suck.
Just spoke with a customer service rep at @ArcBestCorp to get some tracking details, and holy cow, that had to be the friendliest, quickest, best experience I've ever had with a shipper. 👏👏
Not me tweeting again about what a hot pile of trash Quickbooks online is. I should be in bed, but instead I’m trying to make a broken product do things the desktop version had no problem doing.
I get a new notice of a data breach on a pretty regular basis now. Seems like credit monitoring should just be part of the Bill of Rights at this point 🤷🏽♂️
Overclock from @wehackhealth showed up about an hour after I finished my workout. Gonna have to wait til Monday to try it!
Whoever was running @TowerLightsCLE and syncing it to the @311 show lights tonight was great! @pnut @NickHexum
Hey @hulu - I rebooted my cable modem and my ip changed because it’s dynamic… but suddenly you think I’m not home anymore and only give me 3 more changes? Does this happen every time my ip changes?
Self checkouts are my nemesis. Don’t make the beeping sound if you didn’t actually acknowledge what I just scanned, and then accuse me of trying to steal 😤
Had a fun time at the @onepeloton studio today for a ride with @emmalovewell that was also my 1,000th ride. #peloton – at Manhattan, NY
Frank DeLuca retweeted
We had a great time at our annual @cavs Court of Dreams event yesterday! Our employees & clients played a competitive game on the home court, followed by a reception and the Cavs vs. Heat game. We are proud to be the official #cybersecurity provider for the #ClevelandCavaliers!