Jon Lunman retweeted
We can do orders of magnitude more with agents, but it turns out that building bug free reliable software still takes a huge amount of effort. Now that we can do more, we have to much even more effort into making the software reliable.
You can see these companies crank out more features, but rarely are they high quality. You can feel the jank after the honeymoon phase of the first 5 minutes of use.
Jon Lunman retweeted
AI has dramatically accelerated how software is written. But speed was never the real bottleneck.
Despite LLMs, The Mythical Man-Month is still surprisingly relevant. Not because of how code is produced, but because of what actually slows software down: coordination, shared understanding, and conceptual integrity.
AI makes code cheap. It does not make software design, architecture, integration, or alignment free.
In fact, faster code generation can amplify old problems:
* Incoherent abstractions appear sooner
* Integration costs surface later
* “We’re almost done” illusions become stronger
What matters more than ever is strong architecture, clear intent, and technical leadership. The modern leverage point is not the fastest coder, but the person who can frame problems well, guide AI output, and preserve system coherence.
A modern version of Brooks’ Law might be: "Adding more AI to a late or poorly defined project makes it confusing faster."
AI changes the tools. It doesn’t repeal the laws of software engineering.
Jon Lunman retweeted
Last night I watched an elderly fam member navigate her iPhone for ‘important things and emails’. She is 96 and holding on to be tech relevant. Here is what I observed… 🧵
Jon Lunman retweeted
Replying to @Grady_Booch
The fact that SEO is a whole industry was our first clue. If it's really indexing relevance then we shouldn't need to add metadata, game keywords, and link farm things to make them appear to be relevant.
Jon Lunman retweeted
Your brain consumes 20 W of power. A single GPU consumes a kilowatt. A data center has tens of thousands of them, and is still not as powerful as your brain. Chew on that before you say AGI is imminent.
Jon Lunman retweeted
What are the "best practices" when you design REST APIs? There are a lot that solve problems that you might not need to have.
codeopinion.com/how-to-and-h…
Jon Lunman retweeted
So many people are confused about the relation between human cognitive errors and LLM hallucinations that I wrote this short explainer:
Humans say things that aren't true for many different reasons
• Sometimes they lie
• Sometimes they misremember things
• Sometimes they fail to think through what they are saying
• Sometimes they are on drugs
• Sometimes they suffer from mental disorders
etc
LLMs errors result from 𝙖 𝙙𝙞𝙛𝙛𝙚𝙧𝙚𝙣𝙩 𝙪𝙣𝙙𝙚𝙧𝙡𝙮𝙞𝙣𝙜 𝙥𝙧𝙤𝙘𝙚𝙨𝙨. They don't have (e.g.,) intentions, egos, or financial interests, so they don't lie. They don't take drugs. They don't have emotional states.
Instead, LLM "hallucinations" arise, regularly, because (a) they literally don't know the difference between truth and falsehood, (b) they don't have reliably reasoning processes to guarantee that their inferences are correct and (c) they are incapable of fact-checking their own work. Instead, everything that LLMs say -- true or false -- comes from the same process of statistically reconstructing what words are likely in some context. They NEVER fact-check what they say. Some of it is true; some is false. But even with perfect data, the stochastic reconstructive process would still produce some errors. The very process that LLMs use to generalize also creates hallucinations. (In my 2001 book I explain what a different generalization process might look like.)
§
Importantly, the goal of AGI is not to recreate humans; we don't want AGI to lie or suffer from psychiatric disorders, for example. Rather, the goal of AGI should be to build machines that can reliably reason and plan about a wide swathe of the world. The fact that humans sometimes make errors, sometimes deliberately, sometimes accidentally, in no way takes away from -- or repairs -- the limitations of the current approach.
The field of AI will eventually do better, but probably with an AI that is structured differently, in which facts are first-class citizens, rather than something you hope you might get for free with enough data.
TL;DR: Don't console yourself with making something that superficially looks like human errors, if you aspire to AGI.
Jon Lunman retweeted
tale as old as time
Replying to @evertp @Wagepoint
Depressing amount of javascript code for what could have been a fairly straightforward server-rendered multi page HTML app so good luck debugging otherwise ^_^
Jon Lunman retweeted
One of the most common concerns about AI is the risk that it takes a meaningful portion of jobs that humans currently do, leading to major economic dislocation. Often these headlines come out of economic studies that look at various job functions and estimate the impact that AI could have on these roles, and then extrapolates the resulting labor impact. What these reports generally get wrong is the analysis is done in a vacuum, explicitly ignoring the decisions that companies actually make when presented with productivity gains introduced by a new technology -- especially given the competitive nature of most industries.
The thinking generally goes that if a company could, say, be 50% more productive in a particular function, it would mean a commensurate reduction of jobs in that area. For instance, if a certain function (like engineering or sales) required 10 units of labor before, then with a 50% gain in productivity, in the future that same function would now only need ~7 units of labor. The challenge with this type of thinking is that it assumes that companies have maximized the amount of labor they wish they had for a particular function, when in reality many functions are only staffed at the level the company can afford. Further, it assumes that a company is not in a competitive field, and that the company would be complacent and happy about generating the same output as before, just with less costs. Finally, it ignores the fact that productivity gains in a market will lead to increased response from competition, which companies equally have to respond to with more productivity not necessarily more profit. Time and time again this is the type of flawed thinking that we tend to get out of broad economic studies on the labor needs in the economy.
To break this down and make it practical, I thought I'd illustrate the point with the example of an engineering function -- one that already is seeing the benefits of AI starting to roll out. The numbers will all be kept simple, but you can change almost any variable and the point will remain the same. The key to thinking through job impacts is to think through what happens a step or two *after* the productivity gain of AI is experienced.
So, imagine you're a software company that can afford to employee 10 engineers based on your current revenue. By default, those 10 engineers produce a certain amount of output of product that you then sell to customers. If you're like almost any company on the planet, the list of things your customers want from your product far exceeds your ability to deliver those features any time soon with those 10 engineers. But the challenge, again, is that you can only afford those 10 engineers at today's revenue level. So, you decide to implement AI, and the absolute best case scenario happens: each engineer becomes magically 50% more productive. Overnight, you now have the equivalent of 15 engineers working in your company, for the previous cost of 10.
Finally, you can now build the next set of things on your product roadmap that your customers have been asking for. We can't assume it will be 50% more because there are new points of friction and coordination tax that emerge as you have 15 equivalent engineers, but let's say your output goes up meaningfully. Assuming you're acting in your best interests as a company, the features you build make your product that much more compelling, which means at some point (sooner or later) they should result in an incremental gain in revenue. Let's be somewhat conservative on what impact these new features will have on your product, but let's say they generate an incremental 10% of revenue over time or keep customers retained at a 10% greater rate (roughly the same financial benefit).
Now let's assess the downstream impact. Firstly, any growth of revenue will often lead to some functions in the business growing as well to support these new customers, which will directly create new jobs. But further, the company now has to decide whether it remains satisfied with its 10 engineers that have the output of 15, or with their incremental revenue should they hire even more engineers to build the *next* set of features that will make them even more compelling to customers. Unless this company is in some rare monopoly position, they likely will want to build the next set of features even faster than the last set to grow even more quickly. This then means AI has caused the company --counterintuitively-- to hire more engineers than before, because the productivity of each engineer is much higher, allowing them to generate more return per engineer, and thus more revenue.
What's interesting is this analogy works similarly for most functions in a business. In sales, if you could make sales reps 10% more productive (i.e. they sell 10% more of your products/services for the same cost), almost every company in the world would prefer to hire even more sales reps, instead of merely banking the incremental profit. That incremental sales productivity again would lead to downstream implications, like the need to deliver more features to customers, and thus more R&D hiring! Even back-office functions that don't as directly tie to revenue growth, often are a bottleneck to growth . If you can reduce the bottleneck -- say lawyers reviewing contracts, or people processing invoices-- cycle time in businesses accelerates, which almost always lets you serve more customers faster or grow more quickly, again letting a company reinvest those dollars.
In the end, when you step out of the vacuum of just the specific productivity gain of a particular job function, and look at how the whole system will adapt and improve due to that productivity gain, a very different picture of AI's impact on jobs will emerge. Yes there will absolutely be changes to what jobs become more or less in demand in the future, but the competitive nature of companies inevitably ensures that across the whole system companies will be focused on leveraging AI to become more productive.
Jon Lunman retweeted
𝗗𝗲𝗲𝗽 𝗹𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗮𝗻𝗱 𝗚𝗲𝗻𝗔𝗜 𝗮𝗿𝗲 𝘀𝘁𝗮𝗿𝘁𝗶𝗻𝗴 𝘁𝗼 𝗳𝗲𝗲𝗹 𝗹𝗶𝗸𝗲 𝗮 𝗦𝗵𝗮𝗸𝗲𝘀𝗽𝗲𝗮𝗿𝗲𝗮𝗻 𝘁𝗿𝗮𝗴𝗲𝗱𝘆.
I have been warning the field of its limitations for decades.
The field has done everything its power to try to dismiss and silence me.
And now everything I have been warning about is coming home to roost:
2001: 𝗛𝗮𝗹𝗹𝘂𝗰𝗶𝗻𝗮𝘁𝗶𝗼𝗻𝘀, and core problems with 𝗮𝗯𝘀𝘁𝗿𝗮𝗰𝘁𝗶𝗼𝗻 𝗮𝗻𝗱 𝗿𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴
2014: (w E. Davis) 𝗧𝗵𝗲 𝗲𝗰𝗵𝗼 𝗰𝗵𝗮𝗺𝗯𝗲𝗿 𝗲𝗳𝗳𝗲𝗰𝘁 that threatens to contaminate the web
2015 (w E. Davis) The need for research focus on 𝗰𝗼𝗺𝗺𝗼𝗻 𝘀𝗲𝗻𝘀𝗲 and 𝗽𝗵𝘆𝘀𝗶𝗰𝗮𝗹 𝗿𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴.
2016: 𝗘𝗱𝗴𝗲 𝗰𝗮𝘀𝗲𝘀 as the undoing of driverless cars
2018: 𝗕𝗿𝗶𝘁𝘁𝗹𝗲𝗻𝗲𝘀𝘀/𝘂𝗻𝗿𝗲𝗹𝗶𝗮𝗯𝗶𝗹𝗶𝘁𝘆 and 𝗣𝗼𝘁𝗲𝗻𝘁𝗶𝗮𝗹 𝗽𝗹𝗮𝘁𝗲𝗮𝘂𝘀 because of problems with abstraction and reasoning etc.
2018: The 𝗱𝗮𝘁𝗮-𝗴𝘂𝘇𝘇𝗹𝗶𝗻𝗴, leading to the desperate need for data that the NYT just reported, along with obscene corner-cutting and rampant copyright issues
2019: The way in which 𝗟𝗟𝗠s inherited all of the above liabilities from prior deep learning architectures.
2022: 𝗟𝗟𝗠-𝗱𝗿𝗶𝘃𝗲𝗻 𝗽𝗼𝗹𝗹𝘂𝘁𝗶𝗼𝗻 𝗼𝗳 𝘁𝗵𝗲 𝗲𝗰𝗼𝘀𝗽𝗵𝗲𝗿𝗲 threatening trust and democracy.
It’s all happening now. Every single point was originally ignored or dismissed (often most vocally by @yLeCun). Not one of these problems has been adequately addressed.
Now we are left with two things
• an intellectual monoculture, with no alternative approach remotely well-funded
• a pile of unreliable systems that are unlikely to live up to the hype
Only with a more open-minded field can we hope to make progress.
Jon Lunman retweeted
Today's newsletter: I believe that we're in an AI bubble where hype has vastly outpaced adoption, nobody is making any money, and generative AI companies are running out of data and resources to train their models before proving themselves necessary.
wheresyoured.at/bubble-troub…
























