Was software just always too complex for humans?

(anil.recoil.org)

33 points | by nathcd 4 hours ago

14 comments

  • analog31 37 minutes ago
    Take this from a rank amateur "scientific" programmer. I write functions with maybe a half dozen lines, then I piece them together. Sometimes my programs work, sometimes they don't, and I fix them. I've done it this way since I first learned Pascal 40+ years ago. I'm not allowed to touch production code, so don't panic.

    When I ask AI to write code with similar functionality, it's about 10x more code. Likewise when I look at the code for well respected open source projects. It's good code in terms following accepted practices, but hard to read because there's too much of it to see where the actual "working" code is.

    A lot of the code is "defensive," i.e., type and bounds checking, trapping error conditions, etc. When I've looked at the unit tests, most of them test the defensive code.

    I tell my engineering friends: "Engineers like to over-complicate things." Maybe the AI has picked up on this and is serving it back to us.

    • jrmg 10 minutes ago
      > I write functions with maybe a half dozen lines, then I piece them together. Sometimes my programs work, sometimes they don't, and I fix them.

      I think you might be under a misapprehension about how non-rank-amateurs code. Almost all coding is trial-and-error if we're honest. No-one, no matter how well they plan, writes towering pristine monoliths of code.

      The code might sometimes look like that after they're done, but that's only because so much time has been spent testing and 'polishing'.

      Most - maybe all - professional programmers I've worked with spend longer in the debugger or inserting and reading logs than they do in writing code.

      Sometimes it's possible to plan an architecture beforehand and implement it as-designed, but even that is usually a fools errand - the plan needs to change as the architecture meets reality and unforeseen complications arise.

      • stingraycharles 3 minutes ago
        This is accurate, and it’s also that the code of those projects evolved over years, sometimes decades, of time, often by multiple people iteratively improved to fix issues.

        It’s a time management thing as well; we would all like to have infinite time to make things beautiful and perfect, but you have to pick your battles.

        This shouldn’t be the case for AI, but because it’s trained on this type of code, it probably has a tendency to write similar “verbose” code.

        You can still steer it in the correct direction when reviewing its code though.

    • metalcrow 5 minutes ago
      Isn't defensive code just outright better to do? Sure if you're making scripts for single use or small one-offs, it's unneeded. But the moment you make a usable product or a library or a tool that more people use, if your code isn't defensive won't the program just be strictly worse? It's always better to avoid crashes or bugs or exploits and you need to be defensive to do that.
    • aetherspawn 13 minutes ago
      In python yes, but if you add or use only static types it clears this all up and the LLM will generate short, optimal code without edge case fluff.
    • dude250711 16 minutes ago
      > When I ask AI to write code with similar functionality, it's about 10x more code.

      Using AI is still a skill in 2026. You do not have to one-shot it, you can prompt it to reduce the amount of code e.g. by dismissing unlikely edge cases or pointing it at some preexisting reference code.

    • 14113 21 minutes ago
      Software engineers like to over-complicate code in the same way that structural engineers like to over-complicate buildings.
      • Muromec 7 minutes ago
        It's the architects. The love doing things like Sluishuis because it's cool. So next time you see a bug and an architect, hit the architect first.
    • rf15 3 minutes ago
      my experience has been similar, working for over 20 years as an egineer: engineers like to talk too much, even with their code.
    • spoiler 16 minutes ago
      I think a lot of the complexity comes with having to handle unknowns or weird edge conditions, most of which are there because code runs on hardware which follows the laws of physics. So really, we need to blame the physicists for not drafting better laws!
  • yyyk 36 minutes ago
    No, and users of this site know better. Humans built massive reliable software system. Inasmuch as anything was 'too complex', it was the choice to rely on underspecified interfaces, ancient tools and legacy protocols for various reasons (often boiling down to economics of time and resources). But it would have been humanly possible to rebuild the entire software stack using more reliable methods (that the article suggests for LLMs, but there's nothing yet making these methods inaccessible for mere humans), and these reliable methods would have solved nearly all bugs. It would just be a lot less economic and take more time (both directly and as opportunity cost).
    • bgun 13 minutes ago
      Just because it’s humanly possible to build complex systems correctly doesn’t mean they’re likely to evolve that way while under human oversight. There are too many competing incentives within large organizations for any human collective to guarantee this.

      Your comment comes off as a little shaming of organizations that don’t meet your standards, which seems to be most if not all of them including most institutions. Which is your right, but it gives the ick.

      • layer8 6 minutes ago
        I don’t see how AI leads to complex systems being build more correctly, at least at the current state of affairs.

        Regarding the standards not met: That’s because it would be more costly and/or take more time. Correct software is expensive and takes time. I don’t want to shame everyone who has been cutting corners, but the state the industry as a whole is in regarding software quality after 50-ish years really isn’t praiseworthy either. We could and should do better.

  • hypfer 53 minutes ago
    Answering the Headline: Not really, no.

    Things are the way they are not because they would be difficult but because various systemic incentives and goals fought out each other, leading to the outcomes we then saw.

    This balance these various forces fought out relied on a bunch of undocumented assumptions about reality. Every time one of those breaks, you get disruptions. LLMs broke quite a few of them, leading to a lot of CVEs lately.

    • Muromec 10 minutes ago
      I believe it's like with rivers. We just keep adding stuff until we can't deal with complexity anymore and adding stuff takes too much time to get enough value. Nobody stops developing their piece of software while it's still under control.
    • mentalgear 30 minutes ago
      The systemic incentives were acceptable risk / effort.

      It's not that engineers 30, 40, 50, ... years ago didn't know the potential risks of the hard- or software, but business executives found them acceptable, and those companies that provided the best 'bang for you bucks' with acceptable risk surfaced.

      But this equation, which rested on the assumption of a fixed cap of humans being able to have the resources and skills to exploit a system, is now dissolving quickly as all you need now to do e2e cyber penetration or even soon social engineering is compute (no expertise or humans).

      • Muromec 9 minutes ago
        > but business executives found them acceptable, and those companies that provided the best 'bang for you bucks' with acceptable risk surfaced.

        that does not explain all the CVEs in open source projects. or does it?

  • doug_durham 19 minutes ago
    Yes, and users of this site agree. Software complexity has always been at the edge of human abilities. This is why there have been countless attempts at managing complexity such as OOP, functional programming, type-driven development, ... The only reliable software is that which has been stripped down to the bare essentials. The way to allow humans to deal with software complexity is to not write software. SWE in practice is a deliberate balance between cost of development and completeness, where completeness means fully tested and all corner cases accounted for. It has been the general case that shipped software is knowingly incomplete. For the most part that pays off. It will continue to pay off.
  • dgoodell 1 hour ago
    I think it was a business decision to make it too complex for humans. I don’t think fundamentals were prioritized as much as features, resulting in a fragile house of cards.

    There’s a reason it doesn’t look like most engineering. If it was a bridge it would have collapsed long ago and killed people.

  • 0xbadcafebee 2 minutes ago
    [delayed]
  • amelius 57 minutes ago
    The always changing requirements are what make it too complex for humans.
  • randallsquared 46 minutes ago
    Specification has its place, but even with "perfect" pieces of software, the interaction of them, together with their interaction with the physical hardware, mean that you still can't know everything that can happen in the whole system. You can specify a unix tool (do one thing and do it well...), but you can't formally prove all of an operating system plus userland without very strictly limiting what outcomes are possible, and even then novel or buggy hardware or other physical environment variables could produce unexpected results: bugs. The formal specifications themselves, to cover all this, will have to become so complex that no human can understand them, which just means that some potential exploits will be explicitly allowed by the spec, and an evaluator at a given level of intelligence (human or other) may not understand the bug or even that it exists.

    For any system designed by some intelligent agent IA, there's going to be holes/bugs/exploits that can be easily seen by some agent IA*100. This has never been a serious problem in anything before, because humans don't span that range of intelligence, and collaboration doesn't provide such insights.

  • ssrc 53 minutes ago
    Maybe for some software systems it was. For most systems, the problem was just professional immaturity caused by the eternal September state the profession is in since the beginning.
  • maxrev17 32 minutes ago
    It’s just how quickly you can deploy the resource right? Models are not bad but you can deploy tonnes of energy quickly. With people you can’t deploy as quickly. Maybe?
  • yepyoukno 4 hours ago
    Many handle it just fine.
  • 0c3ca83 34 minutes ago
    Betteridges law of headlines strikes again.
  • ex-aws-dude 54 minutes ago
    I mean if you zoom out the whole lesson we keep learning over and over with AI/ML is “we are not smart enough to design this function”
  • AIShill101 4 hours ago
    Yea I agree with the article wholly. This new era of AI assisted or driven coding is going to help us extend our capability way beyond anything humans could've built.
    • dingaling911 1 hour ago
      At the moment, oversight is still heavily required.

      Before, we used to write code and machines converted it to binary.

      Now we will write designs and machines will convert it to code (and then to binary.)

      There is a possibility of models taking over design for somewhat run-of-the-mill applications with minor architectural tweaks. This is the area that "vibe coding" is currently dominating. How many vibe coded apps, written by others, are you using? None. So that tells you something.

      At best, models are currently at VB6 level: domain experts can do a lot of good. But they could've done it with VB6 as well. And MS Access. And Excel.

      I am not sure how we would give the models an idea of taste and beauty because that is often what differentiates good/bad from great designs.

    • onion2k 56 minutes ago
      This new era of AI assisted or driven coding is going to help us extend our capability way beyond anything humans could've built.

      For individuals, sure. I'm fairly sure we're all building things that we couldn't build ourselves if only because we don't have the time needed. For large scale projects like building an OS from scratch or a database engine that's a lot less likely to be true, simply because there are dozens of examples of humans doing that level of complicated work without AI already (the OSs and DB engines we have are very literal evidence of it.) They just need to work collectively. Every part of those projects is understood people a person even if no person understands the whole thing.

      So the problem with doing the same sort of thing with AI becomes a question of whether someone can build the equivalent sort of project with AI by describing something more complex than they understand themselves. And even if they just ignore the complex details and let AI do that bit, how can they validate that the AI has actually done what they asked if they can't understand it? It might have taken a shortcut that works until it doesn't. I hope no one builds an aviation or a life support system by doing that.

      Ultimately, I suspect we want to be able to understand the system even if it limits what we build. A system that fails in an edge case carries significant risk which people probably don't want to be liable for.