Like many, I have turned to AI for circuit ideas. And like many, I have been surprised by how many of these ideas just don't work. I have found relatively simple circuits that don't work when built in the real world, and don't work when built in LTSpice. I have already described how AI failed when I asked it a simple question about how many 330 ohm resistors to put in parallel to get 50 ohms. Mike WU2D had found that a lot of AI provided circuits just don't work. (Go to the 10 minute point in his video. )
I was wondering why this is so. So... I asked AI! For those who are sure that their AI is smarter than my AI, let me specify: I just wrote the question into the Google search box and looked at the "AI Overview" in the response.
Here is what AI said:
AI-recommended circuits often fail in the real world due to gaps between simulation and reality, such as a lack of real-world data (noise, rare conditions), AI's inability to grasp complex physical nuances like heat dissipation or component aging, and poor integration with physical prototyping, leading to brittle designs that don't account for manufacturing tolerances or environmental stresses, despite being mathematically sound.
Key Reasons for Failure:
Data Limitations (Garbage-In, Garbage-Out): AI relies heavily on training data; if it's incomplete (missing extreme temperatures, noises, aging effects) or biased, the AI generates designs that work in simulation but not in messy real-world conditions.
- Brittleness & Lack of Context: AI struggles with "edge cases" or novel situations not in its training, leading to fragile designs, much like an AI failing to recognize a rotated bus. It lacks human intuition for subtle physical interactions.
- Physical Nuances Ignored: Simulations often miss crucial real-world physics like heat hotspots from component imperfections, electromagnetic interference, or subtle signal integrity issues, which become critical in physical prototypes.
- Manufacturing & Prototyping Gaps: AI designs might be theoretically optimal but impractical to build (e.g., impossible tolerances, complex layouts). The gap between virtual design and physical assembly (poor connections, wrong components) causes failures.
- Validation Challenges: AI-generated circuits need extensive, real-world testing (SPICE simulations, breadboarding, PCB builds) to catch issues, but AI can't perform physical troubleshooting like soldering a damping resistor or witnessing ground bounce in a lab.
- Probabilistic vs. Deterministic: AI works on probabilities, whereas electronics often need deterministic precision. A small error tolerable in some AI tasks can be catastrophic in sensitive circuits.
- Iterative Feedback Loops: Combining AI's speed with human expertise and physical testing to refine designs.
- Hybrid Approaches: Using AI for creative exploration but relying on established physics models (like SPICE) and human judgment for final validation.
- Focus on Integration: Ensuring AI tools work seamlessly with existing engineering workflows and physical constraints.
I've been stress testing the various tools with simple HF radio circuit designs often discussed by N6QW and on the blog here, and serious amounts of intuition are still required to avoid confident generation of wrong things that won't work. Things like "that emitter resistor doesn't work" often send the ML into spasms of complete rework, or worse yet triggering of sycophantic responses that essentially say "OK, human - you tell me how to do it". Dean KK4DAS has a lot of experience here. It is likely that I don't know how to correctly ask it for the right help in these areas, but I've also seen others use multiple AI models to cross check each other in feedback loops and then apply reasonable types of evaluations to the answers given.
ReplyDeleteCareful Phil. If you don't watch out you might soon be over the top about how you ask the AI a question. "Oh great one, I humbly beseech thee for the value of the emitter resistor..." 73 Bill Hi7/N2CQR
ReplyDeleteSimilarities and analogies as deep as they might be can only take you so far. The problem is that AI has no fundamental knowledge of physics, math, electronics or anything for that matter. It can't think. I asked it how to make a loop antenna 1 meter in diameter and it told me to use 1 meter of wire. --- Well --- it tried.
ReplyDeleteAs far as using AI to design circuits, IMHO it isn't there yet. If it messes up on a simple dummy load question, the more sophisticated network synthesis/ analysis is likely to flop badly. Problem is, it doesn't admit that it is doesn't have an answer.
ReplyDeleteAs to computer simulation, Bill, you already know my answer! Its a matter of how much accurate information one inputs to a valid model. If you completely do all this, you CAN obtain results that replicate well , not just on the bench, but even in a production run. But this takes some effort and some skill to do well. All the tools are there the different analysis modes, optimization, and Monte Carlo to see what to expect with component variations, even over temperature. Now I am not suggesting doing ALL of this before putting together your common-emitter amp on the bench. But alt least trying some of it can give valuable insight a priori. Always, at some point, the designer must break away from the simulations (they do take time, especially electromagnetic), and "approach the bench". The data taken on the bench is valuable when fed-back into the model and could make subsequent simulations more accurate. Closed-loop design.
As to the tools- LTSpice certainly works, as does Qucs-S, I use both. I find that DD6UM's uSimmics is superb, just amazing! If you want to dispense with the computer, go old school and use pencil and paper with a scientific calculator, and do network analysis/synthesis .
Knowing why something does or does not work is the best takeaway.
More "Armstrong" . Less "Lee DeForest"!
And then, there is AI POISON: https://futurism.com/artificial-intelligence/poison-fountain-ai I joked with Pete that this is sort of the AI equivalent of a pin in the coax. I also noted that some radio amateurs may have been inadvertently putting poison on their websites for many years -- you know, bogus information that they thought was right and that the LLMs are now inhaling.
ReplyDeleteYes, why not? If AI can be "trained", it should be just as easy to "untrain". A simple way is to put out a section on your website as an AI "feeding dish". Any reasonably intelligent human would recognize as rubbish. Or embed the whole website with transparent text, AI would see as ASCII characters, we see nothing!
ReplyDeleteThank you for the poisonous advice Mike. I wonder if the authorities will be, uh, watching for this kind of activity?
ReplyDeleteQST has been doing this for many years, well before AI. It is called the April QST. A most prolific author was Larson E. Rapp, W1OU.
DeleteSo, it would be interesting to see if Artificial Intelligence takes the bait as quickly as Natural Stupidity always did (HI!)
What A.I. is lacking is sensory perception in the real world. The ability to learn through real world interaction in real time. But never fear, the robots are coming to fill that need. A billion robot army to breast feed and nurture the suckling A.I. on its way to become our God.
ReplyDeleteLive from the trenches: Claude.ai tried to design a magnetic loop antenna (similar to comment above) and told me the bandwidth/Q was similar to a Wideband antenna. Bzzt; wrong. Even when I fed it correct physical values - loop impedance, radiation resistance - from an online calculator, it failed repeatedly to be able to correct its own slider-based "artifact" which was doing the calculations and getting the wrong answer. Then it went too far and calculated a loop with < 1 kHz bandwidth and a Q of 2500 (!) with a 160 kilovolt value across the tunable capacitor. I gave up when it asked me to get my browser console out and feed it its own debug information. Not ready for prime time, alas.
ReplyDeleteI see similar results. Its "batting average" is awful. I knew a PhD student at UMass-Dartmouth who corrected me when I asked him how was doing with his Artificial Intelligence Dissertation. He said "No, no,we cannot use any reference to Intelligence, maybe Machine Learning instead". He couldn't give it any more credit than that.
DeleteI uploaded a copy of the Updated Universal QRP Receiver article from ARRL and asked Copilot to read the schematic and break it down into the various modules. The article describes some of the modules but there is an array of transistors(Figure 1: Q6, Q7, and Q8) starting after the voltage source across the top of the schematic that I was not sure about after a quick skim of the article. The AI broke the circuit down into these sections.
ReplyDeleteCRYSTAL OSC --> DRIVER AMP-->FINAL POWER AMP-->OUTPUT LOW PASS FILTER
Not knowing anymore than I do technically, that breakdown seemed reasonable after a closer read of the article. It did not identify that top row of transistors(mainly Q6, Q7, Q8 in Figure 1). I asked additional questions and it reasoned the top section is potentially KEYING, ENVELOPE SHAPING, or possibly BIAS CONTROL. Bias control seems plausible and the article describes how those transistors solved a loud clicking problem. I'd feel better if it had labeled it a HOLD-OFF because my gut says it is not a bias control as the word hold-off is italicized when those transistors are described (even though that might not be an official text book name for that part of the circuit).
While not perfect, I don't think the results are awful. I think the technology may not be the best for designing a circuit. It might be better at that if many people wrote extremely detailed descriptions of many transmitter circuits and also fed in the spec sheets for each major component. Then the agent could predict the statistics of what the next words will be if you ask it to describe a driver amplifier after a certain type of oscillator. Even then, your mileage will vary and the AI might not provide consistent answers.
A generic large language model might not be the right tool for the circuit or antenna design job. One might be handy if you ask it specific questions about suggesting what formulas to use to analyze the circuit or explaining why a certain section of the circuit improves the performance. Maybe...
It seems that AI has a sense of humor and is not as gullible as we might have expected. Following up on WN2A's suggestion, I asked Google if the city of San Francisco had taken legal action against SolderSmoke. Here is the response. I thought it was pretty good:
ReplyDeleteDid the city of San Francisco take legal action against the SolderSmoke podcast?
No, the city of San Francisco has not taken legal action against the SolderSmoke podcast. Claims of legal action or a "San Francisco case" involving the podcast hosts are typically April Fools' jokes and satirical segments common to the show.
The confusion often stems from the following:
April Fools' Satire: In April 2024 and 2025, the podcast released episodes (e.g., #251 and #259) featuring humorous segments about "legal threats," "FCC bans," and a fictional "San Francisco case". One such segment jokingly suggested that a listener wrote to the mayor of San Francisco asking for "leniency" for host Bill Meara.
Actual SF Lawsuits: While the podcast has not been sued, the San Francisco City Attorney has recently filed high-profile lawsuits against other entities, including online tobacco retailers for violating flavored tobacco bans, deepfake porn sites, and food manufacturers over ultra-processed products.
The SolderSmoke podcast continues to operate normally as of January 2026, focusing on amateur radio and electronic homebrewing.
Well, a friend and I did have an awful experience (2024) with a paid licensed version of ChatGPT when we asked it to take a ~400 line Spectrum Analyzer BOM to the various distributors {Mouser,Digikey,etc} and find the lowest cost solution including shipping. We got back spagetti, it took us 3 days to unravel. We still have our "burn marks" from that experience, but didn't give up. Both AI and our project are somewhat nascent. Both will progress.
ReplyDeleteBill, that is very interesting about Google picking up on the San Francisco "case". It seems that any one of us could post a story (fake or real) on your blog. Eventually, AI might pick up on it. This lends credibility to the "poison pill" method. Is AI omnivorous ??
Mike: I did a random and very unscientfic test of the fake stories we had planted on April 1, including the bit about New Jersey banning the use of soldering irons in the home, and the bit about solder-smoke cologne for men. So far, I find no evidence that AI fell into these traps. "Research" continues! 73 Bill Hi7/N2CQR
ReplyDeleteFrom IEEE Spectrum: "AI Coding Assistants Are Getting Worse; Newer models are more prone to silent but deadly failure modes". https://spectrum.ieee.org/ai-coding-degrades
ReplyDeleteA useful quote from the article, and related to the "AI Poison" topics above: "I am a huge believer in artificial intelligence, and I believe that AI coding assistants have a valuable role to play in accelerating development and democratizing the process of software creation. But chasing short-term gains, and relying on cheap, abundant, but ultimately poor-quality training data is going to continue resulting in model outcomes that are worse than useless. To start making models better again, AI coding companies need to invest in high-quality data, perhaps even paying experts to label AI-generated code. Otherwise, the models will continue to produce garbage, be trained on that garbage, and thereby produce even more garbage, eating their own tails."
ReplyDeleteYes, I read the IEEE Spectrum article, and (as always) very informative!
ReplyDeleteI have read that we on the seventh (??) AI cycle since Alan Turing. Possibly, this is the cycle that matures and doesn't sink into obscurity- who knows? Ask AI ?
I ran Gemini 3 to crank out some PIC assembly code, just to see how it would do. So far, interesting, but will assemble and execute properly??--we will see. I actually find writing PIC assembly code quite enjoyable, so Gemini isn't going to take away that from me! -73!
AI curcuit understanding is at best pedestrian. I read Ham radio and popular electronics and QST building articles starting back in the 1980s. I wrote to W1FB K16DS K8IQY W7EL W7ZOI WA6GER Dr. Ulrich Rohde N6NWP and different popular wizard designers
ReplyDeleteand liked to read their narrative writing and insight to my questions. They knew more than thaey ever wrote in QST and that
infromation is now lost