A quote form, a client portal, a planning tool: with AI coding tools it stands in days instead of months. That is real progress and we advise nobody to go back. But we look under the hood of these applications for a living and the same picture keeps coming out. The software does what the demo promises. The problems sit in everything the demo does not show.
What are the risks of software built with AI?
Software built with AI usually runs flawlessly in the demo and fails on what you do not see: missing input validation, APIs open to anyone who knows the address, personal data leaking, GDPR duties covered nowhere and business logic running in the browser instead of on the server. NYU researchers found as early as 2021 that roughly forty percent of AI-generated code contained vulnerabilities. Each of these gaps can be established and closed, preferably before go-live.
Everyone builds their own now and that is no bad thing
The barrier to making software has collapsed in two years. What used to take a development team and half a year, a handy employee now does in a few weeks. For an SME that is an opportunity: you know your own operation better than any external builder and you can finally make tools that fit your process exactly.
The problem is not the building. The problem is that the tools optimise for working code and not for safe code, and that the builder usually lacks the background to see the difference. The app works, everyone is happy and nobody asks the question a professional developer asks first: what happens when someone uses this wrongly?
The five gaps that keep coming back
These are the five patterns we encounter most in reviews of self-built applications. Each is invisible in a demo and each is painful after go-live.
1. Input is not checked
A form expects a name and an email address. What happens when someone pastes in a piece of code, a 2 gigabyte file or a text of ten thousand characters? Without input validation all of it lands unfiltered in your database or in the mails to your team. That starts with junk in your data and ends with an attacker reading your database through an input field.
2. The API is open to anyone who knows the address
Behind every app sits a layer that fetches and writes the data. AI tools often set up that layer without checking who is knocking. Whoever knows the address or simply guesses it can then request orders, customer data or prices directly without ever logging in. And the address sits in plain sight in the code every browser receives.
3. Personal data leaks through the side door
Error messages that show the entire customer record, export buttons without permission checks, log files full of email addresses: self-built apps rarely leak personal data through the front door and almost always through a forgotten side door. One leak is enough for a notification to the data protection authority and an awkward conversation with every affected customer.
4. GDPR is covered nowhere
Who processes the data, where does it live, how long is it kept and on what ground? An AI tool does not ask those questions and neither, usually, does the builder. Suddenly an app is sending customer data to an American service without a processing agreement, without a mention in the privacy statement and without anyone realising.
5. The business logic runs in the browser
The classic from configurators and quote tools: the price calculation sits in the code sent to the visitor. Anyone with a browser can read and change that code. At best a competitor reads off your discount structure. At worst someone orders at a price they typed in themselves and you have a legal problem, because the order is confirmed.
Why the demo tells you nothing
A demo shows the expected path: correct input, a well-meaning user and no attacker at all. Precisely the three assumptions that die after go-live. Real users do unexpected things, bots scan every public URL within days and forms receive their first automated junk within the week. The question is not whether it goes wrong but whether anyone notices when it does, and in self-built apps that is rarely arranged: logging and alerts are almost always missing.
What you can check yourself today
Four checks that require no technical background. Fail one and you know there is more going on.
One: enter something wrong in every field. Text where a number belongs, an empty required field, an absurdly long name. Do you get tidy error messages or does the app act strangely? Two: open the app in an incognito window and paste the address of a page that should sit behind the login. If you still see data, the door is open. Three: ask who holds the processing agreements with every external service the app uses. Silence is an answer. Four: ask the builder where the price calculation runs. If the answer is "in the app itself", ask whether that means the server or the browser.
Have it tested before you go live
Building yourself and testing professionally do not exclude each other; they belong together. We carry out software audits on exactly this kind of application: six control domains, from security and GDPR to functional test scenarios and continuity, with a report that marks per finding what is critical and what can wait. Your team simply builds on afterwards, but with the gaps closed.
Built it yourself and almost ready for go-live?
The SEMANU software audit examines your application on the six domains from this article and delivers a priority list plus a working session with your team. Fixed approach, from 1,850 euros.
Frequently asked questions
Is software built with AI unsafe?
Not by definition, but the odds of gaps are higher. NYU researchers found as early as 2021 that roughly forty percent of AI-generated code contained vulnerabilities. The tools optimise for working code and not for safe code.
Should I stop building with AI then?
No, and for many internal tools it is the fastest route. The point is that an app touching customers, money or personal data gets tested before it goes live, exactly like you have a self-renovated kitchen inspected before connecting the gas.
What exactly does a software audit check?
Six domains: security, personal data and GDPR, functional behaviour with test scenarios, architecture and logic, performance and mobile use, and continuity. The result is a report with a three-level priority list and a working session with your team.
What does such an audit cost and how long does it take?
From 1,850 euros excluding VAT, with a lead time of about one week. For full depth we ask read access to the code under a mutual NDA; without code access a black-box variant exists.