„Software does not follow an audit calendar“

Georg Heise: AI changes the relationship between effort, scope and frequency of testing. AI agents can conduct technical investigations within a shared audit framework, adapt their approach based on the results, and also detect previously unknown vulnerabilities. This provides the ability to use the available resources to investigate more relevant functions, interfaces, and permissions, and to review changes more quickly. However, the extent of this benefit also depends on the quality of the results and the necessary inspection effort. Translating the efficiency gained into a better safety audit is crucial.

Georg Heise: For applications that are constantly being developed further, an annual pentest alone is usually not enough. It provides an important assessment of the condition under review, but no safety evidence for the next 12 months. Software does not follow an audit calendar. Therefore, relevant changes and new threats should trigger additional, targeted reviews. Not every update needs a full pentest; scope and depth must be based on the actual risk.

Georg Heise: For this, we need clear test limits, comprehensible test steps and reliable evidence. A reported vulnerability must be technically verifiable, and it must also be possible to identify the areas that have actually been investigated and where limitations or gaps have remained. Not having found a vulnerability is not proof that a system is secure. Experienced pentesters steer the procedure, review critical results and specifically investigate the areas in which AI is not reliably advancing. Technical responsibility cannot be delegated to a model.

Georg Heise: Our approach is to strengthen experienced pentesters with AI. AI can also take over parts of the actual technical investigation and is not limited to routine preparatory tasks. Pentesters design the testing strategy, examine complex relationships, question results and classify the technical implications for the company. They remain technically active themselves, especially where automated tests reach their limits. The focus is shifting: we need to do fewer individual steps ourselves, but still be responsible for the quality and meaningfulness of the entire audit.

Georg Heise: Yes, and here we have to keep two topics apart: AI as a tool for pen tests and AI systems as the subject of a pen test. In an AI application, we examine the entire system, including data sources, interfaces, permissions, and the actions it is allowed to perform. Manipulated content can be a business risk, especially if it causes an agent to disclose sensitive information or to perform improper actions. We must therefore investigate whether access limits, authorizations and protective measures remain effective even under targeted manipulation. The question is whether the system can be used safely in the intended business process.

Georg Heise: Regulatory requirements and operational safety must not be mutually exclusive. DORA already uses a risk-based approach for the financial companies concerned; it is therefore not just a matter of adhering to fixed audit deadlines. AI-based testing can complement such a program, but it must be appropriate to the respective requirements in terms of scope, quality and demonstrability. They do not automatically replace prescribed test procedures. It is crucial that the audits lead to concrete improvements and that the correction of detected weaknesses is verified in a comprehensible manner.

Georg Heise: I expect pentesting to be more involved in the life cycle of applications. In-depth assessments are supplemented by targeted assessments, for example following relevant changes or new threats. AI will play a greater role in both comprehensive and complementary tests. Efficiency gains can allow more relevant areas to be examined more frequently without making any change a full assessment. It is crucial that we identify risks earlier, support their resolution and review the effectiveness of the measures.