Top 10 Actionable Insights from AI Analysis of PCAOB Datasets
On April 28, 2025, the PCAOB made available downloadable datasets featuring PCAOB inspection findings from audit inspection reports going back to 2018. The PCAOB noted that these datasets were machine-readable, which is a relief because it would be impossible for a human to review PCAOB inspection reports for over 2,000 firms spanning 6+ years! Sounds like a job for ChatGPT (and our new intern, Henry Walworth)! Henry dove right into the project and in no time he had something back to me for my review. “Wow, that was quick,” I thought to myself. “I wonder if it is any good.” Let’s find out together! This post summarizes our process for using ChatGPT to analyze PCAOB datasets of past inspections to reveal the top 10 actionable insights aimed at helping firms improve their audit quality.
Overview of the process to analyze PCAOB inspection report data
The PCAOB inspection report data, which consisted of tens of thousands of cells across three Excel files, was compiled into a single Excel spreadsheet. Once the file was uploaded into OpenAI’s ChatGPT, the AI assistant was prompted to return a list of trends it identified in the data. Unfortunately, there is a limit to the data processing capabilities of the free ChatGPT. As a result, we subsequently purchased ChatGPT Plus version ($20 per month). With its advanced capabilities, the full dataset of tens of thousands of cells was able to be parsed through in seconds.
We prompted ChatGPT several times to generate a list of insights based on the inspection report. Through trial and error, Henry realized that the more specific the prompt, the better the outcome. Generative AI will frustrate you unless it is given specific guidelines. A basic outline was given to the AI assistant as a guide of what GAAP Dynamics wanted in the report:
Top 10 Insights, each including:
- Title
- Description of insight
- Visual
- Explanation of visual
- Actionable recommendation
With this, ChatGPT generated something close to what we wanted, but it still wasn’t good enough to publish. Henry further refined the output within the chat box, and eventually a version was produced that was ready to be revised manually by our team. However, Henry took it a step further.
He created a custom GPT for this specific use case . By providing extremely specific instructions, this process could take seconds instead of needing hours of refinement, allowing GAAP CPAs to create this product in the future without needing to endlessly prompt. In the future, all they need to do is upload an updated Excel file of the inspection report and put it into the custom GPT, which will process the request and deliver a downloadable Word document.
Follow-up: As if OpenAI isn’t making enough money!
As you’ll see below, the data for 2024 is incomplete because all of the 2024 PCAOB inspection reports were not yet released at the time Henry ran his analysis. So, just prior to publishing this blog, I asked him to run it again using custom GPT he made for us. I was hoping a revised analysis would capture some of the 2024 inspection data.
As he was running it, he exclaimed “What the heck?” Turns out to Henry’s custom GPT, which used a python plugin is no longer available. His research indicated that he would have to upgrade to the Teams version. However, this loss in functionality also coincided with the rollout of OpenAI’s GPT-5, which by all accounts was not well orchestrated, with may complaints on social media. It was a nice thought Henry, but you learned a valuable lesson software companies. Just when you learn how to use a new feature, software companies will always try to squeeze you for the upgrade!
Here are the results from running our custom GPT on the PCAOB datasets:
Top 10 actionable insights from AI analysis (with commentary from GAAP Dynamics)
Insight 1: Increasing trend in total audit deficiencies identified in Part I.A
Over the past several inspection cycles, there has been a noticeable increase in the total number of audit deficiencies cited in Part I.A of PCAOB inspection reports. This indicates a growing volume of audit areas where firms failed to obtain sufficient appropriate audit evidence or to perform procedures effectively—raising questions about audit execution under evolving risk conditions.
Actionable recommendation for firms (per ChatGPT)
Audit firms should prioritize a robust root cause analysis for recurring deficiencies and adapt their methodologies to address audit complexity and risk. Enhanced supervision and real-time coaching during audits may also help curtail quality lapses contributing to this trend.
Additional commentary (GAAP Dynamics)
In their Spotlight document providing an update of 2024 inspection activities, the PCAOB noted that firms have worked to improve the quality of their audits and this has resulted in a drop in 2024 deficiency rates compared to 2023. Per the PCAOB, the drivers of improvement by the larger firms include:
- More in-person work
- More focused training
- More resources
- Better supervision and review
Insight 2: Concentration of deficiencies in a few audit areas
A majority of Part I.A deficiencies stem from a limited set of audit areas. Specifically, audit procedures related to revenue recognition, fair value measurements, and internal control testing dominate the findings. This suggests persistent challenges in auditing high-risk and judgmental areas.
Actionable recommendation for firms (per ChatGPT)
Firms should implement specialized training and enhance audit program templates in these high-risk areas. Consider incorporating data analytics and subject matter specialists more routinely in audits involving complex estimates and revenue streams.
Additional commentary (GAAP Dynamics)
This is not a surprise as revenue and related accounts is always the most noted financial statement line item with respect to deficiencies. In their Spotlight document, the PCAOB noted the following examples of deficiencies related auditing revenue:
- Testing disaggregated revenue presentation and disclosures
- Evaluating data used in software-assisted correlation analysis
- Valuation and allocation of bundled sales transactions
- Confirming accounts receivable
- Testing significant assumptions related to percentage of completion
- Performance obligations and ensuring revenue recognized over time is appropriate
- Testing the accuracy and completeness of historical data
- Audit procedures (or lack thereof) related to fraud risk
- Selecting sample sizes
And obviously we love the recommendation for training!
Insight 3: Audit failures concentrated around key auditing standards
A small subset of PCAOB auditing standards accounts for a disproportionate number of Part I.A deficiencies. The most frequently cited standards include AS 2201 (ICFR), AS 2301 (audit risk), and AS 2501 (estimates). These standards often govern complex, judgment-intensive audit procedures.
Actionable recommendation for firms (per ChatGPT)
Firms should reinforce their audit methodology and review programs around these high-risk standards. Emphasizing rigorous planning, documentation, and testing procedures in these areas can help reduce recurring deficiencies.
Additional commentary (GAAP Dynamics)
Again, no surprise here. Focusing on internal controls, risk assessment, and auditing estimates is a wise choice if you want to reduce audit deficiencies. However, I do have a problem with the term “audit failures” in the insight title from ChatGPT. Just because an audit deficiency was noted does not mean the audit opinion was wrong or that the audit was a failure. It simply means that, at the time the firm issued its audit reports, the PCAOB believes that the firm had not obtained sufficient appropriate audit evidence to support its opinion.
Insight 4: Persistent volume of Part I.B findings indicates systemic firm-level weaknesses
The volume of Part I.B findings—typically related to firm-wide policies, procedures, and quality control—has remained consistently high. This highlights enduring weaknesses in audit firm infrastructure, such as independence policies, engagement monitoring, or internal inspections.
Actionable recommendation for firms (per ChatGPT)
Firms should strengthen their system of quality control, with targeted improvements in areas such as auditor independence, engagement reviews, and ongoing training. Embedding continuous improvement feedback loops into quality reviews can help reduce these findings over time.
Additional commentary (GAAP Dynamics)
This is why ChatGPT won’t take my job (at least not yet). Observations related to quality control are noted in Part II of PCAOB inspection reports, not in Part I.B. Part I.B shows deficiencies that do not relate directly to the sufficiency and appropriateness of evidence the firm obtained to support its opinion but nevertheless relate to instances of non-compliance with PCAOB standards or rules. Examples Part I.B deficiencies include critical audit matters (CAMs), audit committee communications, and forms filed with the PCAOB.
That said, I do think firms should strengthen their QC systems and they will be forced to do so when QC 1000 become effective December 15, 2025. Need more information on QC 1000? Check out our Quality Management topic page!
Insight 5: High and persistent average deficiency rates across firms
The average Part I.A deficiency rate across all inspected firms has remained elevated, often near or at 100%. This suggests that most firms, regardless of size or geography, consistently have at least one audit with significant deficiencies identified by the PCAOB.
Actionable recommendation for firms (per ChatGPT)
Firms should reassess their risk assessment frameworks and engagement-level quality controls. Proactive involvement by engagement quality reviewers and centralized risk guidance during planning may help lower the incidence of material audit failures.
Additional commentary (GAAP Dynamics)
I’m not sure ChatGPT’s “fact” that the deficiency rates across all inspected firms is “often near or at 100%.” That makes the auditing profession sound just awful! In our review of annually inspected filers, we noted that 67% of audits reviewed showed no deficiencies. That said, there is a widening gap in deficiency rates between the global network firms (26% in 2024) compared to other annually inspected firms (52%). And the deficiency rate of triennially-inspected, non-affiliated firms outside of the U.S. is a whopping 76% in 2024. We can all agree that the profession must do better.
Insight 6: Strong correlation between risk-based selections and higher deficiency rates
Firms with a higher proportion of audits selected through risk-based criteria consistently exhibit higher Part I.A deficiency rates. This indicates the PCAOB’s selection process is effectively targeting higher-risk engagements.
Actionable recommendation for firms (per ChatGPT)
Firms should proactively identify and internally review high-risk engagements before inspections. Pre-issuance or post-completion reviews by experienced quality reviewers could help mitigate audit failures in engagements that are more likely to be flagged for inspection.
Additional commentary (GAAP Dynamics)
While I have no issue with the recommendation (and actually think that it is a good idea to review high-risk engagements before inspections), I think the insight itself is not really insightful. Like duh. Of course, higher risk engagements will yield more deficiencies!
Insight 7: Internal control failures concentrated in specific areas
Deficiencies related to AS 2201 (auditing internal control over financial reporting) are not evenly distributed. Most are clustered in a few key audit areas—especially revenue, inventory, and accounts receivable—where control design and effectiveness testing frequently fall short.
Actionable recommendation for firms (per ChatGPT)
Audit teams should enhance walkthroughs, control testing precision, and alignment of substantive procedures with control reliance in high-risk areas. More rigorous documentation of control attributes and testing rationale can also reduce PCAOB citations.
Additional commentary (GAAP Dynamics)
Much like our audits of financial statements (as noted above in Insight 2), the main financial statement area of deficiencies in our audits of internal control over financial reporting is revenue. I do agree that audit teams need to enhance their walkthrough procedures and we’re here to help!
Insight 8: Persistent and rising deficiencies in revenue audits
Revenue continues to be one of the most problematic areas in PCAOB inspections. The number of deficiencies cited in revenue audits has not only remained high but shows a rising trend in recent inspection years, signaling sustained execution risks.
Actionable recommendation for firms (per ChatGPT)
Firms should implement more robust audit procedures for revenue recognition, particularly around cutoff testing, significant estimates, and disaggregated disclosures. Early involvement of technical teams and revenue specialists can improve audit precision.
Additional commentary (GAAP Dynamics)
Ok, we get it. Audit teams mess up revenue. This is now the third “insight” from ChatGPT of basically the same thing. I want my $20 back Sam Altman!
Insight 9: Emerging risks evident in growing deficiency areas
Certain audit areas, such as pension assets, inventory valuation, and debt, have shown significant increases in deficiencies over the past four years. These emerging areas reflect evolving risks and highlight where auditors may be underprepared or misaligned with PCAOB expectations.
Actionable recommendation for firms (per ChatGPT)
Firms should proactively review audit approaches in these emerging areas, conduct targeted internal inspections, and update guidance and training to address shifting risk profiles.
Additional commentary (GAAP Dynamics)
Well, now ChatGPT is just mailing it in! Note that the three audit areas noted in ChatGPT’s description are not even found in the chart and their recommendation is just recycled from previous recommendations.
Insight 10: High prevalence of audits with multiple deficiencies
A significant portion of inspected audits with Part I.A findings are classified as having multiple deficiencies. This clustering of issues suggests that problems are often systemic within an audit rather than isolated to a single failure.
Actionable recommendation for firms (per ChatGPT)
Audit firms should strengthen engagement-level oversight, especially for complex or multi-location audits. Early risk alerts and stronger review protocols can help catch cascading issues before they result in multiple deficiencies.
Additional commentary (GAAP Dynamics)
While the formatting of the chart could be much better, the last insight by ChatGPT is actually spot on. Most audits with a deficiency will have multiple deficiencies. And although ChatGPT alluded to it, better EQRs would go a long way to improving audit quality. Also, has you can see on the chart, very few audits inspected required the auditors to change their opinion because it was incorrect.
Closing thoughts
The initial thought of having a custom GPT to instantly analyze thousands and thousands of rows of data from the PCAOB was exciting, but the output didn’t wow me. I’d give it a C- if I were a college professor grading this paper. I would imagine you would feel the same way reading a blog post or taking a training obviously generated by AI, which is why, as long as I am at the helm, GAAP Dynamics won’t do it! You have my word. Will the technology eventually get to a point where we are replaced? Maybe, but it ain’t there yet!
About GAAP Dynamics
We’re a DIFFERENT type of accounting training firm. We view training as an opportunity to empower professionals to make informed decisions at the right time. Whether it’s U.S. GAAP, IFRS, or audit training, we’ve trained thousands of professionals since 2001, including at some of the world’s largest firms. Our promise: Accurate, relevant, engaging, and fun training. Want to know how GAAP Dynamics can help you? Let’s talk!
Disclaimer
This post is for informational purposes only and should not be relied upon as official accounting guidance. While we’ve ensured accuracy as of the publishing date, standards evolve. Please consult a professional for specific advice.
