Please ensure Javascript is enabled for purposes of website accessibility

GPT-4 Answers Tax Questions, Gets Them Mostly Right

Posted on by Going Concern News Desk
screenshot of Ask an AI Accountant answering a tax question about gambling winnings

These days there is a lot of yapping about what AI can and can’t do; it CAN answer bar exam questions so well it’s nearly in the 90th percentile of test-takers, it CAN’T search in real time to find up-to-date answers beyond what it’s been trained on so it won’t be getting stuck in a late-night Wikipedia hole any time soon. Today we’ve learned there’s one more can: it CAN answer tax questions and be broadly correct 84.19% of the time.

The folks at tax app Keeper trained GPT-4 on 2023 tax updates and then set the public loose on it, inviting them to ask their burning tax questions. From there, actual human professionals fact-checked the answers. For example:

screenshot of Ask an AI Accountant tool

One of the reviewers was Isaiah McCoy, a CPA working in Miami. Going into it he tempered his expectations and thought the tool might hit 60/40 right/wrong or even 50/50 just because tax law is so nuanced. “It far exceeded my expectations,” he said. “Its success rate was more like 80/20 or 90/10. I think it did a great job overall, really blew me away.” As for the prospect of getting replaced by AI, he says he feels moderately safe. “I definitely feel like it’s a threat or an opportunity depending on how you look at it,” he said.

By evaluating 215 answers, Keeper learned that the AI does a good job of:

  • Knowing which tax forms apply to which situation
  • Understanding real estate tax implications
  • Naming deductible business expenses for specific industries

And it sucks at:

  • Getting tax form details right, including what certain lines and boxes are intended for
  • Answering questions about tax year applicability
  • Addressing state and local tax nuances

Here’s one it bombed at:

Screenshot of Ask an AI Accountant tax tool

It’s wild how it matched the casual tone of the original question in its response with a “Hey there!”

Feel free to give it a spin yourself, just remember not to input any PII. And don’t threaten it, you don’t want to be on its shit list when the inevitable takeover happens.

Related Posts

California Attorney General Not Amused By ‘Tax Lady’ Roni Deutch Destroying Records, Using Client Refunds to Pay Debts to a NASCAR Team (Allegedly)

If you’re an insomniac and adverse to softcore porn, you’ve probably seen Roni Deutch at some point, talking tax relief for those oppressed by the IRS to the point of it being a hate crime.

Last August, former California Attorney General and current Governor Jerry Brown sued Ron for a “heartless scheme,” of ripping off those people that needed her help settling disputes with the IRS. RD disputed the charges, continues to tout her expertise and is still issuing lame press releases that gives her some appearance of still being in the game.

As annoying as that probably is, the AG really got bent out of shape when it discovered that Deutch was destroying records and using client refunds to pay off some debts.

Harris’ office said Deutch has systematically destroyed documents for months and may have shredded up to 2.7 million pages of records.

The Attorney General said Deutch had been spending $3 million a year on advertising, mainly late at night on cable TV and that only one in 10 clients received any benefit from working with her firm.

The state also said Deutch was supposed to pay $435,000 in refunds by January, but instead released the money to other creditors, including family and friends, a NASCAR racing team and a casino.

Since these types of actions are typically frowned upon, current California AG Kamala Harris has asked a judge to find the Tax Lady to be in contempt of court and be given a free five-day stay at a local prison for each violation. Not sure how that math will work out but that could amount to a lot of NASCAR being watched on prison TV.

“Tax Lady” Roni Deutch has problems of her own [Reuters]

Making Work Pay Tax Credit May Make Taxpayers Pay

Filed under: ironic press releases from the Treasury that we love to get.

News from our favorite federal taxation authority this morning reveals that while the IRS believes they did everything they were supposed to, some taxpayers may have taken their Making Work Pay credits incorrectly, causing them to actually owe money instead of celebrating free money. Oops! The Treasury Inspector General did their best to warn everyone this could happen and, oh look, it did.

Overall, the Internal Revenue Service (IRS) implemented the Making Work Pay Credit as intended by Congress, according to a report publicly released today by the Treasury Inspector General for Tax Administration (TIGTA).

However, the report also found that approximately 13.4 million taxpayers who received the credit may owe taxes because adjustments to the withholding tables did not take into consideration all taxpayer circumstances. For example, single taxpayers with more than one job, joint filers where both spouses work or one or both of them have more than one job, taxpayers who receive pension payments, and Social Security recipients who receive wages are among those who may be negatively affected.

The Making Work Pay credit is an economic stimulus provision of the American Recovery and Reinvestment Act of 2009 (Recovery Act). The credit is advanced to taxpayers by their employers through withholding reductions which results in an increase in taxpayers’ take home pay. The credit is effective for Tax Years 2009 and 2010.

“The Making Work Pay Credit is a key tax credit designed to increase spending and stimulate the economy,” said J. Russell George, the Treasury Inspector General for Tax Administration. “However, many taxpayers who are accustomed to receiving refunds when they file their tax returns may have owed taxes and incurred penalties in 2009 and may yet again in 2010 because they were advanced more of the credit than they were entitled to claim,” Mr. George added. “My office issued a report in November 2009 warning of this possibility and encouraging the IRS to increase outreach and waive penalties for taxpayers who may be negatively affected by the credit. We still believe further actions are needed to ensure no taxpayer is unfairly penalized.”

The November 2009 report warning this could go down mentions that some taxpayers were proactive and adjusted their withholding so as not to be impacted by the potential “free money” presented by this “credit” which, for some taxpayers, will turn into money owed back to the Treasury or even tax penalties.

The credit was advanced to taxpayers by their employers through withholding reductions that result in an increase in take home pay, in the hopes that $400 ($800 for joint filers) more in each eligible taxpayer pocket might help increase spending and stimulate the economy. Because of the nature of the credit, however, some taxpayers may have had their taxes underwithheld at the end of the year.

Intended to stimulate whose economy?