These days there is a lot of yapping about what AI can and can’t do; it CAN answer bar exam questions so well it’s nearly in the 90th percentile of test-takers, it CAN’T search in real time to find up-to-date answers beyond what it’s been trained on so it won’t be getting stuck in a late-night Wikipedia hole any time soon. Today we’ve learned there’s one more can: it CAN answer tax questions and be broadly correct 84.19% of the time.
The folks at tax app Keeper trained GPT-4 on 2023 tax updates and then set the public loose on it, inviting them to ask their burning tax questions. From there, actual human professionals fact-checked the answers. For example:
One of the reviewers was Isaiah McCoy, a CPA working in Miami. Going into it he tempered his expectations and thought the tool might hit 60/40 right/wrong or even 50/50 just because tax law is so nuanced. “It far exceeded my expectations,” he said. “Its success rate was more like 80/20 or 90/10. I think it did a great job overall, really blew me away.” As for the prospect of getting replaced by AI, he says he feels moderately safe. “I definitely feel like it’s a threat or an opportunity depending on how you look at it,” he said.
By evaluating 215 answers, Keeper learned that the AI does a good job of:
- Knowing which tax forms apply to which situation
- Understanding real estate tax implications
- Naming deductible business expenses for specific industries
And it sucks at:
- Getting tax form details right, including what certain lines and boxes are intended for
- Answering questions about tax year applicability
- Addressing state and local tax nuances
Here’s one it bombed at:
It’s wild how it matched the casual tone of the original question in its response with a “Hey there!”
Feel free to give it a spin yourself, just remember not to input any PII. And don’t threaten it, you don’t want to be on its shit list when the inevitable takeover happens.
[A McClatchy-Marist] poll reported that roughly two out of three registered voters — 64 percent — would be in favor of increasing taxes on annual income over $250,000. President Obama reiterated in his deficit-reduction speech last week that he favored allowing taxes to rise on families in that income level. Independents favored that plan of action at roughly the same percentage as the country at large, with more than eight in 10 Democrats also behind the idea. A majority of Republicans, 54 percent, opposed it. The poll was conducted both before and after Obama’s Wednesday speech, with support for higher taxes on wealthier Americans picking up afterward. Meanwhile, fully four in five registered voters oppose cutting Medicare and Medicaid. The House GOP’s fiscal 2012 budget, largely crafted by Rep. Paul Ryan (R-Wis.), makes fundamental long-term changes to both health entitlement programs, converting Medicaid into a block grant and turning Medicare into a type of voucher system. [
That is,