Accuracy

Like any new software product, Direct File was not without bugs. The pilot design enabled

many of these bugs to be addressed before most users ever encountered the product, but

throughout the pilot, taxpayers would occasionally encounter rare glitches, which in some

cases required taxpayers to reset their return and start over, or even prevented them from

filing with Direct File altogether.

However, the category of bug that the IRS spent the most effort to prevent is any bug that

could affect the accuracy of a tax return. That is, a bug that would cause Direct File, despite

14 IRS. IRS Report to Congress: InflaƟon ReducƟon Act §10301(1)(B) IRS-run Direct e-File Tax Return System. Page 15. htps://www.irs.gov/pub/irs-pdf/p5788.pdf

22

being provided with accurate and honest answers by the taxpayer, to nevertheless produce an

inaccurate tax return from those answers. Following modern software development best

practices, the IRS developed a suite of automated tests that were run for each proposed

change to Direct File and before updates could be deployed. The IRS developed a set of more

than 90 tax return scenarios that ran automatically, encompassing the full scope of Direct File,

including every filing status, income source, deduction, credit, form, and schedule. More than

1,600 additional test cases were written to test other aspects of Direct File’s implementation of

tax law.

These testing strategies successfully prevented any tax accuracy regressions, meaning bugs

where Direct File was at one time producing accurate output, but a change to the software

broke the functionality.

However, during the pilot, the team identified four bugs in rare edge cases that were not

covered by tests. Three of these bugs were visible during the public phases of the pilot and

are known to have affected 26 returns (less than 0.02% of the returns filed by Direct File). In

addition to fixing these bugs, the IRS added additional tests to ensure that these issues could

not recur. Affected taxpayers may receive math error notices and a refund of overpayment.

Once data is available, the IRS will review the rates at which Direct File returns were

amended, and the reasons taxpayers had for amending returns. In addition to the taxpayer

frustration caused by needing to amend a return, amended returns are also costly for the IRS

to process. An analysis of amendment rates, and the reasons taxpayers had to file

amendments, could identify further opportunities to improve the accuracy of Direct File,

including in how it guides taxpayers to recognize when they are missing information necessary

to file their return.

Opportunity: Open source

The IRS could take further steps to build public trust and enable independent assessment of

its work. The Direct File product team was given the mandate to develop software that

ensures every taxpayer receives the full benefit of any tax provisions for which they are

eligible. Releasing components of Direct File as open-source software would enable the team

to demonstrate this commitment.

Open-source software refers to code that is publicly available and licensed under terms that

grant others the right to inspect, modify, and reuse that code. Open-sourcing components of

Direct File, and in particular the components that implement and test its tax logic, would serve

a number of purposes. First, it would enable public scrutiny of that code and invite

independent groups to assess its accuracy and report potential issues. Second, other tax

administrators, both in states and internationally, could build upon and contribute to the IRS’s

work, improving the robustness of the software over time and providing additional public value.

Finally, the IRS’s work could serve as a reference for implementation of the Internal Revenue

23

Code as computer code, enabling businesses and others to incorporate the IRS’s code into

their own software and increasing the vibrancy of the tax ecosystem.

Direct File’s source code is not releasable in its current state, as it includes sensitive but

unclassified (SBU) information, such as the implementation of security and fraud protection

measures. However, should Direct File continue in future years, the IRS could prioritize

separating sensitive and non-sensitive components of Direct File to enable open sourcing as

much of the application as possible.